Pre-training language model retrieval method and system based on knowledge graph enhancement

Abstract

The invention provides a pre-training language model retrieval method and system based on knowledge graph enhancement, and the method comprises the steps: carrying out subgraph division processing on a domain knowledge graph, and generating a subgraph index matched with a user retrieval type; updating the domain knowledge graph and the sub-graph index by adopting an incremental training method of a graph neural network; user query is decomposed into an entity set and a relation path through entity recognition, entity disambiguation, entity linking and relation extraction operation; positioning a corresponding sub-graph index based on the entity set and the relation path, and extracting a related sub-graph from the domain knowledge graph; and respectively coding the head entity, the relationship and the tail entity in the related sub-graph triad into basic vectors, and performing semantic fusion processing on the basic vectors to generate knowledge prompt vectors. The method has the advantages of accurately capturing domain structured association logic and responding to domain dynamic iteration knowledge in real time.

Description

Technical Field

[0001] This invention relates to the field of natural language processing technology, and more specifically, to a method and system for retrieving pre-trained language models based on knowledge graph enhancement. Background Technology

[0002] In real-world scenarios, task requirements are highly diverse and complex. General-purpose language models often struggle to deeply adapt to the specific needs of particular domains and fail to fully capture the inherent connections between domain contexts and information. When handling domain-specific retrieval and generation tasks, these models frequently exhibit "illusion" phenomena due to a lack of targeted knowledge support, directly impacting the accuracy of the output and failing to meet the stringent reliability requirements of professional fields such as electronics, healthcare, and finance.

[0003] To address this issue, the industry has proposed supplementing large language models with structured domain knowledge using knowledge graphs. For example, Yu Jizhen, in her graduate thesis "Research on Enhanced Generation Technology for Large Model Retrieval Based on Knowledge Graphs" submitted on May 31, 2024, disclosed a specific technical implementation path: at the level of knowledge graph and large model fusion, by extracting sub-data from the knowledge graph... Figure 3 Tuples are passed to the large model as additional input or contextual information, or triples are used after answer generation to enrich content and verify accuracy, thereby enhancing retrieval. In the medical field, the Neo4j graph database is used to store the medical knowledge graph. Entity recognition is performed on the input question using NLTK, and the identified keywords are transmitted to the knowledge graph. Keywords related to the query question are retrieved from the knowledge graph. A subgraph retrieval method using Cypher queries is used to obtain a knowledge subgraph related to the query question. Finally, the enhanced query text guides the large model to generate a response that is more consistent with medical knowledge.

[0004] While the aforementioned technological research has effectively alleviated the "illusion" problem of large models and improved the quality of responses in specialized domains, the following technical bottlenecks still exist: Although knowledge graphs can provide rich structured knowledge, data updates usually require retraining the entire graph neural network, resulting in a static knowledge coverage problem. Specialized domain knowledge is constantly iterating; if the knowledge graph fails to keep up with updates, large models will continuously call outdated information, leading to outputs that do not conform to current domain understanding, severely limiting the application effectiveness of this technology in scenarios requiring real-time knowledge updates.

[0005] In order to solve the above problems, people have been seeking an ideal technological solution. Summary of the Invention

[0006] The purpose of this invention is to address the shortcomings of existing technologies by providing a knowledge graph-enhanced pre-trained language model retrieval method and system that accurately captures the structured relational logic of a domain and responds in real time to the dynamic iterative knowledge of the domain.

[0007] To achieve the above objectives, the technical solution adopted by the present invention is as follows: In a first aspect, the present invention provides a retrieval method based on a pre-trained language model enhanced by knowledge graphs, comprising: The domain knowledge graph is partitioned into subgraphs to generate subgraph indexes that match the user's search type. The domain knowledge graph and the subgraph indexes are updated using an incremental training method of graph neural networks. The user search type includes queries containing semantically similar words, queries containing explicit relationship chains, and queries combining multiple conditions. Through entity recognition, entity disambiguation, entity linking, and relationship extraction operations, user queries are decomposed into entity sets and relationship paths; based on the entity sets and relationship paths, the corresponding subgraph indexes are located, and relevant subgraphs are extracted from the domain knowledge graph; The related sub Figure 3 The head entity, relation, and tail entity in the tuple are encoded as basic vectors. Semantic fusion processing is performed on the basic vectors to generate knowledge hint vectors. The knowledge hint vectors are then concatenated into the input hints of the pre-trained language model to guide the pre-trained language model to generate retrieval results.

[0008] It adapts to diverse user search types to achieve precise subgraph index construction, and combines incremental training of graph neural networks to complete dynamic updates of the graph and index, significantly improving the retrieval response efficiency and data timeliness of domain knowledge graphs; through entity recognition, disambiguation, link and relation extraction, it completes the structured parsing of user queries, achieving precise positioning of subgraph indexes and efficient extraction of target subgraphs, effectively solving the technical problems of entity ambiguity and matching deviation during the retrieval process; and it will further enhance the understanding of subgraphs. Figure 3 The tuple element encoding is fused into a knowledge hint vector and integrated into the input of the pre-trained language model, realizing a deep combination of graph structured knowledge and the semantic understanding ability of the pre-trained language model. This significantly improves the accuracy, completeness and semantic fit of the retrieval results, and enhances the model's ability to handle complex domain queries.

[0009] In a preferred embodiment, the incremental training method using graph neural networks to update the domain knowledge graph and the subgraph index includes: The method identifies the types of changes to a domain knowledge graph, analyzes the propagation range of the changed nodes using a graph diffusion algorithm, and determines the local subgraphs affected by the changes. The types of changes include adding nodes or edges, modifying entity attributes, and deleting relationships. The graph diffusion algorithm includes random walk algorithms and neighborhood sampling algorithms. The parameters of the graph neural network are divided into global parameters and local parameters; the global parameters are used to capture general relational patterns in the domain and are fixed after initial training; the local parameters are used to fine-tune the local subgraphs affected by the changes. The gradient of neurons corresponding to the affected subgraph is calculated based on the incremental training method, and the gradient mask is used to shield the updates of irrelevant parameters. During the training process, elastic weights are introduced to consolidate the regularization term to protect important parameters obtained from historical training from being overwritten by new data. Update the subgraph index affected by the change based on the local parameters trained using the incremental training method.

[0010] By using the graph diffusion algorithm, the local subgraph regions affected by changes in the knowledge graph are accurately located, enabling precise focusing of incremental updates and avoiding redundant calculations in full graph updates, thus significantly improving index update efficiency. The graph neural network parameters are divided into global and local parameters. Fixed global parameters ensure the stability of general knowledge in the domain, while only fine-tuning local parameters to adapt to changed regions, balancing the continuity of knowledge with the flexibility of updates. The model parameters are gradually updated to adapt to new data through incremental training. Gradient masking further shields irrelevant parameter updates, reducing the computational cost of incremental training. Meanwhile, elastic weights consolidate regularization terms to effectively protect historically important parameters, ensuring the performance stability of the retrieval system after knowledge updates. The subgraph index of affected areas is updated in a targeted manner to ensure the consistency between the index and the knowledge graph, while reducing the impact of the update process on the normal retrieval service of the system and improving the availability of the system.

[0011] In a preferred embodiment, the method further includes fusing path weights in the self-attention calculation of the Transformer layer of the pre-trained language model to enhance multi-hop inference path attention during the generation of retrieval results by the pre-trained language model, including: Filter out multi-hop relationship paths containing entity association logic from the relevant subgraphs; Each relation in the multi-hop relation path is vectorized and encoded, and the weight of each relation is calculated by combining the path length, relation confidence and the degree of entity association, to generate a sparse path weight matrix; The path weight coefficients are dynamically preset according to the query type. During the self-attention calculation process of the Transformer layer of the pre-trained language model, the path weight coefficients and the path weight matrix are fused to obtain the path-enhanced attention output. The formulas for calculating the weights of each relation are as follows: ; in, P ij This represents the path weight between the i-th query term and the j-th document term. r kFor the k-th relation in the path, α k For learnable parameters, f (r k ) represents the relation type weight; The calculation expression for the path-enhanced attention output is: ; Where Q is the query vector, K is the key vector, V is the value vector; λ is the path weight coefficient; P is the path weight matrix; and d is the vector dimension.

[0012] By filtering multi-hop relationship paths containing entity association logic, core knowledge association information is accurately extracted, providing high-quality knowledge basis for attention fusion and avoiding interference from irrelevant paths to attention calculation. Relationship weights are calculated by combining multiple dimensions such as path length and relationship confidence, generating a sparse path weight matrix. This can accurately quantify the importance of each relationship in query and document matching, while the sparse matrix reduces computational overhead.

[0013] By pre-setting path weight coefficients based on query type, the attention fusion strategy can adapt to different types of retrieval needs, such as fact queries and related queries, thereby improving the flexibility and targeting of retrieval. In the self-attention calculation of the Transformer layer, the path-enhanced attention output after fusing path weights achieves deep coupling between domain knowledge and the attention mechanism of the language model, enabling the model to focus on the structured knowledge provided by the knowledge graph and improving the accuracy of query and document matching.

[0014] In a preferred technical solution, a dual-channel scoring and retrieval result ranking step is also included, which includes: Obtain the text semantic matching score and knowledge graph path confidence score output by the pre-trained language model; wherein, the text semantic matching score is generated based on the contextual semantic understanding of the query and candidate content by the pre-trained language model; the path confidence score is calculated based on the degree centrality, relation frequency and topological compactness of the nodes retrieved in the knowledge graph; A learnable weighted function is used as the dual-channel scoring function. The weighted function adopts a gated network or a linear interpolation structure. The weight allocation strategy of text semantic matching score and graph path confidence score is optimized through training iteration to adapt to the retrieval scenario requirements of different fields. The text semantic matching score and the graph path confidence score are input into the dual-channel scoring function, and the weights of the two types of scores are dynamically allocated to obtain a comprehensive ranking score. The search results are sorted and output based on the comprehensive ranking score, and the corresponding graph evidence chains are labeled.

[0015] A dual-channel scoring system combining text semantics and knowledge graphs is constructed. By leveraging the contextual understanding capabilities of language models and the structured association advantages of knowledge graphs, the limitations of a single scoring dimension are overcome, improving the comprehensiveness and accuracy of retrieval result evaluation. A learnable weighting function is used to dynamically allocate weights, combined with the flexibility of gating networks or linear interpolation structures. Through iterative training, the system adapts to different domain scenarios, making the scoring strategy more universal and adaptable.

[0016] When a clear triplet path with strong relevance to the query exists in the knowledge graph, even if the text semantic matching score is low, the ranking of the corresponding search result is still improved by the graph path confidence score. At the same time, the graph evidence chain is labeled so that users can clearly understand the knowledge basis of the search results, which enhances the credibility of the system. The output results based on the comprehensive ranking score ensure the rationality of the search results ranking, so that high-quality results are presented first, which improves the user search experience.

[0017] In a preferred embodiment, concatenating the knowledge hint vector to the input hint of the pre-trained language model to guide the pre-trained language model in generating retrieval results includes: For the relevant sub- Figure 3 The head entity, relation, and tail entity in the tuple are embedded to generate their respective basic vectors; The basic vectors are semantically fused using a relational attention mechanism to generate knowledge hint vectors, which are used to strengthen the guiding role of relations in entity association. The knowledge hint vector is concatenated with the user's query input sequence and used as additional contextual information to input into the pre-trained language model.

[0018] By embedding each component of the triple into a separate representation, the accurate capture of semantic information of entities and relations is ensured. The introduction of a relational attention mechanism to achieve semantic fusion of the basic vectors highlights the core role of relations in entity association, making the generated knowledge hint vector more accurately reflect the structured semantic information of the triple and improving the efficiency of knowledge transfer to the model.

[0019] Knowledge hint vectors are concatenated as additional context into the user query sequence, enabling the pre-trained language model to simultaneously perceive domain-structured knowledge when processing queries. This effectively alleviates the model's misunderstanding of domain terminology and entity relationships, and improves the relevance and accuracy of retrieval results.

[0020] In a preferred embodiment, the method further includes pruning the subgraph, wherein the pruning includes: Preset domain-specific meta-path templates and predefine key meta-path patterns; when a new relation type is added to the knowledge graph, the corresponding derived meta-path is automatically generated; For each node and edge in the knowledge graph, calculate its information contribution entropy in participating in the meta-path to quantify the importance of the node and edge; The knowledge graph is stored in segments according to entity type and managed using a distributed graph database. Within each segment, predefined meta-paths are traversed in parallel to count the path frequency and the information contribution entropy value of the nodes. Based on the maximum allowable subgraph size of the GPU memory capacity, set retention thresholds for nodes and edges; sort nodes and edges in ascending order of information contribution entropy, prioritize retaining low-entropy nodes and edges, and prune high-entropy nodes and edges; if pruning causes key element paths to break, supplement with minimum connecting edges to ensure the connectivity of the subgraph. New streaming data from the knowledge graph is periodically accumulated into micro-batches. The information contribution entropy is recalculated and pruning is performed only on the subgraph regions affected by the new data, thus avoiding full graph pruning updates.

[0021] The critical path pattern is defined based on domain-specific meta-path templates, ensuring that the pruning process retains the core knowledge associations of the domain and avoids the loss of core semantic information. When adding new relations, derivative meta-paths are automatically generated, improving the adaptability of the pruning strategy to the dynamic changes of the knowledge graph. The importance of nodes and edges is quantified by information contribution entropy, giving the pruning operation a clear quantitative basis and achieving precise pruning, which removes redundant information while ensuring the core knowledge value of the subgraph.

[0022] The combination of entity type sharding storage and distributed management with parallel traversal significantly improves the efficiency of meta-path traversal and entropy calculation, adapting to the processing needs of large-scale domain knowledge graphs. Dynamically setting the subgraph size based on GPU memory capacity ensures that the pruned subgraph can fit the model's computing resources, avoiding computational bottlenecks caused by excessively large factor graphs and improving model inference efficiency. Supplementing with minimum connecting edges ensures subgraph connectivity, preventing core knowledge path breaks from affecting retrieval accuracy. Micro-batch incremental pruning avoids redundant computations in full graph updates, reducing the computational cost and time overhead of pruning updates and ensuring system real-time performance.

[0023] Secondly, this invention provides a pre-trained language model retrieval system based on knowledge graph enhancement, applying the pre-trained language model retrieval method based on knowledge graph enhancement as described in any of the first aspects, including an offline processing module and an online retrieval module; the offline processing module is used for dynamic construction of domain knowledge graphs, generation of lightweight subgraph indexes, and incremental optimization, providing efficient data support for online retrieval; the online retrieval module is used to output accurate and interpretable retrieval results through structured query parsing, knowledge-enhanced retrieval, and multimodal fusion sorting.

[0024] The system decouples offline data processing from online retrieval tasks, ensuring the stability of large-scale graph construction in the offline phase while improving the response speed of online retrieval, thus adapting to high-concurrency retrieval scenarios. Based on the knowledge graph-enhanced pre-trained language model retrieval method described in the first aspect, it addresses issues such as static knowledge, weak multi-hop reasoning, and uninterpretable results, significantly improving retrieval performance and domain adaptability. Module parameters and strategies can be flexibly adjusted according to the needs of different vertical domains, reducing domain migration costs.

[0025] In a preferred embodiment, the offline processing module includes: Dynamic graph construction engine: Integrates multimodal data sources, eliminates representational differences between different modalities through graph embedding alignment module, and realizes unified semantic fusion of multi-source heterogeneous data; adopts attribute graph sharding technology to partition and store by entity type, and combines Apache Kafka to build a real-time event stream processing framework for distributed storage of domain knowledge graph, as well as real-time atomic addition, deletion and modification operations of nodes and relationships; The lightweight subgraph index building unit extracts entity, relation, and context features from the domain knowledge graph and performs subgraph partitioning using three strategies: dynamic partitioning based on historical query logs, automatic partitioning based on semantic communities using graph clustering algorithms, and rule-driven partitioning based on domain ontology. This generates a subgraph index that is compatible with the user's retrieval type. The subgraph index includes graph embedding compressed index, prefix tree path index, and hybrid bitmap index. Incremental Update Optimization Unit: Employs an incremental index update mechanism, tracking local changes in the domain knowledge graph and their impact propagation, and only performing targeted repairs and updates on subgraph indexes affected by the changes; through a hierarchical storage strategy for hot and cold subgraphs, frequently accessed hot subgraphs are stored in memory, while infrequently accessed cold subgraphs are compressed and persisted to disk, balancing memory usage and hot data response speed, and combining subgraph federated retrieval and approximate subgraph matching techniques to accelerate queries.

[0026] The dynamic graph construction engine achieves semantic unification and real-time updates for multimodal data, solving the problems of single data sources and static knowledge in traditional graphs, and ensuring the comprehensiveness and timeliness of graph data. It provides multiple subgraph partitioning methods to adapt to the needs of different domains and retrieval scenarios; dynamic partitioning based on historical query logs aligns with user retrieval habits, improving retrieval targeting; semantic community partitioning using graph clustering algorithms achieves semantic aggregation of knowledge, ensuring the semantic integrity of subgraphs; domain ontology-driven partitioning conforms to the domain knowledge system, ensuring the professionalism of subgraphs. It employs a combination of various subgraph index types: graph embedding compression indexes ensure the efficiency of vector retrieval, prefix tree path indexes improve the speed of relation path retrieval, and hybrid bitmap indexes optimize the rapid matching of entities and relations. Multi-index collaboration enhances the comprehensiveness and efficiency of subgraph positioning. The incremental update mechanism combined with cold and hot tiered storage reduces index reconstruction overhead by more than 90%. Subgraph federated retrieval and approximate subgraph matching technology, combined with a 2-3 hop retrieval range limit, ensure that relevant subgraphs cover core related knowledge while avoiding redundant calculations and noise introduction caused by an excessively large retrieval range, thus balancing retrieval accuracy and efficiency. Approximate matching technology improves the speed of retrieving relevant subgraphs in large-scale knowledge graphs, while federated retrieval adapts to the retrieval needs of distributed knowledge graphs.

[0027] In a preferred embodiment, the online retrieval module includes: The query parsing module performs structured decomposition on the natural language query input by the user, uses a pre-trained sequence labeling model to identify entities in the query, and accurately links them to standard nodes of the domain knowledge graph through entity disambiguation and linking techniques; it extracts implicit relation predicates in the query based on a graph attention network relation extraction model, and combines graph pattern reasoning to infer potential multi-hop relation paths; Knowledge-enhanced retrieval unit: Maps entities and relational paths obtained from query parsing to a domain knowledge graph; locates and extracts 2-3 hop relevant subgraphs using subgraph federated retrieval and approximate subgraph matching techniques; and assigns the relevant subgraphs to... Figure 3 The head entity, relation, and tail entity in the tuple are encoded as basic vectors. Semantic fusion processing is performed on the basic vectors to generate knowledge hint vectors. The knowledge hint vectors are concatenated to the input hints of the pre-trained language model. Path weights are fused into the self-attention of the Transformer layer of the pre-trained language model to enhance multi-hop inference path attention. Multimodal joint ranking unit: The text semantic matching score and the graph path confidence score output by the pre-trained language model are combined. The weights of the two types of scores are dynamically allocated through a learnable weighting function. The contribution ratio of each score is visualized by combining attribution analysis technology. The retrieval results are ranked and the graph evidence chain is labeled based on the comprehensive ranking score.

[0028] The query parsing module addresses the semantic ambiguity of short text queries through precise entity recognition, entity disambiguation, entity linking, and relation extraction, providing high-quality input for knowledge-enhanced retrieval and ensuring search accuracy. The lightweight adapter module supplements the pre-trained model with knowledge from relevant subgraphs, avoiding parameter inflation and enabling the pre-trained language model to fully utilize domain knowledge. The path-enhanced attention mechanism strengthens multi-hop reasoning capabilities, allowing the model to accurately capture complex logical connections and improve retrieval performance in multi-hop question-answering scenarios.

[0029] Multimodal joint ranking units overcome the limitations of single-modal retrieval, enabling complementary verification of text semantics and graph structure knowledge. Even with low text matching scores, key results rankings can still be improved based on high-confidence graph paths. Attribution analysis and evidence chain annotation make retrieval results traceable, enhancing credibility and usability in domain applications.

[0030] This invention has outstanding substantive features and significant progress compared to the prior art, specifically: (1) By updating the subgraph index through the incremental training method of graph neural network, only the parameters of the local subgraph affected by the change of knowledge graph are fine-tuned, without the need for retraining the whole graph, which greatly reduces the computational cost of knowledge update, realizes real-time response to dynamic iterative knowledge in the domain, avoids large models calling outdated information, ensures the consistency between the retrieval results and the current domain cognition, and expands the application scope of the technology in scenarios that require real-time knowledge updates.

[0031] (2) The weight matrix generated based on the multi-hop relation path of the knowledge graph is integrated into the Transformer layer of the pre-trained language model, so that the attention mechanism can accurately capture the structured relational logic of the domain, break through the limitation of traditional attention relying only on text co-occurrence, strengthen the attention to multi-hop reasoning path, and improve the understanding and matching accuracy of the pre-trained language model for complex relational queries.

[0032] (3) Construct a dual-channel scoring system of text semantic matching and graph path confidence. Through a learnable weighting function, dynamically adapt to the needs of different fields and realize the complementary verification of the two types of information. This ensures the rationality of the search result ranking and can also trace the reasoning basis through graph evidence chain annotation, thereby enhancing the credibility and interpretability of the search results. Attached Figure Description

[0033] Figure 1 This is a schematic diagram of the incremental training method described in this invention; Figure 2 This is a schematic diagram of attention fusion as described in this invention; Figure 3 This is a schematic diagram of the architecture of the pre-trained language model retrieval system based on knowledge graph enhancement according to the present invention. Detailed Implementation

[0034] The technical solution of the present invention will be further described in detail below through specific embodiments.

[0035] Example 1 This embodiment discloses a retrieval method based on a knowledge graph-enhanced pre-trained language model, including: The domain knowledge graph is partitioned into subgraphs to generate subgraph indexes that match the user's search type. The domain knowledge graph and the subgraph indexes are updated using an incremental training method of graph neural networks. The user search type includes queries containing semantically similar words, queries containing explicit relationship chains, and queries combining multiple conditions. Through entity recognition, entity disambiguation, entity linking, and relationship extraction operations, user queries are decomposed into entity sets and relationship paths; based on the entity sets and relationship paths, the corresponding subgraph indexes are located, and relevant subgraphs are extracted from the domain knowledge graph; The related sub Figure 3 The head entity, relation, and tail entity in the tuple are encoded as basic vectors. Semantic fusion processing is performed on the basic vectors to generate knowledge hint vectors. The knowledge hint vectors are then concatenated into the input hints of the pre-trained language model to guide the pre-trained language model to generate retrieval results.

[0036] Domain knowledge graphs contain massive amounts of entities and relationships. Directly integrating them into pre-trained language models can lead to excessive computational resource consumption and low retrieval efficiency. Based on subgraph partitioning strategies and the differentiated characteristics of user searches, this paper performs subgraph partitioning on the complete domain knowledge graph. Following the retrieval logic and data matching requirements of three retrieval types—queries containing semantically similar words, queries containing explicit relationship chains, and multi-condition combination queries—adaptive subgraph indexes are constructed. This achieves precise segmentation of the retrieval data range and lightweight index construction, allowing the subgraph indexes to directly match the core features of various retrieval needs, avoiding redundant computations in full-graph retrieval.

[0037] An incremental training method using graph neural networks is employed for update operations. Leveraging the feature learning capabilities of graph neural networks for graph-structured data, local model training and parameter iteration are performed only on newly added, modified, and deleted entity and relation data in the domain knowledge graph, as well as changes in the corresponding subgraph indexes. This eliminates the need for full retraining and reconstruction of the graph and indexes, enabling synchronous dynamic updates of the domain knowledge graph and subgraph indexes. This ensures the timeliness of graph data and indexes while significantly reducing the computational and time consumption of the update process.

[0038] By employing a series of mature natural language processing and knowledge graph association technologies, including entity recognition, entity disambiguation, entity linking, and relation extraction, unstructured query statements input by users are parsed in a structured manner: entity recognition locates and extracts core entities in the query; entity disambiguation eliminates semantic ambiguity between entities with the same name; entity linking completes the accurate mapping between query entities and entity nodes in the domain knowledge graph; relation extraction uncovers the implicit relationships between entities in the query; and finally, the user query is transformed into a set of entities and relational paths that can be recognized by the graph.

[0039] Based on the parsed entity set and relationship path, the system performs rapid matching and positioning in the constructed subgraph index. Then, based on the matching results, it extracts subgraph data that is highly relevant to the user's search needs from the domain knowledge graph, thus achieving targeted filtering in the search process. This ensures that the extracted subgraph data fully covers the core needs of the user's query, while avoiding the efficiency loss caused by traversing the entire graph.

[0040] For the extracted relevant subgraphs, based on the core data unit triples of the knowledge graph, vector encoding is performed on the head entity, relation, and tail entity in the triples, transforming discrete entity and relation symbols into computable continuous basic vectors, fully preserving the independent semantic features of each element; then, semantic fusion processing is performed on each basic vector, and the semantic features of the head entity, relation, and tail entity are integrated through feature fusion operations to generate knowledge hint vectors containing complete triple association logic, realizing the vectorized and semantic unified representation of the structured knowledge of the graph.

[0041] By directly concatenating the aforementioned knowledge hint vectors into the input hints of the pre-trained language model, the pre-trained language model, in the process of generating retrieval results, not only relies on its own general semantic understanding capabilities, but is also guided and constrained by the domain graph knowledge contained in the knowledge hint vectors. This ensures that the retrieval results generated by the model not only conform to the semantic intent of the user's query, but also strictly match the professional knowledge such as entity associations and relational logic in the domain knowledge graph, ultimately outputting accurate, complete, and domain-relevant retrieval results.

[0042] Example 2 This embodiment discloses a pre-trained language model retrieval method based on knowledge graph enhancement, and its specific implementation in the military field includes the following steps.

[0043] S1: Managing ultra-large-scale dynamic graphs Attribute graph sharding technology is employed, with storage partitioned according to entity type. For example, in the military field, entities such as troops, equipment, regions, and operational regulations are categorized and sharded to achieve logical data isolation and efficient management. A storage cluster is built based on a distributed graph database, supporting horizontal scaling and capable of handling storage needs for hundreds of millions of nodes and edges, while ensuring millisecond-level subgraph retrieval response speeds. A unified entity ID and relationship identification system is designed to ensure consistency of data associations across shards, supporting multi-hop relationship traversal and subgraph extraction across shards.

[0044] A real-time event stream processing framework built on Apache Kafka encapsulates the add, delete, and modify operations of graph nodes and relationships into atomic events, enabling asynchronous processing and sequential execution of operations and avoiding data conflicts. It provides a standardized streaming update interface, supporting real-time data change pushes from external systems and automatically triggering the graph update process to ensure the timeliness of knowledge. A versioned snapshot mechanism records the historical state of the graph, supports update rollback, and allows for rapid switching to a stable version when update anomalies occur, ensuring system availability.

[0045] It integrates multiple data sources, including multimodal information such as text data, image data, and signal data, such as military operational manuals, satellite imagery, and radar signals. A graph embedding alignment module eliminates modal differences, mapping different modal data to a unified semantic space, achieving semantic unification of multi-source data. For example, it transforms geographic features in satellite imagery into geographic entity nodes and associates radar signals with corresponding equipment IDs. Multimodal data attribute association rules are established to bind multi-dimensional attributes to entities, enriching the knowledge dimensions of entities and improving retrieval accuracy.

[0046] Based on subgraph federated retrieval technology, after parsing the semantics of user queries, relevant subgraph shards are intelligently located. A distributed computing framework is used to load relevant data from multiple shards in parallel, performing cross-shard graph traversal and result aggregation, resulting in a 5-10 times faster query speed compared to traditional full graph traversal. An approximate subgraph matching technique is employed, using the SimHash algorithm to extract subgraph topology and semantic features to generate compact fingerprints. A locality-sensitive hash index is constructed, quickly recalling candidate subgraphs. Fine-grained sorting is then performed based on Hamming distance or graph edit distance, tolerating 10%-15% structural differences while improving retrieval efficiency by more than 20 times. A hierarchical storage strategy for hot and cold subgraphs is combined, storing frequently accessed hot subgraphs in memory and persistently storing infrequently accessed cold subgraphs to disk using columnar compression. An LRU caching strategy dynamically adjusts data levels, and the Apache Arrow in-memory computing engine enables seamless cross-layer data integration, balancing memory usage and hot data response speed.

[0047] A distributed locking mechanism is employed to ensure data consistency during concurrent updates across multiple nodes, preventing data corruption caused by simultaneous modifications to the same node or relationship. Data validation rules are introduced to verify the legality of newly added or modified entity attributes and relationship logic, filtering invalid data, such as validating the range of equipment performance parameters and the rationality of combat relationships. An edge-cloud collaborative storage architecture is established, with frontline edge devices storing local hotspot data and performing lightweight updates. Critical changes are synchronized to the cloud for in-depth processing and global synchronization, utilizing the 5G tactical network to achieve second-level parameter synchronization, while also supporting incremental data synchronization after network reconnection.

[0048] S2: Subgraph Partitioning and Index Generation Subgraph partitioning strategies include: dynamic partitioning based on query patterns, automatic partitioning based on semantic communities, and rule-driven domain partitioning. Dynamic partitioning based on query patterns analyzes high-frequency entities and relationship paths from historical query logs, constructing subgraphs around core entities. For example, in the military domain, subgraphs are generated around high-frequency associated entity clusters such as "strategic location-battlefield environment-infrastructure" and "carrier battle group-destroyer-missile model," accurately adapting to users' high-frequency retrieval needs and improving query hit rate. Automatic partitioning based on semantic communities uses graph clustering algorithms such as Louvain and spectral clustering to identify semantically close sub-regions in the knowledge graph. It quantifies the strength of associations between entities by combining entity attributes, relationship types, and contextual co-occurrence frequency, and uses community discovery algorithms to cluster strongly associated entities into the same community while minimizing cross-community connections, forming business-meaning subgraph units suitable for complex association analysis scenarios. Rule-driven domain partitioning predefines subgraph boundaries based on the domain ontology. For example, in the military domain, subgraphs are partitioned according to hierarchical relationships such as "theater zone, service branch, army group, division / brigade / regiment level units," ensuring that subgraphs conform to the domain knowledge system, have clear boundaries, and are highly specialized.

[0049] The three subgraph partitioning strategies form a three-tiered complementary system of "rule-driven, semantic community, and query pattern". Regular queries prioritize matching rule-based subgraphs, complex relational queries call semantic community subgraphs, and high-frequency hot queries adapt to dynamically partitioned subgraphs. When one partition becomes invalid due to data changes, other partitions can provide backup support, improving the system's resilience to dynamic changes.

[0050] The subgraph index structure includes: graph embedding compression-based indexes, prefix tree path-based indexes, and hybrid bitmap-based indexes. Graph embedding compression-based indexes map entities and relations in the knowledge graph into low-dimensional dense vectors using embedding algorithms such as TransE and RotatE. Binarization or scalar quantization of these vectors achieves spatial compression, constructing a hierarchical index structure of "entity ID - compressed vector, relation type - subgraph pointer". This index reduces storage overhead by 60%-80% while preserving semantic relevance. Combined with an approximate nearest neighbor retrieval algorithm, it achieves millisecond-level semantic matching, suitable for semantically similar query scenarios such as "entities with similar functions to a certain equipment". Prefix tree path-based indexes encode high-frequency, multi-hop query paths in the knowledge graph into string sequences, for example, "carrier battle group - destroyer - YJ-21 missile" is encoded as "CarrierGroup_Destroyer_YJ-21", constructing a hierarchical character node tree structure. During queries, the system drills down character by character, with leaf nodes directly pointing to the memory address of the corresponding subgraph. This reduces the time complexity of chained queries, making it particularly suitable for scenarios with strong path dependencies, such as combat sequence tracing and equipment system association analysis. Based on a hybrid bitmap index, entities within the knowledge graph subgraph are encoded using binary bits, constructing a two-dimensional bit matrix of "relation type - entity ID," with bit values ​​indicating the existence of the relation in the subgraph. Bitwise operations are used to quickly batch filter entity combinations that meet multiple relation constraints, such as parallel matching of combat units "belonging to the Eastern Theater Command AND deployed in the South China Sea." Combined with run-length encoding compression technology, storage space is reduced by 60%-85%, and GPU-accelerated bit operations are supported, making it suitable for high-concurrency scenarios with multi-condition joint queries.

[0051] The system first analyzes the query type and intelligently routes the query to the optimal index. Queries containing semantically similar words match the graph embedding compressed index, queries with explicit relationship chains call the prefix tree path index, and queries with multiple conditions trigger the hybrid bitmap index. Through the collaboration of multiple indexes, the system ensures retrieval efficiency for different query paradigms while avoiding the functional limitations of a single index. For example, during updates, the bitmap index is used to provide temporary services while the graph embedding and prefix tree indexes are updated asynchronously, balancing update efficiency and retrieval continuity.

[0052] S3: Execute incremental training methods like Figure 1As shown, the system monitors real-time changes to the knowledge graph, including new node / edge additions (e.g., adding new missiles and their performance parameters in the military field); entity attribute modifications (e.g., updating the detection range of air defense radars); and relationship deletions (e.g., abolishing equipment usage restrictions in old operational regulations). This ensures that all three types of changes are captured without omission. Based on graph diffusion algorithms, such as random walks and neighborhood sampling, the system analyzes the propagation range of changes to nodes, accurately identifying the affected local subgraphs. For example, when adding "Missile M2 - Belongs to - 5th Armored Division," only the 5th Armored Division and its directly associated equipment and personnel nodes are marked as the area to be updated, avoiding indiscriminate processing of the entire graph.

[0053] The parameters of the graph neural network are divided into global parameters and local parameters. Global parameters are used to capture general relational patterns in the domain, such as "command hierarchy" and "equipment type" in the military field, and remain fixed after initial training. Local parameters are adapted to specific subgraph structures, such as a unit's dedicated equipment library, and are fine-tuned only for subgraphs affected by changes. Gradient masking is used to calculate gradients only for neurons corresponding to affected subgraphs, shielding updates of irrelevant parameters and significantly reducing computational overhead. Elastic weights are introduced during training to reinforce regularization terms, protecting important parameters obtained from historical training from being overwritten by new data and ensuring model performance stability. For high-time-sensitive domains such as the military, scheduling priorities are set according to the urgency of changes. High-frequency changes such as wartime intelligence trigger immediate incremental updates, while low-frequency tasks such as supplementing historical battle data are processed in batches. Subgraph embedding caching is used for local subgraphs to reduce redundant encoding, and parameters are compressed through binarized GNNs to improve update efficiency on edge devices, such as tactical tablets.

[0054] Dedicated update strategies are designed for different types of subgraph indexes: graph embedding compression indexes employ incremental training and block updates to reduce reconstruction overhead; prefix tree path indexes adapt new paths through dynamic Trie and lazy reconstruction; hybrid bitmap indexes utilize bit segment expansion and versioned bitmaps for rapid updates. The impact range of entity changes is analyzed based on reverse adjacency lists, and only affected subgraph indexes are dynamically patched, reducing index reconstruction overhead by over 90%. A versioned snapshot mechanism records the historical state of the graph, enabling atomic switching between old and new indexes, allowing for rapid rollback to a stable version in case of update anomalies. A hierarchical storage strategy for hot and cold subgraphs prioritizes updating frequently accessed subgraph blocks, ensuring query consistency while improving update efficiency. After local GNN parameter updates, hint vector updates are automatically triggered based on the query. For example, when a "disabled condition - extreme cold climate" triple is added to "Missile M2," the adapter re-encodes the triple and generates the latest hint for injection into the large model. The path weight matrix is ​​updated synchronously to ensure that the path-enhanced attention mechanism promptly addresses newly introduced rules, guaranteeing the continuity of knowledge transfer.

[0055] Employing an edge-cloud collaborative architecture, frontline edge devices store local hotspot data and perform lightweight updates. For example, when modifying company-level equipment data, critical changes are synchronized to the cloud for in-depth fine-tuning and global synchronization, leveraging the 5G tactical network to achieve second-level parameter synchronization. A rollback fault tolerance mechanism is added; if an update causes anomalies such as search result conflicts, it automatically switches to the previous stable version. A / B testing verifies the effectiveness of new parameters, ensuring no performance degradation after updates and guaranteeing efficient and accurate local updates to large-scale dynamic knowledge graphs.

[0056] S4: Perform subgraph pruning Based on business scenarios in the military vertical domain, predefined key meta-path patterns are implemented. The military domain includes troop deployment chains (troops - deployed to - area - threatened - enemy equipment) and equipment association chains (radar - detection - target - intercepted - missile), etc. Dynamic meta-path expansion is also supported; when a new relationship type is added to the knowledge graph, such as "electronic jamming" in the military domain, the corresponding derived meta-path "radar - electronic jamming - communication system" is automatically generated. A storage strategy based on entity type sharding is adopted, dividing the knowledge graph into multiple logically independent subgraphs. For example, the military domain is divided into "troops," "equipment," and "area" subgraphs. Distributed graph databases, such as Neo4j clusters, are used for management, laying the foundation for parallel pruning operations.

[0057] For each node and edge in the knowledge graph, its information contribution entropy in participating in predefined meta-paths is calculated to quantify the importance of nodes and edges. Lower entropy values ​​indicate that the node or edge carries more core knowledge in the meta-path, and thus has greater value for domain retrieval; higher entropy values ​​indicate higher redundancy, and should be prioritized for pruning. Predefined meta-paths are traversed in parallel within each shard, and a MapReduce architecture is used to statistically analyze path frequency and node information contribution entropy values, avoiding efficiency bottlenecks caused by single-shard processing and improving the pruning speed of large-scale knowledge graphs.

[0058] The maximum allowable subgraph size is derived by working backward from the GPU memory capacity. Combined with the domain retrieval accuracy requirements, retention thresholds for nodes and edges are set to ensure that the pruned subgraph can adapt to model computational resources while preserving core knowledge connections. Nodes and edges are sorted in ascending order of their information contribution entropy, prioritizing the retention of low-entropy nodes and edges and pruning high-entropy ones. If pruning operations may cause key element paths to break, minimum connecting edges are automatically added to ensure subgraph connectivity and prevent the loss of core inference paths. After pruning, simulated retrieval tests are conducted to verify the completeness of subgraph knowledge coverage and retrieval response speed. If core knowledge is missing or performance is substandard, the retention thresholds are dynamically adjusted and the pruning process is re-executed.

[0059] By periodically accumulating new stream data from the knowledge graph into micro-batches, such as every 5 seconds, frequent pruning operations are avoided. The information contribution entropy of nodes and edges is recalculated only for the subgraph regions affected by new data, eliminating the need for full-graph pruning updates and significantly reducing update overhead. The incrementally pruned subgraphs are merged with the original subgraphs, and the corresponding subgraph indexes are updated synchronously, using methods such as embedded compressed indexes, prefix tree path indexes, and hybrid bitmap indexes. This ensures that the pruned subgraphs can promptly support online retrieval services, guaranteeing the timeliness and accuracy of knowledge retrieval.

[0060] S5: Parse the natural language query input by the user. This system employs pre-trained sequence annotation models such as BERT+CRF to accurately identify entities in user natural language queries, covering core entity types in areas such as names, organizations, equipment, terminology, and geographical regions. For example, in military queries, it identifies entities such as "Surface-to-Air Missile System M" and "South China Sea." For ambiguous entities in the query, it combines standard node information from the knowledge graph and calculates features such as entity context similarity and frequency of associations to eliminate entity ambiguity. For example, "Liaoning" may correspond to a province or an aircraft carrier; by associating the query context with the knowledge graph, its specific meaning is determined. The disambiguated entities are then accurately mapped to standard nodes in the domain knowledge graph, establishing a unique association between the query entity and the graph entity, laying the foundation for subsequent subgraph extraction.

[0061] By leveraging relation extraction models such as graph attention networks, implicit relational predicates are extracted from query text, such as extracting the "assigned" relation from "missile models assigned to a certain unit". Combining the graph patterns of the domain knowledge graph, potential multi-hop relational paths are inferred based on the identified entities and extracted relations. For example, from "Zhukov-command-Stalingrad Campaign", a multi-hop path "Zhukov-command campaign-Stalingrad Campaign-deployment-T-34 tank-manufacturer-Ural tank factory" is inferred. The confidence scores of entity recognition and relation extraction are jointly calculated to filter out entities and relations with low confidence, ensuring the accuracy of structured semantic representation and avoiding interference from erroneous information in subsequent retrievals.

[0062] By combining the context of the query, the correlation between entities and relationships is strengthened. For example, in the military query "detection radius of a certain radar in rainy and foggy weather," the context of "rainy and foggy weather" and "detection radius" helps to determine the "performance impact" relationship between "radar" and "detection radius." The parameters and features of entity recognition and relationship extraction models are optimized for the characteristics of different vertical domains. For example, in the military domain, the recognition accuracy of entities and relationships such as "troops," "equipment," and "deployment" is enhanced. Lightweight model structures or model distillation techniques are adopted to reduce the computational overhead of query parsing, ensuring millisecond-level response even in high-concurrency scenarios, adapting to the real-time retrieval needs of search engines, intelligent question answering, and other applications. The query parsing ultimately outputs a structured set of entities, relational predicates, and multi-hop relational paths, such as "Entity set: {Surface-to-air missile system M}, relational predicate: {disabled conditions}, multi-hop path: Surface-to-air missile system M - disabled conditions - strong electromagnetic interference environment," providing accurate basis for subsequent subgraph indexing based on entities and paths and extraction of relevant subgraphs.

[0063] S6: Extract relevant subgraphs Based on the structured entity set, relational predicates, and multi-hop relational paths output by the query parsing, the relevant subgraph extraction process is initiated. Using a distributed storage architecture based on a large-scale dynamic graph, a unified entity ID and relational identifier system is employed to locate the subgraph shard where the entity resides, laying the foundation for cross-shard subgraph extraction. Lightweight indexes corresponding to the target subgraph shard are loaded, including graph embedding compressed indexes, prefix tree path indexes, and hybrid bitmap indexes. These indexes are used to quickly filter candidate entities and relations relevant to the query, reducing unnecessary traversals.

[0064] By extracting and parsing query semantic features through subgraph federated retrieval, the system intelligently routes queries to matching subgraph shards. For example, a military query for "missile models assigned to the South China Sea Fleet" is routed to subgraph shards related to "naval organization" and "equipment assignment." Distributed computing frameworks, such as the Apache Arrow memory engine, are used to load candidate data from multiple related shards in parallel. Cross-shard graph traversal is performed based on the Pregel model, connecting multi-hop relationships between entities. The cross-shard traversal results are aggregated and deduplicated, extracting 2-3 hop subgraph structures containing query entities and relationship paths. This ensures that the subgraph covers core relational knowledge, resulting in a query speed improvement of 5-10 times compared to traditional full graph traversal.

[0065] By extracting the subgraph topology corresponding to the query through approximate subgraph matching, such as node degree distribution, relational path patterns, and semantic features, a compact fingerprint is generated. A locality-sensitive hash index is retrieved to quickly recall candidate subgraphs similar to the query fingerprint, tolerating 10%-15% structural differences, adapting to scenarios with fragmented battlefield intelligence and data noise interference. Candidate subgraphs are finely sorted based on Hamming distance or graph edit distance, selecting the subgraph with the highest relevance to the query, resulting in retrieval efficiency more than 20 times higher than exact matching.

[0066] The subgraph extraction depth is strictly controlled to 2-3 hops to avoid redundancy and increased computational overhead due to excessive traversal, while ensuring coverage of the core related entities and relationships required for the query. Based on the subgraph pruning results, subgraphs composed of low-entropy nodes and edges are extracted first, eliminating highly redundant information to ensure lightweight and high information density. Connectivity checks are performed on the extracted subgraphs; if key relationships are broken, the minimum connecting edges are automatically added to ensure the integrity of the subgraph and the usability of inference.

[0067] In the military field, for queries with strong path dependencies such as combat sequences and equipment allocation, subgraph federated retrieval is prioritized to enhance the accurate connection of multi-hop relationships; it is adapted to dynamic battlefield data and supports the rapid extraction of subgraphs related to newly added equipment and tactical rules.

[0068] S7: Perform triple encoding on the relevant subgraph and concatenate it with the input cues from the pre-trained language model. By introducing structured relational path information from knowledge graphs, the model's attention is dynamically allocated, enhancing the capture of multi-hop logical relationships, improving the understanding and matching accuracy of pre-trained language models for complex queries, and simultaneously enhancing the interpretability of retrieval results.

[0069] For each triple in the relevant subgraph, its head entity, relation, and tail entity are independently embedded. Pre-trained word vectors or domain-specific entity encoders, such as those for military equipment, are used to transform entities and relations into low-dimensional, dense base vectors, ensuring accurate capture of entity semantics and relational logic. For special elements such as domain-specific terms and long-tail entities, the encoder is fine-tuned using domain-specific corpora to improve the semantic representation accuracy of the base vectors and avoid semantic biases caused by general encoders.

[0070] A relational attention mechanism is introduced, using relation vectors as the core, to calculate the association strength between entity vectors and relation vectors, thereby strengthening the guiding role of relations in entity association. Based on the association strength, fusion weights are dynamically allocated, and the basic vectors of head entities, relations, and tail entities are semantically fused to generate compact and structured knowledge hint vectors. The semantic relevance between entities and relations is preserved during the fusion process, ensuring that the knowledge hint vector accurately reflects the logical relationships of triples, such as the constraint relationship of "Surface-to-Air Missile System M - Disabled Condition - Strong Electromagnetic Interference Environment".

[0071] The generated knowledge hint vector is sequentially concatenated with the user's natural language query input sequence to form a composite input structure of "user query and knowledge hint vector". The concatenated input sequence serves as additional contextual information and is directly input into the pre-trained language model without modifying the model's original backbone parameters, achieving flexible knowledge injection. For long queries or scenarios with multiple triples, a batch concatenation strategy is adopted, prioritizing the retention of knowledge hint vectors corresponding to core triples to ensure that the input sequence length meets the model's limitations.

[0072] S8: Integrate path weights in the self-attention calculation of the Transformer layer of the pre-trained language model. like Figure 2 As shown, multi-hop relationship paths containing core entity association logic are filtered from the relevant subgraph, prioritizing paths with high semantic matching to user queries and compact topological structures. For example, for the military query "detection radius of a certain radar in rainy and foggy weather," the multi-hop path "radar-deployment environment-rainy and foggy weather-performance impact-detection radius" is filtered out. Graph neural networks or relation rotation encoding are used to vectorize each relation in the path, transforming relations such as "deployment environment" and "performance impact" into low-dimensional dense vectors to ensure accurate semantic representation of the relations. The encoded relation vectors are then normalized to eliminate dimensional differences, laying the foundation for subsequent weight calculations.

[0073] The weights of each relationship are calculated by combining multi-dimensional features, including path length, relationship confidence, and entity affinity. This is achieved through a formula. Calculate the path weight between the i-th query term and the j-th document term, where α k For learnable parameters, f (r k The weights are for relation types, such as "disabled conditions" and "commands," where core relations have higher weights than "general associations." A sparse path weight matrix P is constructed, assigning non-zero values ​​only to word pairs related to the knowledge path, significantly reducing computational overhead and improving the efficiency of attention computation.

[0074] The path weight coefficient λ is dynamically preset based on the query type. For strong logic queries such as military rule queries, it is set to 0.5, and for ordinary descriptive queries, it is set to 0.2, achieving scenario adaptation of the attention strategy. In the self-attention calculation of the Transformer layer of the pre-trained language model, the path weight coefficient and the path weight matrix are fused, and the formula is used... The path-enhanced attention output is obtained, where Q is the query vector, K is the key vector, V is the value vector, and d is the vector dimension. The path-enhanced attention output after self-attention fusion with path weights is processed sequentially through residual connections, layer normalization, and a feedforward network to obtain the final output of the Transformer layer. By strengthening the attention weight allocation of word pairs related to knowledge paths, for example, in military retrieval scenarios, the semantic association between "radar" and "detection radius" constructed through the "weather influence" path can be strengthened, effectively improving the model's multi-hop reasoning ability on the knowledge graph.

[0075] In conjunction with the triplet encoding module, the generation of the path weight matrix relies on the triplet relationship information of related subgraphs, ensuring consistency between attention fusion and knowledge injection. Collaborating with the incremental update module, the path weight matrix is ​​updated synchronously when new relationships or paths are added to the knowledge graph, ensuring the attention mechanism promptly captures new structured knowledge. Connecting with the dual-channel scoring module, the model output after attention fusion serves as the basis for calculating the text semantic matching score, providing support for subsequent comprehensive ranking.

[0076] S9: Perform dual-channel scoring and sorting of search results The pre-trained language model receives a composite input of "user query and knowledge hint vector," and performs joint inference through self-attention and cross-attention mechanisms. Based on the contextual semantic understanding of the query and candidate content, a softmax layer outputs a relevance probability as the text semantic matching score. This score reflects the degree of fit between the query and the result at the natural language level, and incorporates the optimization effects of knowledge injection and attention enhancement. Subgraph paths related to the query are extracted from the knowledge graph, and path reliability is quantified based on graph structure statistical features. Core statistical indicators include node degree centrality, relation frequency, and topological density. Path confidence scores are calculated using predefined or learnable functions such as weighted averages, reflecting the strength of structured knowledge support.

[0077] A learnable weighting function is used as the fusion core, supporting both gated network and linear interpolation structures. The gated network dynamically adjusts the weight ratio of the two types of scores through adaptive gating units, while linear interpolation achieves smooth fusion through learnable coefficients. Both structures can adapt to the retrieval characteristics of different domains. The weight allocation strategy is iteratively optimized through training data, and the function parameters are adjusted using labeled data to ensure that the weight ratio of text semantic matching score and graph path confidence score aligns with the actual needs of the domain. For example, in strong logical queries in the military domain, the weight of the graph path confidence score is increased; in general descriptive queries, the emphasis is placed on the text semantic matching score.

[0078] Text semantic matching score and graph path confidence score are input into a dual-channel scoring function to generate a comprehensive ranking score. Search results are then sorted in descending order of score to ensure that highly relevant and reliable results are presented first. Each ranked result is labeled with its corresponding core knowledge graph path, such as "Surface-to-Air Missile System M - Disabled Conditions - Strong Electromagnetic Interference Environment," clearly demonstrating the structured knowledge basis of the results and enhancing user trust in the search results. An attention heatmap illustrates the calculation basis of the text semantic matching score, and subgraph paths highlight the core contribution of the graph path confidence score, intuitively visualizing the contribution ratio of the two types of scores and improving the interpretability of the results.

[0079] Example 3 like Figure 3 As shown, this embodiment discloses a pre-trained language model retrieval system based on knowledge graph enhancement, applied in the military field. The system adopts a distributed deployment mode of offline processing cluster and online retrieval service. The offline processing module is deployed on the cloud server cluster; the online retrieval module is deployed on edge computing nodes and the cloud. The edge nodes are responsible for real-time query parsing and lightweight retrieval at the front line, while the cloud is responsible for complex subgraph extraction and multimodal fusion sorting. Data synchronization is achieved through the network.

[0080] The offline processing module includes: (1) Dynamic graph construction engine It integrates multimodal data sources, including military text data, satellite imagery data, radar signal data, and dynamic troop deployment data. A graph embedding alignment module maps geographic features in satellite imagery to geographic entity nodes and associates radar signals with corresponding equipment IDs, eliminating representational differences among multimodal data and achieving semantic uniformity. Attribute graph sharding technology is employed, dividing the data into four sub-graph shards based on entity type: "Troop Organization," "Military Equipment," "Operational Area," and "Tactical Rules," stored in a Neo4j cluster, supporting millisecond-level sub-graph retrieval. A real-time event stream processing framework is built based on Apache Kafka to receive atomic change events such as equipment updates and deployment adjustments pushed by the frontline intelligence system, automatically triggering add, delete, and modify operations on graph nodes and relationships to ensure the timeliness of knowledge.

[0081] (2) Lightweight subgraph index building unit The system extracts entity, relationship, and contextual features from each fragment and employs three subgraph partitioning strategies: First, it constructs hotspot subgraphs around core entities such as "carrier battle group" and "air defense system" based on historical operational query logs. Second, it utilizes the Louvain algorithm to identify semantic community subgraphs such as equipment coordination and unit affiliation. Third, it predefines the boundaries of subgraphs at the "theater zone-service-army group-brigade" hierarchical levels based on military domain ontology. Finally, it generates three types of lightweight subgraph indexes: a graph embedding compression index is constructed using the TransE algorithm, reducing storage overhead by 75%; a prefix tree path index is constructed by encoding high-frequency paths such as "equipment-deployment area-combat mission"; and a hybrid bitmap index is constructed for relationships such as "affiliation" and "disabled conditions," supporting multi-condition joint queries.

[0082] (3) Incremental update optimization unit An incremental index update mechanism is adopted, using a reverse adjacency list to track the impact of changes such as equipment additions and deployment adjustments. Only the affected subgraph indexes are patched and updated, reducing index reconstruction overhead by 92%. A hierarchical storage system for hot and cold subgraphs is implemented. Hot subgraphs such as recently active theater deployments and new equipment parameters are stored in memory, while cold subgraphs such as historical battle records and retired equipment archives are compressed and stored on disk in Parquet format. This is combined with an LRU caching strategy to dynamically adjust data levels. Linked subgraph federated retrieval and approximate subgraph matching technologies enable parallel loading of relevant data across shards, improving the response speed of complex queries by 8 times.

[0083] The online search module includes: (1) Query parsing module Users input the query "Under what scenarios should the deployment of the surface-to-air missile system M be suspended" through a tactical terminal. The module uses a BERT+CRF model to identify the entity "surface-to-air missile system M", and links it to the standard nodes of the graph through entity disambiguation and entity linking technology. It uses a graph attention network to extract the implicit relation "disabled conditions" and infers the multi-hop relation path "surface-to-air missile system M - disabled conditions - strong electromagnetic interference environment".

[0084] (2) Knowledge-enhanced retrieval unit Entities and relational paths are mapped to a military knowledge graph. Subgraph federated retrieval locates relevant fragments related to "military equipment" and "tactical rules," extracting two-hop related subgraphs containing nodes and relationships such as surface-to-air missile system M, disabling conditions, and strong electromagnetic interference environments. A lightweight adapter module employing a low-rank matrix transformation structure is used to transform the relevant subgraphs... Figure 3The head entity, relation, and tail entity in the tuple are encoded as basic vectors. Semantic fusion processing is performed on the basic vectors to generate knowledge hint vectors. The knowledge hint vectors are concatenated to the input hints of the pre-trained language model. A path weight matrix is ​​introduced in the Transformer layer and superimposed with the self-attention score to strengthen the focus on the path of "surface-to-air missile system M-disabled condition-strong electromagnetic interference environment".

[0085] (3) Multimodal joint sorting unit The pre-trained language model outputs a text semantic matching score of 0.88 and a graph path confidence score of 0.95. The path confidence score is calculated based on node degree centrality, relation frequency, and topological compactness. A gating network is used to dynamically assign weights α=0.4 and β=0.6 to the text semantic matching score and the graph path confidence score, resulting in a comprehensive ranking score of 0.922.

[0086] Based on the comprehensive score ranking output, the evidence chain of the graph is labeled as "Surface-to-air missile system M-disabled condition-strong electromagnetic interference environment", and the contribution ratio is visualized through attention heatmap.

[0087] The system supports second-level response to battlefield intelligence retrieval needs, improves the accuracy of multi-hop queries by 15%, and enhances the interpretability of retrieval results by 40%. It can provide real-time and accurate structured knowledge support for combat decision-making and is adapted to dynamic knowledge update scenarios during wartime.

[0088] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit them; although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications can still be made to the specific implementation of the present invention or equivalent substitutions can be made to some technical features without departing from the spirit of the technical solutions of the present invention, and all such modifications and substitutions should be covered within the scope of the technical solutions claimed in the present invention.

Claims

1. A retrieval method based on a pre-trained language model augmented with knowledge graphs, characterized in that, include: Perform subgraph partitioning on the domain knowledge graph to generate a subgraph index that matches the user's search type; An incremental training method using graph neural networks is employed to update the domain knowledge graph and the subgraph index; wherein, the user retrieval types include queries containing semantically similar words, queries containing explicit relationship chains, and multi-condition combination queries; Through entity recognition, entity disambiguation, entity linking, and relationship extraction operations, user queries are decomposed into entity sets and relationship paths; based on the entity sets and relationship paths, the corresponding subgraph indexes are located, and relevant subgraphs are extracted from the domain knowledge graph; The head entity, relation, and tail entity in the relevant subgraph triples are encoded into basic vectors, and semantic fusion processing is performed on the basic vectors to generate knowledge hint vectors. The knowledge hint vectors are then concatenated into the input hints of the pre-trained language model to guide the pre-trained language model to generate retrieval results.

2. The retrieval method for pre-trained language models based on knowledge graph enhancement according to claim 1, characterized in that, The incremental training method using graph neural networks updates the domain knowledge graph and the subgraph index, including: The method identifies the types of changes to a domain knowledge graph, analyzes the propagation range of the changed nodes using a graph diffusion algorithm, and determines the local subgraphs affected by the changes. The types of changes include adding nodes or edges, modifying entity attributes, and deleting relationships. The graph diffusion algorithm includes random walk algorithms and neighborhood sampling algorithms. The parameters of the graph neural network are divided into global parameters and local parameters; the global parameters are used to capture general relational patterns in the domain and are fixed after initial training; the local parameters are used to fine-tune the local subgraphs affected by the changes. The gradient of neurons corresponding to the affected subgraph is calculated based on the incremental training method, and the gradient mask is used to shield the updates of irrelevant parameters. During the training process, elastic weights are introduced to consolidate the regularization term to protect important parameters obtained from historical training from being overwritten by new data. Update the subgraph index affected by the change based on the local parameters trained using the incremental training method.

3. The retrieval method for pre-trained language models based on knowledge graph enhancement according to claim 2, characterized in that, It also includes fusing path weights in the self-attention computation of the Transformer layer of the pre-trained language model to enhance multi-hop inference path attention during the generation of retrieval results by the pre-trained language model, including: Filter out multi-hop relationship paths containing entity association logic from the relevant subgraphs; Each relation in the multi-hop relation path is vectorized and encoded, and the weight of each relation is calculated by combining the path length, relation confidence and the degree of entity association, to generate a sparse path weight matrix; The path weight coefficients are dynamically preset according to the query type. During the self-attention calculation process of the Transformer layer of the pre-trained language model, the path weight coefficients and the path weight matrix are fused to obtain the path-enhanced attention output. The formulas for calculating the weights of each relation are as follows: ; in, P ij This represents the path weight between the i-th query term and the j-th document term. r k For the k-th relation in the path, α k For learnable parameters, f (r k ) represents the relation type weight; The calculation expression for the path-enhanced attention output is: ； Where Q is the query vector, K is the key vector, V is the value vector; λ is the path weight coefficient; P is the path weight matrix; and d is the vector dimension.

4. The retrieval method for pre-trained language models based on knowledge graph enhancement according to claim 3, characterized in that, It also includes a dual-channel scoring and search result ranking step, which includes: Obtain the text semantic matching score and knowledge graph path confidence score output by the pre-trained language model; wherein, the text semantic matching score is generated based on the contextual semantic understanding of the query and candidate content by the pre-trained language model; the path confidence score is calculated based on the degree centrality, relation frequency and topological compactness of the nodes retrieved in the knowledge graph; A learnable weighted function is used as the dual-channel scoring function. The weighted function adopts a gated network or a linear interpolation structure. The weight allocation strategy of text semantic matching score and graph path confidence score is optimized through training iteration to adapt to the retrieval scenario requirements of different fields. The text semantic matching score and the graph path confidence score are input into the dual-channel scoring function, and the weights of the two types of scores are dynamically allocated to obtain a comprehensive ranking score. The search results are sorted and output based on the comprehensive ranking score, and the corresponding graph evidence chains are labeled.

5. The retrieval method for pre-trained language models based on knowledge graph enhancement according to claim 4, characterized in that, The step of concatenating the knowledge hint vector into the input hint of the pre-trained language model to guide the pre-trained language model to generate retrieval results includes: Embedding representations are performed on the head entity, relation, and tail entity in the triples of the relevant subgraphs to generate their respective basic vectors; The basic vectors are semantically fused using a relational attention mechanism to generate knowledge hint vectors, which are used to strengthen the guiding role of relations in entity association. The knowledge hint vector is concatenated with the user's query input sequence and used as additional contextual information to input into the pre-trained language model.

6. The retrieval method for pre-trained language models based on knowledge graph enhancement according to claim 5, characterized in that, It also includes pruning the subgraph, the pruning including: Preset domain-specific meta-path templates and predefine key meta-path patterns; when a new relation type is added to the knowledge graph, the corresponding derived meta-path is automatically generated; For each node and edge in the knowledge graph, calculate its information contribution entropy in participating in the meta-path to quantify the importance of the node and edge; The knowledge graph is stored in segments according to entity type and managed using a distributed graph database. Within each segment, predefined meta-paths are traversed in parallel to count the path frequency and the information contribution entropy value of the nodes. Based on the maximum allowable subgraph size of the GPU memory capacity, set retention thresholds for nodes and edges; sort nodes and edges in ascending order of information contribution entropy, prioritize retaining low-entropy nodes and edges, and prune high-entropy nodes and edges; if pruning causes key element paths to break, supplement with minimum connecting edges to ensure the connectivity of the subgraph. New streaming data from the knowledge graph is periodically accumulated into micro-batches. The information contribution entropy is recalculated and pruning is performed only on the subgraph regions affected by the new data, thus avoiding full graph pruning updates.

7. A retrieval system for a pre-trained language model based on knowledge graph augmentation, characterized in that, The knowledge graph-based pre-trained language model retrieval method according to any one of claims 1-6 includes an offline processing module and an online retrieval module; the offline processing module is used for dynamic construction of domain knowledge graphs, generation of lightweight subgraph indexes, and incremental optimization, providing efficient data support for online retrieval; the online retrieval module is used to output accurate and interpretable retrieval results through structured query parsing, knowledge-enhanced retrieval, and multimodal fusion sorting.

8. The retrieval method for pre-trained language models based on knowledge graph enhancement according to claim 7, characterized in that, The offline processing module includes: Dynamic graph construction engine: Integrates multimodal data sources, eliminates representational differences between different modalities through graph embedding alignment module, and realizes unified semantic fusion of multi-source heterogeneous data; adopts attribute graph sharding technology to partition and store by entity type, and combines Apache Kafka to build a real-time event stream processing framework for distributed storage of domain knowledge graph, as well as real-time atomic addition, deletion and modification operations of nodes and relationships; The lightweight subgraph index building unit extracts entity, relation, and context features from the domain knowledge graph and performs subgraph partitioning using three strategies: dynamic partitioning based on historical query logs, automatic partitioning based on semantic communities using graph clustering algorithms, and rule-driven partitioning based on domain ontology. This generates a subgraph index that is compatible with the user's retrieval type. The subgraph index includes graph embedding compressed index, prefix tree path index, and hybrid bitmap index. Incremental Update Optimization Unit: Employs an incremental index update mechanism, tracking local changes in the domain knowledge graph and their impact propagation, and only performing targeted repairs and updates on subgraph indexes affected by the changes; through a hierarchical storage strategy for hot and cold subgraphs, frequently accessed hot subgraphs are stored in memory, while infrequently accessed cold subgraphs are compressed and persisted to disk, balancing memory usage and hot data response speed, and combining subgraph federated retrieval and approximate subgraph matching techniques to accelerate queries.

9. The retrieval method for pre-trained language models based on knowledge graph enhancement according to claim 8, characterized in that, The online search module includes: The query parsing module performs structured decomposition on the natural language query input by the user, uses a pre-trained sequence labeling model to identify entities in the query, and accurately links them to standard nodes of the domain knowledge graph through entity disambiguation and linking techniques; it extracts implicit relation predicates in the query based on a graph attention network relation extraction model, and combines graph pattern reasoning to infer potential multi-hop relation paths; The knowledge-enhanced retrieval unit maps the entities and relational paths obtained from query parsing to a domain knowledge graph. It locates and extracts 2-3 hop relevant subgraphs using subgraph federated retrieval and approximate subgraph matching techniques. The head entity, relation, and tail entity in the triples of the relevant subgraphs are encoded as base vectors. Semantic fusion processing is performed on these base vectors to generate knowledge hint vectors. These knowledge hint vectors are concatenated into the input hints of a pre-trained language model. Path weights are fused into the self-attention of the Transformer layer of the pre-trained language model to enhance multi-hop inference path focus. Multimodal joint ranking unit: The text semantic matching score and the graph path confidence score output by the pre-trained language model are combined. The weights of the two types of scores are dynamically allocated through a learnable weighting function. The contribution ratio of each score is visualized by combining attribution analysis technology. The retrieval results are ranked and the graph evidence chain is labeled based on the comprehensive ranking score.

Images