A method and system for privatizing knowledge import and co-evolution

Abstract

The application discloses a kind of private knowledge import and coevolution method and system, belong to artificial intelligence and natural language processing technical field, based on multimodal fusion and dynamic knowledge graph realizes private knowledge import and coevolution, include: receiving enterprise internal including at least two modal private data, by quality perception-based adaptive fusion module, feature extraction and weighted fusion are carried out to the private data, generate unified knowledge vector representation;Based on the knowledge vector representation, by event-driven dynamic knowledge graph construction module, automatically extract fact triple, and utilize incremental graph learning algorithm to update existing knowledge graph in real time;Query processing;Result generation and feedback;Coevolution.The application realizes private knowledge import and question and answer, can be used in enterprise internal environment, can fuse multimodal data, dynamically construct and update knowledge graph, and realize model and knowledge base coevolution.

Description

Technical Field

[0001] This invention relates to the fields of artificial intelligence and natural language processing technology, specifically a method and system for privatized knowledge import and co-evolution. Background Technology

[0002] With the rise of Large Language Models (LLMs), leveraging their powerful understanding and generation capabilities to empower enterprise knowledge management has become a popular research direction. Enterprises aim to build an internal "knowledge brain," enabling employees to quickly and accurately acquire internal knowledge through natural language dialogue. However, existing technological solutions face significant challenges when applying general-purpose large models to private, highly confidential enterprise environments:

[0003] 1. Limitations of mainstream RAG (Retrieval Augmentation) solutions: RAG solutions, represented by LangChain, import knowledge through external vector databases. While this approach is simple to deploy, it has several shortcomings:

[0004] Shallow links in knowledge: Retrieving text blocks solely through semantic similarity lacks the understanding and reasoning ability of the deep logical relationships between knowledge, which can easily lead to factual errors or logically inconsistent answers.

[0005] Poor multimodal support: It is difficult to effectively integrate and utilize structured and unstructured information in images (such as architecture diagrams and flowcharts), tables, and audio and video (such as meeting recordings).

[0006] Untimely knowledge updates: Knowledge base updates are usually done manually or in batches, which cannot reflect the dynamic changes in enterprise knowledge in real time.

[0007] 2. The dilemma of model fine-tuning solutions: Fine-tuning all or part of the parameters of a large model can "inject" knowledge into the model, but it is costly and has the problem of "catastrophic forgetting", making it difficult to adapt to the frequent updates of the enterprise knowledge base.

[0008] 3. Challenges of the long context window approach: Directly stuffing a large number of documents into the model's long context window may seem simple, but it actually faces problems such as huge computational overhead, dilution of key information ("needle in a haystack"), and a decrease in the model's reasoning ability on long sequences.

[0009] 4. The red line of data security and privacy: Enterprise internal data, especially core data such as R&D, finance, and strategy, has extremely high confidentiality and cannot be processed through public cloud APIs. It must be deployed and run in a completely private environment.

[0010] In summary, existing technologies lack a comprehensive solution that can securely, efficiently, and cost-effectively process multimodal private knowledge while ensuring real-time knowledge updates and deep reasoning capabilities. Summary of the Invention

[0011] The technical objective of this invention is to address the above-mentioned shortcomings by providing a method and system for private knowledge import and co-evolution, enabling private knowledge import and question answering, applicable to enterprise internal environments, capable of integrating multimodal data, dynamically constructing and updating knowledge graphs, and achieving co-evolution of models and knowledge bases.

[0012] The technical solution adopted by this invention to solve its technical problem is:

[0013] A method for importing and co-evolving privatized knowledge, based on multimodal fusion and dynamic knowledge graph, is proposed. The implementation of this method includes the following steps:

[0014] Data fusion: Receive private data from within the enterprise, including at least two modalities, and use a quality-aware adaptive fusion module to extract features and perform weighted fusion on the private data to generate a unified knowledge vector representation;

[0015] Knowledge graph construction and updating: Based on the knowledge vector representation, an event-driven dynamic knowledge graph construction module is used to automatically extract fact triples and use an incremental graph learning algorithm to update the existing knowledge graph in real time.

[0016] Query processing: Receive user queries, use the knowledge graph to retrieve and prune relevant knowledge subgraphs, and inject the knowledge subgraphs and user queries into a large language model;

[0017] Result generation and feedback steps: The large language model generates a response under the guidance of the knowledge subgraph and records the user's interactive feedback on the response;

[0018] Co-evolution: Based on the interactive feedback, a model-graph bidirectional feedback module is launched to adjust or correct the confidence of the triples in the knowledge graph, and to use the corrected knowledge to perform lightweight parameter updates on the large language model.

[0019] This method first scores and then merges multimodal data such as text, images, audio, and video according to clarity. Event-triggered real-time extraction of ternary data forms a dynamic knowledge graph. The graph guides the large model to focus on key facts and distills attention. User feedback provides real-time correction to the graph and low-rank fine-tuning of the model, creating a secure and private knowledge brain. Addressing the pain points of RAG's shallow retrieval, slow updates, and difficulty in fusing multimodal data, it uses quality-weighted fusion, incremental graph learning, and bidirectional graph-model evolution to both protect privacy and reduce training costs. This allows knowledge to evolve in real-time with enterprise business, resulting in more accurate and coherent answers.

[0020] Furthermore, the quality-aware adaptive fusion module specifically includes:

[0021] A quality assessment subnetwork is set up for each data modality to calculate quality scores from multiple dimensions, including sharpness, completeness, signal-to-noise ratio, etc.

[0022] The quality score is used as a dynamic weight and input into a gated recurrent unit (GRU) or attention mechanism to perform weighted fusion of feature vectors from different modalities.

[0023] Furthermore, the event-driven dynamic knowledge graph construction module specifically includes:

[0024] Monitor file creation, modification, or deletion events in an enterprise's internal document system, code repository, or communication tools;

[0025] When an event is detected, the unstructured information of the relevant document is extracted to extract entities, relationships, and events.

[0026] By employing time-sensitive graph neural network algorithms (such as Temporal Graph Networks, TGN), newly extracted triples are integrated into the existing knowledge graph, enabling incremental updates of node and relation embeddings without requiring full recalculation.

[0027] Furthermore, the incremental graph learning algorithm includes a time-sensitive graph neural network.

[0028] Furthermore, the process of generating responses by a large language model under the guidance of a knowledge subgraph is specifically a graph-guided attention distillation method, the implementation of which includes:

[0029] The entities and relations in the knowledge subgraph are encoded as special prompts or embedding vectors;

[0030] In the attention computation layer of a large language model, the attention weights associated with the cue or embedding vector are increased, forcing the model to prioritize the core facts provided by the graph when generating responses;

[0031] The attention distribution of a pre-trained teacher model with strong knowledge reasoning ability is used as a regularization term to supervise and guide the attention pattern of the current student model (i.e., the large language model) to achieve knowledge distillation.

[0032] Furthermore, the model-map bidirectional feedback module specifically includes:

[0033] Graph Validation and Correction: When a user provides negative feedback to the model's response (such as "incorrect answer"), the system identifies the key triples that the response relies on and lowers their confidence; or when a user provides a correct answer, it is updated to the graph as a new knowledge source.

[0034] Lightweight model adaptation: Knowledge is periodically sampled from subgraphs of the knowledge graph with high confidence, and large language models are continuously trained through efficient parameter fine-tuning methods to keep their internal knowledge synchronized with the graph.

[0035] Furthermore, efficient fine-tuning methods for the parameters include low-rank adaptation (LoRA) or prompt tuning.

[0036] This invention also claims a private knowledge import and co-evolution system, comprising:

[0037] A quality-aware adaptive multimodal fusion module is used to achieve data fusion;

[0038] An event-driven dynamic knowledge graph construction and incremental update module is used to implement graph construction and updating;

[0039] A long context processing module based on graph guidance and attention distillation is used to implement query processing and result generation and feedback.

[0040] The model-graph bidirectional feedback and evolution module is used to achieve the co-evolution of large-scale language models;

[0041] The system achieves private knowledge import and collaborative evolution through the methods described above.

[0042] The present invention also claims a device for implementing privatized knowledge import and co-evolution, comprising: at least one memory and at least one processor;

[0043] The at least one memory is used to store a machine-readable program;

[0044] The at least one processor is used to call the machine-readable program to implement the above method.

[0045] The present invention also claims a computer-readable medium storing computer instructions that, when executed by a processor, implement the above-described method.

[0046] Compared with existing technologies, the privatized knowledge import and co-evolution method and system of the present invention have the following advantages:

[0047] 1. High security: The entire system can be deployed entirely within the enterprise intranet, and the data does not leave the enterprise, thus completely solving data privacy and security issues.

[0048] 2. Strong comprehension ability: Through dynamic knowledge graphs, the system is no longer a simple text matching system, but is capable of deep logical reasoning and relationship mining.

[0049] 3. Timeliness: The event-driven incremental update mechanism ensures that the knowledge base keeps pace with the company's development, and the content never becomes outdated.

[0050] 4. Self-optimization: The unique collaborative evolution closed loop enables the system to continuously improve itself during use, becoming more intelligent with use. Attached Figure Description

[0051] Figure 1 This is a diagram illustrating the architecture of a method for importing and co-evolving private knowledge according to an embodiment of the present invention;

[0052] Figure 2 This is a flowchart illustrating a method for importing and co-evolving private knowledge according to an embodiment of the present invention. Detailed Implementation

[0053] The present invention will be further described below with reference to specific embodiments.

[0054] This invention provides a method for private knowledge import and co-evolution, which achieves private knowledge import and co-evolution based on multimodal fusion and dynamic knowledge graphs. This addresses issues in existing enterprise knowledge management systems such as high data privacy risks, lagging knowledge updates, insufficient multimodal information processing capabilities, and insufficient model interaction depth. The method proposes a knowledge co-evolution framework, which includes: a quality-aware adaptive multimodal fusion module for processing heterogeneous data such as text, images, and audio / video, and dynamically adjusting fusion weights based on data quality; an event-driven dynamic knowledge graph construction and incremental update module for real-time capture of internal knowledge changes and efficient updating of the knowledge graph using incremental graph learning algorithms; a long context processing module based on graph guidance and attention distillation, which uses the structured information of the knowledge graph to guide the attention allocation of large language models, improving the accuracy and efficiency of long text understanding; and a model-graph bidirectional feedback and evolution module, which verifies and corrects the knowledge in the knowledge graph by analyzing the model's inference results and user feedback, and simultaneously uses the corrected graph knowledge to perform lightweight adaptation of the model, forming a closed-loop self-consistent improvement of system capabilities.

[0055] The implementation of this method includes the following steps:

[0056] Data fusion: Receive private data from within the enterprise, including at least two modalities, and use a quality-aware adaptive fusion module to extract features and perform weighted fusion on the private data to generate a unified knowledge vector representation;

[0057] Knowledge graph construction and updating: Based on the knowledge vector representation, an event-driven dynamic knowledge graph construction module is used to automatically extract fact triples and use incremental graph learning algorithms (such as time-sensitive graph neural networks) to update the existing knowledge graph in real time.

[0058] Query processing: Receive user queries, use the knowledge graph to retrieve and prune relevant knowledge subgraphs, and inject the knowledge subgraphs and user queries into a large language model;

[0059] Result generation and feedback steps: The large language model generates a response under the guidance of the knowledge subgraph and records the user's interactive feedback on the response;

[0060] Co-evolution: Based on the interactive feedback, a model-graph bidirectional feedback module is launched to adjust or correct the confidence of the triples in the knowledge graph, and to use the corrected knowledge to perform lightweight parameter updates on the large language model.

[0061] The quality-aware adaptive fusion module specifically includes:

[0062] A quality assessment subnetwork is set up for each data modality to calculate quality scores from multiple dimensions, including sharpness, completeness, signal-to-noise ratio, etc.

[0063] The quality score is used as a dynamic weight and input into a gated recurrent unit (GRU) or attention mechanism to perform weighted fusion of feature vectors from different modalities.

[0064] The event-driven dynamic knowledge graph construction module specifically includes:

[0065] Monitor file creation, modification, or deletion events in an enterprise's internal document system, code repository, or communication tools;

[0066] When an event is detected, the unstructured information of the relevant document is extracted to extract entities, relationships, and events.

[0067] By employing time-sensitive graph neural network algorithms (such as Temporal Graph Networks, TGN), newly extracted triples are integrated into the existing knowledge graph, enabling incremental updates of node and relation embeddings without requiring full recalculation.

[0068] In the query processing and result generation and feedback steps, the process of generating responses by a large language model under the guidance of a knowledge subgraph is specifically a graph-guided attention distillation method, the implementation of which includes:

[0069] The entities and relations in the knowledge subgraph are encoded as special prompts or embedding vectors;

[0070] In the attention computation layer of a large language model, the attention weights associated with the cue or embedding vector are increased, forcing the model to prioritize the core facts provided by the graph when generating responses;

[0071] The attention distribution of a pre-trained teacher model with strong knowledge reasoning ability is used as a regularization term to supervise and guide the attention pattern of the current student model (i.e., the large language model) to achieve knowledge distillation.

[0072] The model-map bidirectional feedback module specifically includes:

[0073] Graph Validation and Correction: When a user provides negative feedback to the model's response (such as "incorrect answer"), the system identifies the key triples that the response relies on and lowers their confidence; or when a user provides a correct answer, it is updated to the graph as a new knowledge source.

[0074] Lightweight model adaptation: Knowledge is periodically sampled from subgraphs of the knowledge graph with high confidence. Large language models are continuously trained using efficient fine-tuning methods such as low-rank adaptation (LoRA) or prompt tuning, so that their internal knowledge keeps in sync with the graph.

[0075] The method will be described in further detail below with specific examples, but the implementation of the method is not limited to this.

[0076] The overall system architecture for implementing this method consists of four main parts: a data access layer, a knowledge processing layer, an application interaction layer, and a co-evolution layer, forming a complete closed loop of data flow and control flow.

[0077] Implementation Case: A complete internal enterprise knowledge Q&A process.

[0078] Suppose that Xiao Wang, a developer at a software company, asks the system: "What are the latest architecture design diagrams and related performance bottlenecks for 'Project Sky Dome'?"

[0079] Step 1: Multimodal data access and adaptive fusion.

[0080] 1. The system continuously monitors the company's internal Confluence (documentation), GitLab (code), and shared drives. It recently discovered a document on Confluence titled "Sky Dome Project V2.0 Architecture Design," which includes a text description and a PNG architecture diagram. Simultaneously, there were code commit records related to performance optimization on GitLab.

[0081] 2. Quality perception fusion module starts:

[0082] Text processing: Extract text from the document and encode it into text feature vectors using models such as BERT.

[0083] Image processing: Image feature vectors of the architecture diagram are extracted using CLIP or a similar visual model. Simultaneously, a lightweight image quality assessment network (such as a CNN-based sharpness classifier) ​​analyzes the PNG image, deeming it "highly sharp and with distinct elements," and assigns a high quality score of wimage=0.95.

[0084] Code processing: Analyze comments and changes in code commit history, extract key information, and encode it into code feature vectors.

[0085] Adaptive fusion: This process fuses feature vectors from text, images, and code using an attention mechanism. Due to the high quality score of images, their features receive higher attention weights during fusion, ensuring that information in the architecture graph is fully utilized. The result is a unified multimodal knowledge vector.

[0086] Step 2: Event-driven incremental update of the knowledge graph.

[0087] 1. Since the "Sky Dome Project V2.0 Architecture Design" is a new document, the event-driven module captures this "creation" event and triggers the knowledge extraction process.

[0088] 2. The system extracts triples from the fused multimodal knowledge vector using a joint extraction model, such as:

[0089] (Sky Dome Project, has_architecture, Architecture Diagram_V2.0)

[0090] (Architecture Diagram_V2.0, contains_component, Message Queue Kafka)

[0091] (Message queue Kafka, has_performance_issue, data backlog)

[0092] (Sky Dome Project, related_commit, commit_id_xyz)

[0093] 3. Incremental Graph Learning Module Launch: The system uses the Temporal Graph Network (TGN) algorithm to integrate these new triples and their timestamps (document creation times) into the existing enterprise knowledge graph. TGN updates the embedding representations of nodes such as "Sky Dome Project" and "Kafka Message Queue" without recalculating the entire graph, making the process highly efficient.

[0094] Step 3: Graph-guided query processing.

[0095] 1. The system received Xiao Wang's question. First, the question was analyzed to identify the core entity "Sky Dome Project" and the query intent "architecture design" and "performance bottleneck".

[0096] 2. In the dynamic knowledge graph, the system traverses and searches with "Sky Dome Project" as the central node to quickly find a knowledge subgraph that contains the aforementioned new knowledge.

[0097] 3. The system combines this structured subgraph (instead of the original long document) with the user question to form a special input, which is then fed into a large language model (such as a privately deployed Llama-3).

[0098] Step 4: Attention distillation and result generation.

[0099] 1. Within LLM, a graph-guided attention distillation mechanism comes into play. Entities and relations in the subgraph are encoded as special tokens. In the Transformer's self-attention layer, computations associated with these special tokens receive higher weights.

[0100] 2. When generating the answer, the model focuses heavily on key information such as "Architecture Diagram_V2.0" and "Data Backlog," ultimately generating the response: "For the latest architecture design of 'Project Sky Dome,' please refer to the architecture diagram ([attached image]) in the document 'Project Sky Dome V2.0 Architecture Design.' According to the latest analysis, the main performance bottleneck lies in the potential data backlog issue in the message queue Kafka; related discussions can be found in GitLab's commit_id_xyz."

[0101] Step 5: Co-evolution.

[0102] 1. After reading the answer, Xiao Wang clicked the "Answer is very helpful" button.

[0103] 2. The model-graph bidirectional feedback module receives this positive feedback signal. The system will increase the confidence score of the triples (message queue Kafka, has_performance_issue, data backlog) on ​​which this answer depends.

[0104] 3. Suppose another senior architect, Mr. Li, sees the answer and adds a comment: "The root cause of data backlog is insufficient downstream service consumption capacity."

[0105] The system captures this comment, parses it, and uses it as a new knowledge source. After review, it adds a new triple to the knowledge graph: (data backlog, caused_by, insufficient downstream service consumption capacity). The graph evolves.

[0106] 4. During a nighttime maintenance window, the system initiated the lightweight model adaptation process. It sampled a batch of high-confidence knowledge from the knowledge graph (including new knowledge supplemented by Engineer Li) and generated question-answer pairs. Then, using LoRA technology, it performed dozens of fine-tuning steps on the LLM. In this way, the model's "intrinsic knowledge" was updated, and the model evolved. The next time someone asks a similar question, the model may be able to directly answer the root cause.

[0107] The above implementation examples demonstrate how this method achieves secure import, real-time updating, and deep understanding of multimodal knowledge through a complete closed loop, ultimately leading to the collaborative evolution of system capabilities.

[0108] This method can build a secure, efficient, and self-optimizing private knowledge brain, with broad application prospects in areas such as enterprise knowledge Q&A, intelligent decision support, and risk management. It can be implemented as an intranet Q&A platform, decision support system, and risk control assistant. Components include a quality assessment network, a TGN graph update engine, and a LoRA fine-tuning module, helping companies transform scattered documents into a thinking knowledge base, saving manpower, reducing the risk of data leakage, and continuously adding value.

[0109] This invention also provides a private knowledge import and co-evolution system, comprising:

[0110] A quality-aware adaptive multimodal fusion module is used to achieve data fusion.

[0111] The event-driven dynamic knowledge graph construction and incremental update module is used to implement graph construction and updates.

[0112] A long context processing module based on graph guidance and attention distillation is used to implement query processing and result generation and feedback.

[0113] The model-graph bidirectional feedback and evolution module is used to achieve the co-evolution of large language models.

[0114] The quality-aware adaptive multimodal fusion module is responsible for processing diverse data from various sources and formats within the enterprise. Instead of simply splicing different modalities together, it introduces a quality assessment mechanism that dynamically evaluates the reliability of input data (such as image clarity, text integrity, and audio signal-to-noise ratio) and adaptively adjusts their weights in the fusion process accordingly. This ensures that high-quality information dominates knowledge construction and suppresses the influence of low-quality or noisy data.

[0115] Through an event-driven dynamic knowledge graph construction and incremental update module, the system can monitor internal knowledge sources (such as Git commits, Confluence page updates, and new files in shared folders) in real time. Once a change occurs, the system immediately triggers the knowledge extraction and graph update process. More importantly, the system employs advanced incremental graph learning algorithms (such as time-sensitive graph neural networks), enabling new knowledge to be seamlessly integrated into the existing graph, updating only the affected local areas, greatly improving update efficiency and ensuring the real-time nature of knowledge.

[0116] An innovative graph-guided approach is proposed for long context processing modules based on graph guidance and attention distillation. When processing user queries, the system first retrieves a highly relevant "knowledge subgraph" from the knowledge graph. This structured subgraph is then used to "guide" the attention mechanism of the main model, forcing the model to focus on key entities and relationships contained within the subgraph. Furthermore, through attention distillation, a more powerful "teacher model" with stronger knowledge reasoning capabilities instructs the current model on how to allocate attention, further improving the accuracy and efficiency of the responses.

[0117] The model-graph bidirectional feedback and evolution module is the core of achieving "co-evolution." It includes:

[0118] Model feedback to the knowledge graph: After the model answers questions based on the knowledge graph, user feedback (such as likes and corrections) becomes a signal to verify the correctness of the knowledge graph. For knowledge that is proven to be incorrect, the system will automatically lower its confidence level in the knowledge graph or correct it.

[0119] The knowledge graph nourishes the model: The system periodically feeds back high-confidence, stable knowledge from the knowledge graph to the large model through parameter-efficient fine-tuning techniques (such as LoRA). This allows the model to continuously and cost-effectively absorb new knowledge without forgetting old knowledge, achieving synchronous evolution between the model and the knowledge base.

[0120] This system implements private knowledge import and co-evolution using the private knowledge import and co-evolution method described in the above embodiments. The specific steps are as follows:

[0121] Data fusion: Receive private data from within the enterprise, including at least two modalities, and use a quality-aware adaptive fusion module to extract features and perform weighted fusion on the private data to generate a unified knowledge vector representation;

[0122] Knowledge graph construction and updating: Based on the knowledge vector representation, an event-driven dynamic knowledge graph construction module is used to automatically extract fact triples and use incremental graph learning algorithms (such as time-sensitive graph neural networks) to update the existing knowledge graph in real time.

[0123] Query processing: Receive user queries, use the knowledge graph to retrieve and prune relevant knowledge subgraphs, and inject the knowledge subgraphs and user queries into a large language model;

[0124] Result generation and feedback steps: The large language model generates a response under the guidance of the knowledge subgraph and records the user's interactive feedback on the response;

[0125] Co-evolution: Based on the interactive feedback, a model-graph bidirectional feedback module is launched to adjust or correct the confidence of the triples in the knowledge graph, and to use the corrected knowledge to perform lightweight parameter updates on the large language model.

[0126] The quality-aware adaptive fusion module specifically includes:

[0127] A quality assessment subnetwork is set up for each data modality to calculate quality scores from multiple dimensions, including sharpness, completeness, signal-to-noise ratio, etc.

[0128] The quality score is used as a dynamic weight and input into a gated recurrent unit (GRU) or attention mechanism to perform weighted fusion of feature vectors from different modalities.

[0129] The event-driven dynamic knowledge graph construction module specifically includes:

[0130] Monitor file creation, modification, or deletion events in an enterprise's internal document system, code repository, or communication tools;

[0131] When an event is detected, the unstructured information of the relevant document is extracted to extract entities, relationships, and events.

[0132] By employing time-sensitive graph neural network algorithms (such as Temporal Graph Networks, TGN), newly extracted triples are integrated into the existing knowledge graph, enabling incremental updates of node and relation embeddings without requiring full recalculation.

[0133] In the query processing and result generation and feedback steps, the process of generating responses by a large language model under the guidance of a knowledge subgraph is specifically a graph-guided attention distillation method, the implementation of which includes:

[0134] The entities and relations in the knowledge subgraph are encoded as special prompts or embedding vectors;

[0135] In the attention computation layer of a large language model, the attention weights associated with the cue or embedding vector are increased, forcing the model to prioritize the core facts provided by the graph when generating responses;

[0136] The attention distribution of a pre-trained teacher model with strong knowledge reasoning ability is used as a regularization term to supervise and guide the attention pattern of the current student model (i.e., the large language model) to achieve knowledge distillation.

[0137] The model-map bidirectional feedback module specifically includes:

[0138] Graph Validation and Correction: When a user provides negative feedback to the model's response (such as "incorrect answer"), the system identifies the key triples that the response relies on and lowers their confidence; or when a user provides a correct answer, it is updated to the graph as a new knowledge source.

[0139] Lightweight model adaptation: Knowledge is periodically sampled from subgraphs of the knowledge graph with high confidence. Large language models are continuously trained using efficient fine-tuning methods such as low-rank adaptation (LoRA) or prompt tuning, so that their internal knowledge keeps in sync with the graph.

[0140] This invention also provides a device for implementing privatized knowledge import and co-evolution, comprising: at least one memory and at least one processor;

[0141] The at least one memory is used to store a machine-readable program;

[0142] The at least one processor is used to call the machine-readable program to implement the private knowledge import and co-evolution method described in the above embodiments.

[0143] This invention also provides a computer-readable medium storing computer instructions. When executed by a processor, the computer instructions cause the processor to perform the privatized knowledge import and co-evolution method described in the above embodiments. Specifically, a system or apparatus equipped with a storage medium storing software program code that implements the functions of any of the above embodiments can be provided, and the computer (or CPU or MPU) of the system or apparatus can read and execute the program code stored in the storage medium.

[0144] In this case, the program code read from the storage medium can itself implement the function of any of the above embodiments, and therefore the program code and the storage medium storing the program code constitute part of the present invention.

[0145] Storage media embodiments for providing program code include floppy disks, hard disks, magneto-optical disks, optical disks (such as CD-ROM, CD-R, CD-RW, DVD-ROM, DVD-RAM, DVD-RW, DVD+RW), magnetic tapes, non-volatile memory cards, and ROMs. Alternatively, program code can be downloaded from a server computer via a communication network.

[0146] Furthermore, it should be clear that not only can the program code read by the computer be executed, but also the operating system or other components operating on the computer can be instructed based on the program code to perform some or all of the actual operations, thereby realizing the function of any of the embodiments described above.

[0147] Furthermore, it is understood that the program code read from the storage medium is written to the memory set in the expansion board inserted into the computer or to the memory set in the expansion unit connected to the computer. Then, based on the instructions of the program code, the CPU or other components installed on the expansion board or expansion unit execute some and all of the actual operations, thereby realizing the function of any of the embodiments described above.

[0148] The present invention has been shown and described in detail above with reference to the accompanying drawings and preferred embodiments. However, the present invention is not limited to these disclosed embodiments. Based on the above embodiments, those skilled in the art will know that more embodiments of the present invention can be obtained by combining the code review methods in the different embodiments. These embodiments are also within the protection scope of the present invention.

Claims

1. A method for privatizing knowledge importation and co-evolution, comprising: The private knowledge import and co-evolution are realized based on multi-modal fusion and dynamic knowledge graph, and the implementation of the method includes the following steps: Data fusion: receiving private data of at least two modalities in the enterprise, performing feature extraction and weighted fusion on the private data through a quality perception-based adaptive fusion module, and generating a unified knowledge vector representation; Graph construction and updating: based on the knowledge vector representation, automatically extracting fact triples through an event-driven dynamic knowledge graph construction module, and updating the existing knowledge graph in real time using an incremental graph learning algorithm; Query processing: receiving user queries, searching and pruning related knowledge sub-graphs using the knowledge graph, and injecting the knowledge sub-graphs and user queries into a large language model; Result generation and feedback: the large language model generates a reply under the guidance of the knowledge sub-graph, and records the user's interactive feedback on the reply; Co-evolution: based on the interactive feedback, starting the model-graph bidirectional feedback module, adjusting or correcting the confidence of the triples in the knowledge graph, and updating the parameters of the large language model using the corrected knowledge; The process of generating a reply by a large language model under the guidance of a knowledge sub-graph is specifically a graph-guided attention distillation method, which includes the following steps: Encode the entities and relationships in the knowledge sub-graph into special prompts or embedding vectors; In the attention calculation layer of the large language model, increase the attention weight related to the prompts or embedding vectors to force the model to pay more attention to the core facts provided by the graph when generating a reply; The attention distribution of a pre-trained teacher model with strong knowledge reasoning ability is used as a regularization term to supervise and guide the attention pattern of the current student model, achieving knowledge distillation.

2. A method of privatizing knowledge import and co-evolution as claimed in claim 1, wherein, The quality perception-based adaptive fusion module specifically includes: Set a quality evaluation subnetwork for each data modality to calculate a quality score from multiple dimensions including clarity, integrity, and signal-to-noise ratio; Input the quality score as a dynamic weight into a gated recurrent unit or attention mechanism to perform weighted fusion on the feature vectors of different modalities.

3. A method of privatizing knowledge import and co-evolution as claimed in claim 1, wherein, The event-driven dynamic knowledge graph construction module specifically includes: Listen to file creation, modification, or deletion events in the enterprise's internal document system, code repository, or communication tools; When an event is detected, trigger entity, relationship, and event extraction on the unstructured information in the related document; Use a time-sensitive graph neural network algorithm to integrate the newly extracted triples into the existing knowledge graph, achieving incremental update of node and relationship embedding without full recalculation.

4. The method of claim 1, wherein the method is a method of privatizing knowledge import and co-evolution, characterized in that, The incremental graph learning algorithm includes a time-sensitive graph neural network.

5. A method of privatizing knowledge import and co-evolution as claimed in claim 1, wherein, The model-graph bidirectional feedback module specifically includes: Graph verification and correction: when the user provides negative feedback on the model's reply, the system locates the key triples on which the reply is based and reduces their confidence; or when the user provides the correct answer, update it to the graph as new knowledge source; Model lightweight adaptation: periodically sample knowledge from high-confidence knowledge graph subgraphs, continuously train large language models through parameter-efficient fine-tuning methods to keep their internal knowledge synchronized with the graph.

6. A method of privatizing knowledge import and co-evolution as claimed in claim 1, wherein, The parameter-efficient fine-tuning method includes low-rank adaptation or prompt fine-tuning.

7. A privatized knowledge importation and co-evolution system, comprising: It includes: An adaptive multi-modal fusion module based on quality perception is used to realize data fusion. An event-driven dynamic knowledge graph construction and incremental update module is used to realize graph construction and update. A long context processing module based on graph guidance and attention distillation is used to realize query processing and result generation and feedback. A model-graph bidirectional feedback and evolution module is used to realize the co-evolution of large language models. The system realizes the import and co-evolution of private knowledge through the method of any one of claims 1-6.

8. A private knowledge importation and co-evolution implementation apparatus, characterized by, It includes: At least one memory and at least one processor; The at least one memory is used to store machine readable programs; The at least one processor is used to call the machine readable programs to realize the method of any one of claims 1-6.

9. A computer readable medium characterized by The computer readable medium stores computer instructions, and the computer instructions are executed by the processor to realize the method of any one of claims 1-6.

Images