Knowledge base dynamic construction and intelligent retrieval method based on large model

Abstract

The invention discloses a knowledge base dynamic construction and intelligent retrieval method based on a large model, and relates to the technical field of computer retrieval, a DeepSeek large model is taken as a teacher model, a lightweight student model is trained through a specialized knowledge distillation strategy, and local deployment is realized in combination with a containerization technology; the specialized knowledge distillation strategy comprises dynamic modal perception distillation, knowledge graph structure guide distillation and online incremental distillation; according to the method, the overall process efficiency is remarkably improved compared with a traditional scheme through multi-modal data automatic processing and lightweight model local deployment; cross-modal semantic alignment and joint reasoning are realized, and the accuracy of retrieval and answer generation is improved; a real-time updating mechanism and version management enable knowledge to always meet actual requirements, and the logic contradiction occurrence rate is remarkably reduced; the fact reliability of the generated content is greatly improved by a fact anchoring and double-stage verification mechanism; local deployment and containerization isolation design avoid leakage of sensitive information, and meet the requirements of data security compliance.

Description

Technical Field

[0001] This invention relates to the field of computer retrieval technology, and in particular to a method for dynamic construction and intelligent retrieval of knowledge bases based on large models. Background Technology

[0002] With the advancement of digital transformation, enterprises and institutions are accumulating a continuously expanding scale of multimodal data, including text, images, audio, and video, making the demand for efficient knowledge management and intelligent retrieval increasingly urgent. Existing technologies have many shortcomings in processing multimodal knowledge: Multimodal data preprocessing relies heavily on manual operations, resulting in inefficient cleaning, labeling, and feature extraction processes that struggle to meet the demands for rapid processing of massive datasets. Large-scale model deployments often depend on the cloud, posing risks of sensitive data leakage. Furthermore, knowledge distillation strategies not optimized for multimodal scenarios lead to significant performance losses in lightweight models, failing to balance deployment flexibility with inference accuracy. Knowledge graphs are mostly statically constructed, lacking real-time update mechanisms, making it difficult to adapt to dynamic data changes. Moreover, conflict resolution methods are simplistic, failing to guarantee knowledge consistency. The shallow integration of multimodal data results in a lack of unified semantic relationships between different modalities, leading to weak cross-modal retrieval and inference capabilities. During retrieval enhancement generation, the segmentation strategy lacks specificity, the hybrid retrieval mechanism is incomplete, and there are insufficient measures to suppress answer illusions, impacting the reliability of output results. Knowledge update mechanisms fail to effectively combine event-driven and batch scheduling, easily leading to update conflicts in high-concurrency scenarios. They also lack coordination with incremental model learning, hindering the synchronous optimization of knowledge and the model.

[0003] These problems mean that existing knowledge base systems cannot meet the needs of practical applications in terms of efficiency, accuracy, dynamic adaptability, and security. There is an urgent need for a technical solution that balances efficient processing, deep integration, dynamic updates, and accurate generation. Summary of the Invention

[0004] To address the aforementioned technical problems, this invention provides a method for dynamic construction and intelligent retrieval of a knowledge base based on a large model. The technical solution adopted is as follows: A method for dynamic construction and intelligent retrieval of knowledge bases based on large models includes the following steps: Step 1: Collect multimodal data (text, images, audio, and video) from multiple sources, and use a large model to automatically clean, standardize, label, classify, and extract features to form a multimodal data pool; Step 2: Using the DeepSeek large model as the teacher model, train a lightweight student model through a specialized knowledge distillation strategy, and combine it with containerization technology to achieve local deployment; the specialized knowledge distillation strategy includes dynamic modality-aware distillation, knowledge graph structure-guided distillation, and online incremental distillation; Step 3: Dynamic multimodal knowledge graph construction. Utilize lightweight large models to extract entities, relationships, and attributes from multimodal data to construct a knowledge graph. Set up a real-time dynamic update mechanism and implement version control and conflict resolution. Step 4: Integrate multimodal data into a unified semantic space through cross-modal semantic mapping, and integrate large models and rule engines to achieve joint reasoning; Step 5: Perform dynamic semantic segmentation and hybrid retrieval based on Ragflow, combine large model context to generate and suppress answer illusion; Step 6: Adopt an update mechanism that combines event-driven and batch scheduling, utilize large models and distributed computing to maintain knowledge consistency, build a multimodal retrieval interface, optimize system performance, and iteratively optimize the model through user feedback.

[0005] Optionally, in step 2, dynamic modality-aware distillation specifically involves: for different modal data such as text, image, and audio, using the entity relationship attention weights and visual feature maps of the teacher model as soft labels, and training the student model through a modality-adaptive loss function to make the student model adapt to the core knowledge of each modality.

[0006] Optionally, the knowledge graph structure-guided distillation in step 2 specifically involves using the knowledge graph constructed in step 3 as a structured knowledge anchor point. During the distillation process, the entity connectivity and relation confidence of the graph are introduced as weighting factors. The knowledge corresponding to core entities with high connectivity is strengthened during distillation, while the distillation weight is reduced for fuzzy relations with low confidence, so that the student model inherits the inherent structural logic of the knowledge.

[0007] Optionally, the online incremental distillation mechanism described in step 2 is as follows: when a new node or relationship is added to the knowledge graph, an incremental distillation subtask is triggered. The teacher model generates incremental soft labels only for the new knowledge fragments. By freezing the old knowledge layer of the student model and fine-tuning the newly added knowledge layer, the student model can absorb new knowledge without forgetting the old knowledge.

[0008] Optionally, the deployment of the lightweight student model in step 2 adopts the Ollam framework and Docker containerization technology, combined with a modal and task dual-trigger loading mechanism, that is, when a specific modal query is received, only the corresponding model sub-module is activated.

[0009] Optionally, the multiple sources mentioned in step 1 include enterprise documents, databases, IoT sensors, and web crawler resources; the automatic cleaning using large models includes removing noise, duplicates, and irrelevant content, and unifying the data format includes converting audio to text and extracting features from images.

[0010] Optionally, the construction of the dynamic multimodal knowledge graph in step 3 specifically includes: The knowledge graph is stored using Neo4j graph database. The graph consists of entities, attributes and relationships extracted from multimodal data by a lightweight large model, and supports multi-hop association queries between entities. By monitoring file system changes, database transaction logs, and message queue pushes, data addition or modification events are captured in real time, triggering a lightweight large model to re-analyze the changed data and automatically performing knowledge graph update operations, including adding entity nodes, deleting outdated attributes, and adjusting the weights of relationship edges. A version chain and timestamp management mechanism is introduced to maintain historical versions for each node and relationship in the knowledge graph, supporting time-based tracing and rollback. For knowledge conflicts, a lightweight large model is used to resolve conflicts by combining confidence analysis and user feedback: the large model first scores the reliability of the source and semantic consistency of the conflicting information. If the confidence difference exceeds the threshold, the high-confidence information is automatically adopted; if the difference is below the threshold, the user feedback interface is triggered, and the graph is updated after manual confirmation.

[0011] Optionally, the construction of the multimodal fusion and joint inference engine in step 4 specifically includes: By employing the CLIP cross-modal encoder or a custom multimodal encoder based on DeepSeek, text, image, and audio data are mapped to a unified semantic embedding space, enabling computationally achievable semantic similarity among different modalities. A two-layer joint reasoning engine combining semantic retrieval and logical reasoning is constructed: the first layer performs similarity retrieval on the user's multimodal queries in a unified semantic space to obtain relevant entities and multimodal data fragments from the knowledge graph; the second layer uses a lightweight large model to perform contextual logical reasoning based on the retrieval results to generate natural language answers containing multimodal associations. This approach associates objects in images with technical documents described in text, and voice commands and operational process nodes in audio, enabling cross-modal knowledge linking of images, text, and audio, and supporting multimodal joint retrieval.

[0012] Optionally, the intelligent optimizations for enhanced generation in step 5 specifically include: Dynamic semantic chunking is implemented based on the Ragflow framework. The chunking strategy combines sentence boundaries, topic clustering, and knowledge graph entity density: text paragraphs containing dense entities are segmented by entity groups, and narrative texts are segmented by plot topics, ensuring the semantic integrity of the chunks and the targeting of retrieval. Text or feature vector similarity retrieval is performed using FAISS, keyword exact matching is performed using Elasticsearch, and multi-hop path retrieval of knowledge graphs is performed using Neo4j. The results of multi-source retrieval are weighted and fused according to confidence level. During the generation phase, fact anchors are inserted into the prompts. These fact anchors include knowledge graph entity IDs and authoritative document sources, forcing the large model to anchor the generation of facts. After generation, the system undergoes a two-stage verification process: self-verification by a large model and verification based on user feedback. Answers that appear to be hallucinations are automatically recalled and regenerated.

[0013] Optionally, step 6, knowledge updating and consistency maintenance under high concurrency, specifically includes: For data sources with high real-time requirements, event-driven instant updates are achieved through Kafka message queues; for batch historical data, batch updates are performed through scheduled tasks, and the two types of update tasks avoid concurrency conflicts through a distributed lock mechanism. The lightweight large model uses incremental learning technology, which freezes the weights of the old knowledge layer when updating new knowledge, and only fine-tunes the new knowledge layer. At the same time, it automatically aligns the model's newly added entity or relation predictions with the knowledge graph to ensure the consistency between the model output and the graph structure. This implementation leverages the Spark distributed computing framework to achieve parallel updates and consistency checks for large-scale knowledge graphs: the graph is partitioned by entities, and the addition and deletion operations of nodes or relations are performed in a distributed manner; a lightweight large model performs a global logical consistency check on the distributed updated graph, and the check results are automatically corrected by the knowledge graph's constraint rule engine. By collecting user feedback on the accuracy of knowledge and the relevance of retrieval, and adjusting the weights of hybrid retrieval, the prompt word strategy generated by the large model, and the update priority of the knowledge graph through reinforcement learning, the system achieves self-iterative optimization.

[0014] In summary, the present invention has at least one of the following beneficial technical effects: This invention provides a method for dynamic construction and intelligent retrieval of knowledge bases based on large models. Automated processing of multimodal data and local deployment of lightweight models significantly improve the overall process efficiency compared to traditional solutions. It achieves cross-modal semantic alignment and joint reasoning, improving the accuracy of retrieval and answer generation. Real-time update mechanisms and version management ensure that knowledge always meets actual needs, significantly reducing the incidence of logical contradictions. Fact anchoring and two-stage verification mechanisms greatly improve the factual reliability of generated content. Local deployment and containerized isolation design prevent the leakage of sensitive information and meet data security compliance requirements. Attached Figure Description

[0015] Figure 1 This is a flowchart illustrating the method for dynamic construction and intelligent retrieval of a knowledge base based on a large model, as described in this invention. Figure 2 This is a schematic diagram of the knowledge graph for the operation and maintenance of intelligent substation equipment in this invention. Detailed Implementation

[0016] The present invention will be further described in detail below with reference to the accompanying drawings.

[0017] This invention discloses a method for dynamic construction and intelligent retrieval of a knowledge base based on a large model.

[0018] Reference Figure 1 and Figure 2 Example 1, a method for dynamic construction and intelligent retrieval of a knowledge base based on a large model, includes the following steps: Step 1: Collect multimodal data (text, images, audio, and video) from multiple sources, and use a large model to automatically clean, standardize, label, classify, and extract features to form a multimodal data pool; Step 2: Using the DeepSeek large model as the teacher model, train a lightweight student model through a specialized knowledge distillation strategy, and combine it with containerization technology to achieve local deployment; the specialized knowledge distillation strategy includes dynamic modality-aware distillation, knowledge graph structure-guided distillation, and online incremental distillation; Step 3: Dynamic multimodal knowledge graph construction. Utilize lightweight large models to extract entities, relationships, and attributes from multimodal data to construct a knowledge graph. Set up a real-time dynamic update mechanism and implement version control and conflict resolution. Step 4: Integrate multimodal data into a unified semantic space through cross-modal semantic mapping, and integrate large models and rule engines to achieve joint reasoning; Step 5: Perform dynamic semantic segmentation and hybrid retrieval based on Ragflow, combine large model context to generate and suppress answer illusion; Step 6: Adopt an update mechanism that combines event-driven and batch scheduling, utilize large models and distributed computing to maintain knowledge consistency, build a multimodal retrieval interface, optimize system performance, and iteratively optimize the model through user feedback.

[0019] By adopting the above technical solution, the core principle of this technical solution is to solve the shortcomings of traditional knowledge bases in terms of efficiency, adaptability, accuracy and security through the collaborative design of deep fusion of multimodal data, lightweight optimization of large models, dynamic knowledge construction and enhanced retrieval, and achieve efficient, intelligent and secure knowledge management and retrieval.

[0020] The core principle of multimodal data processing is to leverage the multimodal understanding capabilities of large models to automatically complete data cleaning, standardization, annotation, and feature extraction. After unified processing, data from different modalities are formed into a data pool containing structured, semi-structured, and unstructured data, providing a high-quality, multi-dimensional data source for subsequent knowledge construction and avoiding the inefficiency and errors of manual processing.

[0021] The principle behind lightweight deployment of large models is to achieve a balance between performance and deployment flexibility based on specialized knowledge distillation strategies. Dynamic modality-aware distillation designs specific distillation objectives for the characteristics of different modalities, enabling student models to accurately learn the core knowledge of each modality. Knowledge graph-guided distillation uses structured knowledge as an anchor, allowing student models to inherit the inherent logic of knowledge rather than merely imitating surface output. Online incremental distillation adapts to the needs of dynamic knowledge updates, achieving the absorption of new knowledge and the retention of old knowledge through layered freezing and fine-tuning. Containerized local deployment, through environment isolation and modular loading, reduces resource consumption while avoiding the privacy leakage risks associated with cloud deployment.

[0022] The principle behind constructing a dynamic multimodal knowledge graph is to leverage the efficient reasoning capabilities of lightweight, large models to extract entities, relationships, and attributes from multimodal data, thus building a knowledge graph with multi-dimensional connections. A real-time dynamic update mechanism triggers model re-analysis by monitoring data source changes, enabling instant iteration of the knowledge graph; version control and timestamp management ensure the traceability of knowledge; and a conflict resolution mechanism that integrates confidence analysis and user feedback, through a combination of automated machine judgment and human intervention, ensures the logical consistency of the knowledge graph.

[0023] The principle of multimodal fusion and joint reasoning is to map data from different modalities to a unified semantic space through a cross-modal encoder, breaking down modal barriers and enabling multimodal data to have computable semantic relevance. The two-layer joint reasoning engine first obtains relevant knowledge graph entities and data fragments through semantic retrieval, and then the large model performs contextual logic reasoning to realize the associative invocation of multimodal knowledge and generate an integrated answer that fits the query requirements.

[0024] The principle behind enhanced retrieval optimization is to use a dynamic semantic segmentation strategy, combining sentence boundaries, topic clustering, and entity density, to ensure the semantic integrity and retrieval relevance of text segments. The hybrid retrieval mechanism integrates semantic similarity matching from vector retrieval, precise matching from keyword retrieval, and multi-hop path retrieval from knowledge graphs, improving knowledge recall and comprehensiveness. The core principle of illusion suppression is to force the model to anchor to authoritative knowledge sources through factual anchoring, followed by a two-stage mechanism of large-scale model self-verification and user feedback verification to filter and correct generated content that deviates from facts, ensuring the reliability of the answers.

[0025] The principle behind knowledge updates and system optimization employs a hybrid update mechanism combining event-driven and batch scheduling to adapt to the update needs of both real-time and batch historical data. A distributed lock mechanism avoids update conflicts in high-concurrency scenarios. A distributed computing framework enables parallel updates and consistency checks of large-scale knowledge graphs, improving processing efficiency. Incremental learning of lightweight models and automatic alignment with the knowledge graph ensure consistency between model output and knowledge structure. User feedback, through reinforcement learning, adjusts retrieval weights, generation strategies, and knowledge update priorities, forming a closed-loop iteration of data, knowledge, models, and feedback to continuously improve system performance.

[0026] In Example 2, the dynamic modality-aware distillation in step 2 specifically involves: for different modal data such as text, image, and audio, the entity relationship attention weights and visual feature maps of the teacher model are used as soft labels, and the student model is trained through a modality-adaptive loss function to make the student model adapt to the core knowledge of each modality.

[0027] In Example 3, the knowledge graph structure-guided distillation in step 2 specifically involves using the knowledge graph constructed in step 3 as a structured knowledge anchor point. During the distillation process, the entity connectivity and relation confidence of the graph are introduced as weighting factors. The knowledge corresponding to core entities with high connectivity is strengthened during distillation, while the distillation weight is reduced for fuzzy relations with low confidence, so that the student model inherits the inherent structural logic of the knowledge.

[0028] Example 4, the online incremental distillation mechanism described in step 2 is as follows: when a new node or relationship is added to the knowledge graph, the incremental distillation subtask is triggered. The teacher model generates incremental soft labels only for the new knowledge fragments. By freezing the old knowledge layer of the student model and fine-tuning the newly added knowledge layer, the student model can absorb new knowledge without forgetting the old knowledge.

[0029] Example 5: The deployment of the lightweight student model in step 2 adopts the Ollam framework and Docker containerization technology, combined with a modal and task dual-trigger loading mechanism, that is, when a specific modal query is received, only the corresponding model sub-module is activated.

[0030] By adopting the above technical solution, the core principle of dynamic modality-aware distillation is to design differentiated distillation strategies based on the characteristic differences of different modalities of text, image, and audio data to ensure that the lightweight student model accurately inherits the core knowledge of each modality. The core knowledge of text data is reflected in the relational logic between entities; therefore, the entity relationship attention weights of the teacher model are used as soft labels to guide the student model in learning the key points of entity interactions in the text. The core information of image data is contained in the visual feature distribution; therefore, the visual feature map of the teacher model is used as a soft label to enable the student model to master the feature representation of key objects in the image. Simultaneously, through modality-adaptive loss functions, such as the loss function that measures the difference in probability distribution for text adaptation and the loss function that measures feature similarity for image adaptation, the student model can specifically optimize the learning objectives of each modality during training, avoiding the inadequacy of a single distillation strategy for multimodal data, and ultimately achieving efficient learning and retention of core knowledge of different modalities by the student model.

[0031] The principle of knowledge graph structure-guided distillation is to use the structured features of the knowledge graph as a directional guide for the distillation process, enabling lightweight student models to not only learn the output of teacher models but also inherit the inherent logical structure of knowledge. Core entities with high connectivity in the knowledge graph, such as key concepts that frequently relate to other entities, are the pillars of the knowledge system. By increasing the distillation weight of their corresponding knowledge, the learning intensity of the student model for core knowledge is strengthened. Conversely, fuzzy relationships with low confidence, such as those with unreliable or disputed sources, are prone to knowledge bias if over-distilled; therefore, their weight is reduced to minimize error propagation. This distillation strategy, using the connectivity of knowledge graph entities and the confidence of relationships as weighting factors, allows student models to anchor themselves to the structural framework of knowledge during the learning process, ensuring that the learned knowledge is not only accurate but also conforms to the logical connections of the overall knowledge system, thus improving the model's ability to understand and apply complex knowledge.

[0032] The principle of the online incremental distillation mechanism is to solve the problem of compatibility between old and new knowledge in a lightweight student model under dynamically updated knowledge scenarios through hierarchical processing and triggered learning. When new nodes or relationships are added to the knowledge graph, the system only triggers the incremental distillation subtask for these new knowledge fragments, avoiding retraining the entire model and reducing computational resource consumption. The teacher model generates exclusive soft labels for new knowledge, and the student model freezes the network layer weights corresponding to the mastered old knowledge during training, only fine-tuning the network layers corresponding to the newly added knowledge. This ensures that old knowledge is not forgotten while quickly absorbing new knowledge. This mechanism enables the lightweight model to adapt to the dynamic changes of the knowledge graph in real time, keeping it synchronized with the latest knowledge, overcoming the limitations of traditional static distillation in dealing with knowledge updates, and achieving continuous evolution of model performance.

[0033] The principle behind lightweight student model deployment is to balance resource efficiency and response speed through containerization technology and an on-demand loading mechanism. The Ollama framework provides model lifecycle management capabilities, and combined with Docker containerization technology, it enables the isolation and standardization of the model's runtime environment, ensuring consistent deployment across different devices while reducing environment configuration complexity. The modal and task-triggered loading mechanism activates only the corresponding sub-modules within the model based on the modal type and specific task of the input query, rather than loading the entire model, significantly reducing memory and computational resource consumption. This deployment method improves model response efficiency while ensuring local operational security, enabling lightweight models to flexibly adapt to resource-constrained scenarios on edge devices or local servers.

[0034] Example 6: The multiple sources mentioned in step 1 include enterprise documents, databases, IoT sensors, and web crawler resources; the automatic cleaning using a large model includes removing noise, duplicates, and irrelevant content, and unifying the data format includes converting audio to text and extracting features from images.

[0035] Example 7, step 3 of constructing the dynamic multimodal knowledge graph specifically includes: The knowledge graph is stored using Neo4j graph database. The graph consists of entities, attributes and relationships extracted from multimodal data by a lightweight large model, and supports multi-hop association queries between entities. By monitoring file system changes, database transaction logs, and message queue pushes, data addition or modification events are captured in real time, triggering a lightweight large model to re-analyze the changed data and automatically performing knowledge graph update operations, including adding entity nodes, deleting outdated attributes, and adjusting the weights of relationship edges. A version chain and timestamp management mechanism is introduced to maintain historical versions for each node and relationship in the knowledge graph, supporting time-based tracing and rollback. For knowledge conflicts, a lightweight large model is used to resolve conflicts by combining confidence analysis and user feedback: the large model first scores the reliability of the source and semantic consistency of the conflicting information. If the confidence difference exceeds the threshold, the high-confidence information is automatically adopted; if the difference is below the threshold, the user feedback interface is triggered, and the graph is updated after manual confirmation.

[0036] By adopting the above technical solutions, through a storage architecture adapted to the characteristics of multimodal data, a real-time data update mechanism, traceable version management, and human-machine collaborative conflict resolution strategies, the dynamism, consistency, and availability of knowledge graphs can be achieved.

[0037] It natively supports a graphical storage structure for entities and relationships, which can efficiently organize entities, attributes and relationships extracted from multimodal data. It is naturally adapted to the needs of multi-hop association queries and can quickly traverse multi-level associations between entities, providing efficient data access support for subsequent joint reasoning and cross-modal retrieval.

[0038] A multi-channel monitoring mechanism comprehensively captures data source changes. File system change monitoring, database transaction log parsing, and message queue push reception correspond to change awareness for different types of data sources, ensuring that newly added or modified data can be detected in a timely manner. Change events trigger a lightweight large-scale model to re-extract entities and relationships from the target data, automatically performing update operations such as adding nodes, deleting outdated attributes, and adjusting relationship weights. This allows the knowledge graph to reflect data changes in real time, avoiding the problem of static graphs being disconnected from actual data.

[0039] Each node and relationship in the knowledge graph is bound to a unique timestamp and version identifier, forming a complete version chain according to the update sequence. This mechanism ensures the traceability of the historical state of knowledge and supports rolling back to a specific time point in the graph version based on business needs. This facilitates the investigation of the root cause of knowledge errors and provides support for knowledge queries in specific historical periods, ensuring the rigor of knowledge management.

[0040] A complementary mechanism combining automated machine judgment and human intervention is established. The lightweight, large-scale model first quantifies and scores the reliability of conflicting information sources (e.g., the authority of the data publisher) and semantic consistency (e.g., its fit with the existing knowledge system). Processing scenarios are then categorized based on confidence level differences: when the difference exceeds the threshold, high-confidence information is automatically adopted to ensure processing efficiency; when the difference is below the threshold, a user feedback interface is triggered to introduce human confirmation, avoiding knowledge bias caused by machine misjudgment. This human-machine collaborative conflict resolution approach ensures processing efficiency while maximizing the logical consistency and accuracy of the knowledge graph.

[0041] Example 8, the construction of the multimodal fusion and joint inference engine in step 4 specifically includes: By employing the CLIP cross-modal encoder or a custom multimodal encoder based on DeepSeek, text, image, and audio data are mapped to a unified semantic embedding space, enabling computationally achievable semantic similarity among different modalities. A two-layer joint reasoning engine combining semantic retrieval and logical reasoning is constructed: the first layer performs similarity retrieval on the user's multimodal queries in a unified semantic space to obtain relevant entities and multimodal data fragments from the knowledge graph; the second layer uses a lightweight large model to perform contextual logical reasoning based on the retrieval results to generate natural language answers containing multimodal associations. This approach associates objects in images with technical documents described in text, and voice commands and operational process nodes in audio, enabling cross-modal knowledge linking of images, text, and audio, and supporting multimodal joint retrieval.

[0042] By adopting the above technical solution, the principle of cross-modal semantic mapping is to utilize the feature extraction capabilities of CLIP or a custom multimodal encoder to convert different modal data such as text, images, and audio into vectors in a unified semantic embedding space. Different modal data have quantifiable semantic similarity in this space, solving the modality gap problem in traditional single-modal processing. This allows text descriptions to be directly associated with corresponding images and audio at the semantic level, laying the foundation for cross-modal retrieval and reasoning.

[0043] The efficiency and accuracy of reasoning are improved through division of labor and collaboration. The first layer of semantic retrieval is based on similarity calculation in a unified semantic space, which quickly locates knowledge graph entities and multimodal data fragments related to the user's multimodal query, ensuring the relevance and comprehensiveness of the retrieval results. The second layer of logical reasoning uses a lightweight large model to perform contextual integration and logical analysis based on the retrieved multimodal information, linking scattered knowledge fragments into coherent natural language answers, solving the problem that a single retrieval method cannot handle complex semantic relationships.

[0044] The principle of cross-modal knowledge linking is to establish direct associations between entities of different modalities to construct a complete multimodal knowledge network. By binding object entities in images with text technical documents and voice commands in audio with operational process nodes, knowledge is no longer limited to a single modality. Users can start from any modality to obtain cross-modal related information, enabling joint retrieval scenarios such as searching text by image and searching processes by audio, thus improving the flexibility and comprehensiveness of knowledge acquisition.

[0045] Example 9, the intelligent optimization generated by the retrieval enhancement in step 5 specifically includes: Dynamic semantic chunking is implemented based on the Ragflow framework. The chunking strategy combines sentence boundaries, topic clustering, and knowledge graph entity density: text paragraphs containing dense entities are segmented by entity groups, and narrative texts are segmented by plot topics, ensuring the semantic integrity of the chunks and the targeting of retrieval. Text or feature vector similarity retrieval is performed using FAISS, keyword exact matching is performed using Elasticsearch, and multi-hop path retrieval of knowledge graphs is performed using Neo4j. The results of multi-source retrieval are weighted and fused according to confidence level. During the generation phase, fact anchors are inserted into the prompts. These fact anchors include knowledge graph entity IDs and authoritative document sources, forcing the large model to anchor the generation of facts. After generation, the system undergoes a two-stage verification process: self-verification by a large model and verification based on user feedback. Answers that appear to be hallucinations are automatically recalled and regenerated.

[0046] By adopting the above technical solutions, the generation effect is enhanced through precise retrieval and multi-constraint optimization, improving the relevance and reliability of the answers. Dynamic semantic segmentation adapts the segmentation strategy according to text characteristics, ensuring the semantic integrity of the segments and the accuracy of retrieval; hybrid retrieval integrates vector similarity matching, keyword precise matching, and graph multi-hop path retrieval, and the weighted fusion improves the comprehensiveness of knowledge recall; fact anchoring forces the model to bind to authoritative knowledge sources, and two-stage verification filters out content that deviates from the facts, effectively suppressing illusions and ensuring that the generated answers are true and credible.

[0047] Example 10, step 6, specifically includes knowledge updating and consistency maintenance under high concurrency: For data sources with high real-time requirements, event-driven instant updates are achieved through Kafka message queues; for batch historical data, batch updates are performed through scheduled tasks, and the two types of update tasks avoid concurrency conflicts through a distributed lock mechanism. The lightweight large model uses incremental learning technology, which freezes the weights of the old knowledge layer when updating new knowledge, and only fine-tunes the new knowledge layer. At the same time, it automatically aligns the model's newly added entity or relation predictions with the knowledge graph to ensure the consistency between the model output and the graph structure. This implementation leverages the Spark distributed computing framework to achieve parallel updates and consistency checks for large-scale knowledge graphs: the graph is partitioned by entities, and the addition and deletion operations of nodes or relations are performed in a distributed manner; a lightweight large model performs a global logical consistency check on the distributed updated graph, and the check results are automatically corrected by the knowledge graph's constraint rule engine. By collecting user feedback on the accuracy of knowledge and the relevance of retrieval, and adjusting the weights of hybrid retrieval, the prompt word strategy generated by the large model, and the update priority of the knowledge graph through reinforcement learning, the system achieves self-iterative optimization.

[0048] By adopting the above technical solutions, and through a hybrid update mechanism adapted to different data needs, collaborative optimization of the model and knowledge graph, and distributed processing, the efficiency and consistency of knowledge updates in high-concurrency scenarios are ensured. The hybrid update mechanism responds to the update needs of real-time data and batch data respectively, and distributed locks avoid concurrency conflicts; incremental learning enables the model to efficiently absorb new knowledge without forgetting old knowledge, and automatic alignment with the knowledge graph ensures consistent output; the Spark distributed framework enables large-scale parallel updates of the knowledge graph and global consistency checks, and the rule engine automatically corrects logical contradictions; user feedback is used to adjust key system parameters through reinforcement learning, forming a closed-loop iterative continuous optimization of performance.

[0049] The following describes the implementation principle of the present invention using specific embodiments: Application of the intelligent substation equipment operation and maintenance knowledge base system: Step 1: Multimodal Operation and Maintenance Data Acquisition and Preprocessing: The system collects textual data (substation equipment manuals, maintenance records from the past three years, State Grid's operation and maintenance regulations, and fault case studies), image data (infrared thermal images, equipment defect photos, and installation drawings), audio data (recordings of abnormal equipment noises and operation and maintenance voice commands), and sensor data (transformer oil temperature, SF6 gas pressure, and bus current / voltage time-series data). It utilizes DeepSeek's large model to clean up duplicate and redundant data, filter out sensor anomalies, standardize the format (audio-to-text, image ROI feature extraction, and time-series data interpolation and completion), automatically label equipment types and fault categories, extract text entities and relationships, and extract feature vectors from non-text data, forming a multimodal operation and maintenance data pool that combines structured and unstructured data.

[0050] Step 2: Lightweight Deployment and Optimization of the Large-Scale Operation and Maintenance Model: Using a finely tuned DeepSeek-Industrial large-scale model for industrial applications as the teacher model, a lightweight student model with 8B parameters is trained: Dynamic modality-aware distillation uses entity relationship attention weights for text and visual feature maps as soft labels for images, paired with a differential loss function; Knowledge graph structure-guided distillation assigns a high weight of 0.85 to core entities such as "transformer" and a low weight of 0.25 to fuzzy relationships; Online incremental distillation freezes the old knowledge layer when the graph is updated, only fine-tuning the new knowledge layer. Deployed on a substation edge server (NVIDIA A30 graphics card) via Ollam+Docker, modality-task dual-trigger loading is enabled, and only the visual processing module is activated when receiving infrared image diagnostics.

[0051] Step 3: Construction of a dynamic multimodal knowledge graph: A knowledge graph is built using Neo4j, containing nodes for equipment, faults, maintenance, and spare parts. Node attributes include equipment model and fault level, supporting multi-hop queries between equipment, faults, and handling solutions. Data updates are captured by monitoring file changes, database transaction logs, and Kafka message queues. For example, when a sensor detects low oil level, the model automatically adds a fault node and associates it with maintenance relationships. Timestamps and version numbers are introduced for management; knowledge conflicts are scored based on confidence level, and a manual review channel is maintained.

[0052] Step 4: Multimodal fusion and joint reasoning: A CLIP encoder is used to map multimodal data to a unified semantic space, achieving semantic alignment between transformer high-temperature infrared images and winding overheating fault text. A two-layer inference engine is constructed: when maintenance personnel upload switchgear infrared images for querying, the engine first retrieves relevant entities and data fragments from the image, and then a lightweight model generates a fault handling plan containing operating steps, torque parameters, and retesting standards. Cross-modal associations are established between infrared images and maintenance procedures, as well as voice commands and operating processes, supporting image-to-text and voice-to-process queries.

[0053] Step 5: Search Enhancement and Optimization: Based on Ragflow's topic clustering and dynamic entity density segmentation (300-600 characters per segment), it integrates FAISS vector retrieval (weight 0.4), Elasticsearch keyword retrieval (weight 0.35), and Neo4j graph multi-hop retrieval (weight 0.25). It generates prompts and inserts them into knowledge graph entity IDs and authoritative procedure source fact anchors. After generation, it uses a two-stage process of model self-validation and feedback from operations personnel to suppress illusions, ensuring the answers closely match real-world practice.

[0054] Step 6: Knowledge Update and System Optimization A hybrid update mechanism is adopted: real-time sensor data is updated instantly within 500ms via Kafka, and fault cases and new procedures are updated in batches on the 10th of each month. Distributed locks prevent concurrent conflicts. The geographic map is partitioned and updated in parallel by device type using Spark, with incremental model learning and automatic alignment with the geographic map. A multimodal retrieval interface for text, image, and voice is built, with high-frequency queries cached in Redis. Feedback is collected, and parameters are adjusted quarterly through reinforcement learning, achieving an accuracy rate of over 97%.

[0055] The above are all preferred embodiments of the present invention and are not intended to limit the scope of protection of the present invention. Therefore, all equivalent changes made in accordance with the structure, shape and principle of the present invention should be covered within the scope of protection of the present invention.

Claims

1. A method for dynamic construction and intelligent retrieval of knowledge base based on a large model, characterized in that: Includes the following steps: Step 1: Collect multimodal data (text, images, audio, and video) from multiple sources, and use a large model to automatically clean, standardize, label, classify, and extract features to form a multimodal data pool; Step 2: Using the DeepSeek large model as the teacher model, train a lightweight student model through a specialized knowledge distillation strategy, and combine it with containerization technology to achieve local deployment; the specialized knowledge distillation strategy includes dynamic modality-aware distillation, knowledge graph structure-guided distillation, and online incremental distillation; Step 3: Dynamic multimodal knowledge graph construction. Utilize lightweight large models to extract entities, relationships, and attributes from multimodal data to construct a knowledge graph. Set up a real-time dynamic update mechanism and implement version control and conflict resolution. Step 4: Integrate multimodal data into a unified semantic space through cross-modal semantic mapping, and integrate large models and rule engines to achieve joint reasoning; Step 5: Perform dynamic semantic segmentation and hybrid retrieval based on Ragflow, combine large model context to generate and suppress answer illusion; Step 6: Adopt an update mechanism that combines event-driven and batch scheduling, utilize large models and distributed computing to maintain knowledge consistency, build a multimodal retrieval interface, optimize system performance, and iteratively optimize the model through user feedback.

2. The method for dynamic construction and intelligent retrieval of knowledge base based on a large model according to claim 1, characterized in that, In step 2, dynamic modality-aware distillation specifically involves: for different modalities of data such as text, image, and audio, the entity relationship attention weights and visual feature maps of the teacher model are used as soft labels, and the student model is trained through a modality-adaptive loss function to make the student model adapt to the core knowledge of each modality.

3. The method for dynamic construction and intelligent retrieval of knowledge base based on a large model according to claim 2, characterized in that, In step 2, the knowledge graph structure-guided distillation specifically involves using the knowledge graph constructed in step 3 as a structured knowledge anchor point. During the distillation process, the entity connectivity and relation confidence of the graph are introduced as weighting factors. The knowledge corresponding to core entities with high connectivity is strengthened during distillation, while the distillation weight is reduced for fuzzy relations with low confidence, so that the student model inherits the inherent structural logic of the knowledge.

4. The method for dynamic construction and intelligent retrieval of knowledge base based on a large model according to claim 3, characterized in that, The online incremental distillation mechanism described in step 2 is as follows: when a new node or relationship is added to the knowledge graph, the incremental distillation subtask is triggered. The teacher model generates incremental soft labels only for the new knowledge fragments. By freezing the old knowledge layer of the student model and fine-tuning the newly added knowledge layer, the student model can absorb new knowledge without forgetting the old knowledge.

5. The method for dynamic construction and intelligent retrieval of knowledge base based on a large model according to claim 4, characterized in that, The deployment of the lightweight student model in step 2 adopts the Ollam framework and Docker containerization technology, combined with a modal and task dual-trigger loading mechanism, that is, when a specific modal query is received, only the corresponding model sub-module is activated.

6. The method for dynamic construction and intelligent retrieval of knowledge base based on a large model according to claim 5, characterized in that, The multiple sources mentioned in step 1 include enterprise documents, databases, IoT sensors, and web crawler resources; the automatic cleaning using large models includes removing noise, duplicates, and irrelevant content, and unifying the data format includes converting audio to text and extracting features from images.

7. The method for dynamic construction and intelligent retrieval of knowledge base based on a large model according to claim 6, characterized in that, Step 3, the construction of the dynamic multimodal knowledge graph, specifically includes: The knowledge graph is stored using Neo4j graph database. The graph consists of entities, attributes and relationships extracted from multimodal data by a lightweight large model, and supports multi-hop association queries between entities. By monitoring file system changes, database transaction logs, and message queue pushes, data addition or modification events are captured in real time, triggering a lightweight large model to re-analyze the changed data and automatically performing knowledge graph update operations, including adding entity nodes, deleting outdated attributes, and adjusting the weights of relationship edges. A version chain and timestamp management mechanism is introduced to maintain historical versions for each node and relationship in the knowledge graph, supporting time-based tracing and rollback. For knowledge conflicts, a lightweight large model is used to resolve conflicts by combining confidence analysis and user feedback: the large model first scores the reliability of the source and semantic consistency of the conflicting information. If the confidence difference exceeds the threshold, the high-confidence information is automatically adopted; if the difference is below the threshold, the user feedback interface is triggered, and the graph is updated after manual confirmation.

8. The method for dynamic construction and intelligent retrieval of knowledge base based on a large model according to claim 7, characterized in that, Step 4, the construction of the multimodal fusion and joint inference engine, specifically includes: By employing the CLIP cross-modal encoder or a custom multimodal encoder based on DeepSeek, text, image, and audio data are mapped to a unified semantic embedding space, enabling computationally achievable semantic similarity among different modalities. A two-layer joint reasoning engine combining semantic retrieval and logical reasoning is constructed: the first layer performs similarity retrieval on the user's multimodal queries in a unified semantic space to obtain relevant entities and multimodal data fragments from the knowledge graph; the second layer uses a lightweight large model to perform contextual logical reasoning based on the retrieval results to generate natural language answers containing multimodal associations. This approach associates objects in images with technical documents described in text, and voice commands and operational process nodes in audio, enabling cross-modal knowledge linking of images, text, and audio, and supporting multimodal joint retrieval.

9. The method for dynamic construction and intelligent retrieval of knowledge base based on a large model according to claim 8, characterized in that, Step 5, which involves intelligent optimization to enhance retrieval, specifically includes: Dynamic semantic chunking is implemented based on the Ragflow framework. The chunking strategy combines sentence boundaries, topic clustering, and knowledge graph entity density: text paragraphs containing dense entities are segmented by entity groups, and narrative texts are segmented by plot topics, ensuring the semantic integrity of the chunks and the targeting of retrieval. Text or feature vector similarity retrieval is performed using FAISS, keyword exact matching is performed using Elasticsearch, and multi-hop path retrieval of knowledge graphs is performed using Neo4j. The results of multi-source retrieval are weighted and fused according to confidence level. During the generation phase, fact anchors are inserted into the prompts. These fact anchors include knowledge graph entity IDs and authoritative document sources, forcing the large model to anchor the generation of facts. After generation, the system undergoes a two-stage verification process: self-verification by a large model and verification based on user feedback. Answers that appear to be hallucinations are automatically recalled and regenerated.

10. The method for dynamic construction and intelligent retrieval of a knowledge base based on a large model according to claim 9, characterized in that, Step 6, knowledge updating and consistency maintenance under high concurrency, specifically includes: For data sources with high real-time requirements, event-driven instant updates are achieved through Kafka message queues; for batch historical data, batch updates are performed through scheduled tasks, and the two types of update tasks avoid concurrency conflicts through a distributed lock mechanism. The lightweight large model uses incremental learning technology, which freezes the weights of the old knowledge layer when updating new knowledge, and only fine-tunes the new knowledge layer. At the same time, it automatically aligns the model's newly added entity or relation predictions with the knowledge graph to ensure the consistency between the model output and the graph structure. This implementation leverages the Spark distributed computing framework to achieve parallel updates and consistency checks for large-scale knowledge graphs: the graph is partitioned by entities, and the addition and deletion operations of nodes or relations are performed in a distributed manner; a lightweight large model performs a global logical consistency check on the distributed updated graph, and the check results are automatically corrected by the knowledge graph's constraint rule engine. By collecting user feedback on the accuracy of knowledge and the relevance of retrieval, and adjusting the weights of hybrid retrieval, the prompt word strategy generated by the large model, and the update priority of the knowledge graph through reinforcement learning, the system achieves self-iterative optimization.

Images