An AI knowledge base construction method based on multi-modal fusion and adaptive update

Abstract

The application provides an AI knowledge base construction method based on multi-modal fusion and adaptive update, relates to the technical field of artificial intelligence and knowledge management, cuts the collected multi-source data into multiple knowledge blocks according to semantic logic, and performs cross-modal semantic alignment of text and image on the knowledge blocks containing images and texts; an embedding model is dynamically selected based on a sensitive level, and the output vector and the corresponding metadata are stored in a vector database and are isolated by partitioning according to the sensitive level; when a user query is received, semantic similarity retrieval and entity relationship logical retrieval are performed to obtain an optimized context, which is input into a locally deployed large language model; knowledge point value evaluation is performed on user behavior data, a knowledge point life cycle state table is generated, and based on the knowledge point correlation relationship, a predictive update work order list is generated. Thus, multi-modal data deep fusion, knowledge security efficient processing, complex query accurate response and knowledge base adaptive update are realized.

Description

Technical Field

[0001] This application relates to the fields of artificial intelligence and knowledge management technology, and more specifically, to a method for constructing an AI knowledge base based on multimodal fusion and adaptive updating. Background Technology

[0002] With the continuous improvement of enterprise informatization and the explosive growth of data volume, efficient knowledge management has become key to improving organizational efficiency and competitiveness. As a core tool for knowledge management, artificial intelligence knowledge base systems have evolved from rule-based systems to those combining traditional machine learning models, and then to those incorporating knowledge graphs and deep learning models.

[0003] However, current technologies still have significant limitations in achieving truly intelligent and efficient knowledge management when facing complex application scenarios. Existing knowledge base systems suffer from the following deficiencies: (1) Insufficient knowledge relevance and difficulty in dynamic updates. Traditional knowledge bases based on simple vector retrieval systems often struggle to explicitly express and effectively capture the complex deep semantic relationships and causal logic between knowledge fragments (e.g., drug A may cause side effects B, which in turn requires treatment C, forming a complete clinical pathway). This leads to the system easily missing information or returning discrete, unrelated answers when dealing with complex queries that require multi-step reasoning, requiring users to piece together information themselves, thus affecting decision-making efficiency. In addition, the existing system's knowledge update mechanism is often not flexible enough. In many cases, adding new knowledge may require a full reconstruction of the entire vector space or manual intervention to adjust model parameters, resulting in high update costs and long cycles, making it difficult to adapt to the rapid iteration of knowledge. (2) The contradiction between efficiency and accuracy in handling complex queries. Existing technologies face challenges when handling complex queries involving multiple conditions (e.g., "anesthesia plan for patients with renal insufficiency undergoing spinal surgery"). Simply relying on vector similarity retrieval may miss key information due to the inability to understand the multi-layered logic behind the query; while relying on traditional knowledge graphs for multi-hop reasoning, although the correlation is strong, the query response time may increase exponentially with the increase of the depth of the correlation path, making it difficult to meet the experience requirements of real-time interaction. This shows that there is an insurmountable contradiction between the accuracy (recall and precision) of retrieval and the response speed. (3) Limited ability of multimodal data fusion and deep understanding. Although some existing systems can process data in multiple formats (text, audio, etc.), they are still lacking in achieving deep, semantic-level cross-modal information fusion and unified understanding. For example, the system may independently process the text in a document, the adjacent charts and their explanatory texts, but fails to truly understand the deep semantic connections and mutual explanatory relationships between them, making it difficult to generate answers based on a comprehensive understanding of multimodal information. (4) Insufficient integration and adaptive intelligence. Many solutions may focus on a certain part of the technology stack (such as focusing on retrieval or focusing on question-answer generation), which makes them perform poorly in terms of overall adaptive capabilities and personalized services. For example, it may not be able to perform self-optimization and predictive knowledge updates based on actual user feedback, behavioral patterns, and newly generated data streams. The knowledge base lacks "activity" and maintenance still largely depends on manual intervention. Summary of the Invention

[0004] In view of this, the purpose of this application is to provide an AI knowledge base construction method based on multimodal fusion and adaptive updating, which can realize deep fusion of multimodal data, secure and efficient processing of knowledge, accurate response to complex queries, and adaptive updating of the knowledge base.

[0005] In a first aspect, embodiments of this application provide a method for constructing an AI knowledge base based on multimodal fusion and adaptive updating, the method comprising the following steps: Collect multi-source data, and parse and clean the collected multi-source data to obtain preprocessed data; the multi-source data includes unstructured documents, audio data, and structured / semi-structured data; The preprocessed data is divided into multiple knowledge blocks according to semantic logic, and cross-modal semantic alignment of text and images is performed on the knowledge blocks containing images and text. The knowledge blocks are identified by sensitivity level, and the embedding model is dynamically selected based on the sensitivity level. The vectors output by different embedding models and the corresponding metadata are stored in the vector database and partitioned and isolated according to sensitivity level. When a user query is received, semantic similarity retrieval and entity relationship logic retrieval are performed, and the retrieval results of the two paths are merged and reordered to obtain an optimized context, which is then input into the locally deployed large language model. The knowledge point value of user behavior data is evaluated, a knowledge point lifecycle status table is generated, and the work order list is predictively updated based on the knowledge point lifecycle status table and the knowledge point association relationship.

[0006] In some embodiments, the process of collecting multi-source data and parsing and cleaning the collected multi-source data to obtain preprocessed data includes the following steps: Collect data from multiple sources by connecting to the corresponding database tables or API interfaces; Multi-source data in different formats are parsed and converted into a unified text format. Natural language processing tools are then used to perform data cleaning operations such as word segmentation, removal of useless characters, and entity recognition to obtain preprocessed data.

[0007] In some embodiments, the step of dividing the preprocessed data into multiple knowledge blocks according to semantic logic, and performing cross-modal semantic alignment of text and images on the knowledge blocks containing images and text, includes the following steps: A recursive character text segmenter is used to cut the preprocessed data into multiple knowledge blocks of a set length that contain overlapping areas according to semantic logic. For knowledge blocks containing both images and text, identify the images and their associated descriptive text; The visual language pre-trained model CLIP encodes images and their associated descriptive text into a unified vector space, resulting in cross-modal semantically aligned knowledge blocks.

[0008] In some embodiments, the steps of identifying the sensitivity level of the knowledge block, dynamically selecting an embedding model based on the sensitivity level, storing the vectors output by different embedding models and their corresponding metadata in a vector database, and partitioning and isolating them according to sensitivity level include the following steps: Based on business needs, the Drools rule engine is used to preset sensitivity judgment rules to determine the three levels of standards: public, internal, and confidential. The sensitivity level of the knowledge block is identified by using a fine-tuned BERT model combined with the rule engine Drools. The embedded model is scheduled according to the sensitivity level; among them, the public / internal level schedules the cloud-based intelligent model, and the confidential level schedules the locally deployed lightweight model. After standardizing the dimensions of the vectors output by different embedding models, they are stored in the vector database along with the corresponding metadata. The vectors are then partitioned and isolated according to their sensitivity levels, and different access strategies are configured for different partitions.

[0009] In some embodiments, when a user query is received, semantic similarity retrieval and entity relationship logic retrieval are performed, and the retrieval results of the two paths are fused and reordered to obtain an optimized context, which is then input into a locally deployed large language model, including the following steps: When a user query is received, a dual-path hybrid retrieval is performed; that is, semantic similarity retrieval is performed in the vector database to match knowledge blocks that are semantically related to the query; and entity relationship logic query is performed in the knowledge graph database to mine the relationships between knowledge points. The results from the two search paths are merged and reordered to obtain an optimized context; among them, results that simultaneously contain semantic similarity and logical relevance are retained first. The optimized context is input into a locally deployed large language model to restrict the large language model from generating answers based on the provided context.

[0010] In some embodiments, the step of evaluating the value of user behavior data for knowledge points, generating a knowledge point lifecycle status table, and predictively updating the work order list based on the knowledge point lifecycle status table and knowledge point relationships includes the following steps: The system captures user behavior data from all dimensions and performs cleaning and aggregation processing; the behavior data includes interaction data collected from the front end and log records collected from the back end. A value assessment model is built based on aggregated behavioral data. The real-time popularity weight of each knowledge point is calculated by integrating multi-dimensional behavioral indicators, and a knowledge point lifecycle status table is generated. Among them, multi-dimensional behavioral indicators include page visit frequency, dwell time, and positive feedback ratio. The work order list is predictively updated based on the knowledge point lifecycle status table and the knowledge point associations stored in the knowledge graph database; the work order list includes task type and associated knowledge point ID.

[0011] In some embodiments, when the preprocessed data is divided into multiple knowledge blocks according to semantic logic using a recursive character text segmenter, the character length of the knowledge block is 600-2000; the character length of the overlapping area is 100-200.

[0012] Secondly, embodiments of this application provide an AI knowledge base construction apparatus based on multimodal fusion and adaptive updating, the apparatus comprising: The acquisition module is used to collect multi-source data and parse and clean the collected multi-source data to obtain preprocessed data; the multi-source data includes unstructured documents, audio data, and structured / semi-structured data; The knowledge block segmentation module is used to segment the preprocessed data into multiple knowledge blocks according to semantic logic, and to perform cross-modal semantic alignment of text and images for knowledge blocks containing images. The sensitivity level identification module is used to identify the sensitivity level of the knowledge block, dynamically select the embedding model based on the sensitivity level, and store the vectors output by different embedding models and the corresponding metadata into the vector database, which is partitioned and isolated according to the sensitivity level. The dual-path retrieval module is used to perform semantic similarity retrieval and entity relationship logic retrieval when receiving user queries, and to fuse and reorder the retrieval results of the two paths to obtain an optimized context, which is then input into the locally deployed large language model. The adaptive update module is used to evaluate the value of knowledge points in user behavior data, generate a knowledge point lifecycle status table, and predictively update the work order list based on the knowledge point lifecycle status table and the knowledge point association relationship.

[0013] Thirdly, an electronic device provided in this application includes a processor, a memory, and a bus. The memory stores machine-readable instructions executable by the processor. When the electronic device is running, the processor communicates with the memory via the bus. When the machine-readable instructions are executed by the processor, the steps of the AI ​​knowledge base construction method based on multimodal fusion and adaptive updating described in any of the first aspects are executed.

[0014] Fourthly, embodiments of this application provide a computer-readable storage medium storing a computer program, which, when executed by a processor, performs the steps of the AI ​​knowledge base construction method based on multimodal fusion and adaptive updating described in any of the first aspects.

[0015] This application describes an AI knowledge base construction method based on multimodal fusion and adaptive updating. The method involves collecting multi-source data, parsing and cleaning it to obtain preprocessed data, then segmenting the preprocessed data into multiple knowledge blocks according to semantic logic, and performing cross-modal semantic alignment of text and images in knowledge blocks containing images. Sensitivity levels are identified for each knowledge block, and embedding models are dynamically selected based on these sensitivity levels. Vectors output by different embedding models and their corresponding metadata are stored in a vector database, partitioned and isolated according to sensitivity levels. When a user query is received, semantic similarity retrieval and entity relationship logic retrieval are performed, and the retrieval results from the two paths are fused and reordered to obtain an optimized context, which is then input into a locally deployed large language model. User behavior data is used to evaluate the value of knowledge points, generating a knowledge point lifecycle status table. Based on this table and knowledge point relationships, a predictive update of the work order list is performed. This achieves deep multimodal data fusion, secure and efficient knowledge processing, accurate response to complex queries, and adaptive updating of the knowledge base. Attached Figure Description

[0016] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0017] Figure 1 A flowchart of the AI ​​knowledge base construction method based on multimodal fusion and adaptive updating described in an embodiment of this application is shown; Figure 2 This document illustrates a flowchart illustrating how the preprocessed data is divided into multiple knowledge blocks according to semantic logic, as described in an embodiment of this application. Figure 3 The flowchart illustrates the process described in this application for storing vectors output by different embedding models and their corresponding metadata into a vector database and isolating them by sensitivity level. Figure 4 A flowchart illustrating the predictive update work order list described in an embodiment of this application is shown; Figure 5 This illustration shows a schematic diagram of the structure of the AI ​​knowledge base construction device based on multimodal fusion and adaptive updating described in an embodiment of this application; Figure 6 A structural block diagram of the electronic device described in an embodiment of this application is shown. Detailed Implementation

[0018] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. It should be understood that the accompanying drawings in this application are for illustrative and descriptive purposes only and are not intended to limit the scope of protection of this application. Furthermore, it should be understood that the schematic drawings are not drawn to scale. The flowcharts used in this application illustrate operations implemented according to some embodiments of this application. It should be understood that the operations in the flowcharts may not be implemented in sequence, and steps without logical contextual relationships may be reversed or implemented simultaneously. In addition, those skilled in the art, guided by the content of this application, may add one or more other operations to the flowcharts, or remove one or more operations from the flowcharts.

[0019] Furthermore, the described embodiments are merely some, not all, of the embodiments of this application. The components of the embodiments of this application described and illustrated herein can typically be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of this application provided in the accompanying drawings is not intended to limit the scope of the claimed application, but merely to illustrate selected embodiments of the application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are within the scope of protection of this application.

[0020] It should be noted that the term "comprising" will be used in the embodiments of this application to indicate the presence of the features declared thereafter, but does not exclude the addition of other features.

[0021] In view of the technical problems eliminated by the background technology, this application provides an AI knowledge base construction method based on multimodal fusion and adaptive updating, which can realize deep fusion of multimodal data, safe and efficient processing of knowledge, accurate response to complex queries, and adaptive updating of the knowledge base.

[0022] See the instruction manual appendix Figure 1 This application provides an AI knowledge base construction method based on multimodal fusion and adaptive updating, the method comprising the following steps: S1. Collect multi-source data, and parse and clean the collected multi-source data to obtain preprocessed data; among which, multi-source data includes unstructured documents, audio data, and structured / semi-structured data; S2. The preprocessed data is divided into multiple knowledge blocks according to semantic logic, and cross-modal semantic alignment of text and images is performed on the knowledge blocks containing images and text. S3. The knowledge block is identified by sensitivity level, and the embedding model is dynamically selected based on the sensitivity level. The vectors output by different embedding models and the corresponding metadata are stored in the vector database and partitioned and isolated according to sensitivity level. S4. When a user query is received, semantic similarity retrieval and entity relationship logic retrieval are performed, and the retrieval results of the two paths are merged and reordered to obtain the optimized context, which is then input into the locally deployed large language model. S5. Evaluate the knowledge point value of user behavior data, generate a knowledge point lifecycle status table, and predictively update the work order list based on the knowledge point lifecycle status table and the knowledge point association relationship.

[0023] Step S1 mainly involves multi-source data acquisition and preprocessing.

[0024] In one embodiment, multi-source data access is achieved by connecting to unstructured documents (PDF, Word), audio data (conference recordings, etc.), and structured / semi-structured data (database tables, API interfaces). Furthermore, the collected multi-format data is parsed and converted. Document parsing uses PyMuPDF and python-docx to extract text and structures such as titles, paragraphs, and tables; scanned documents / image text is extracted using the OCR engine PaddleOCR; audio data is converted to text using the Whisper speech recognition engine, achieving format unification from non-text data to text data. Further, the natural language processing tool SpaCy is used to perform word segmentation, removal of useless characters (redundant spaces, special symbols), and entity recognition operations to filter low-value information and improve text purity. Standardized terminology (e.g., "AI" is standardized as "artificial intelligence") ensures consistency in expression; sensitive information such as personal identification numbers and internal keys is identified and masked to ensure data security and output high-quality preprocessed data.

[0025] In other embodiments, for the text converted from audio, voiceprint recognition technology can be combined to assign weights to the speech segments of different speakers, distinguish the information of the core speaker from the auxiliary speaker, and prioritize the retention of key content.

[0026] This breaks down the format barriers of multimodal data (unstructured documents, audio, structured data, etc.), enabling unified data access, standardized cleaning, and quality control, thus providing a high-quality, unpolluted data foundation for subsequent in-depth processing.

[0027] In step S2, the main task is to perform deep semantic fusion processing of multimodal data to address the technical shortcomings of existing technologies that only perform parallel processing of multimodal data without deep semantic association.

[0028] See the instruction manual appendix Figure 2 The step of dividing the preprocessed data into multiple knowledge blocks according to semantic logic, and performing cross-modal semantic alignment of text and images on the knowledge blocks containing images and text, includes the following steps: S201. A recursive character text segmenter is used to cut the preprocessed data into multiple knowledge blocks of a set length and containing overlapping areas according to semantic logic. S202. For knowledge blocks containing images and text, identify the images and their associated descriptive text; S203. The image and its associated descriptive text are encoded into a unified vector space through the visual language pre-trained model CLIP to obtain cross-modal semantically aligned knowledge blocks.

[0029] In step S201, when performing knowledge block splitting, the Recursive Character Text Splitter from the LangChain framework can be used. Unlike traditional mechanical splitting based on a fixed number of characters, this tool prioritizes splitting based on natural semantic delimiters (such as paragraphs, periods, and semicolons), ensuring that each split knowledge block possesses independent and complete semantic logic, avoiding situations where "a sentence is split into two knowledge blocks" or "an operation process is split into multiple knowledge blocks." Preferably, the character length of the knowledge block is controlled within the range of 800-1500 characters, which is the optimal range verified through extensive experiments. Shorter than 800 characters can lead to overly fine knowledge granularity, failing to include complete knowledge point information; longer than 1500 characters increases the computational cost of subsequent vectorization and reduces the accuracy of retrieval. Simultaneously, an overlap zone of 100-200 characters is set, meaning that adjacent knowledge blocks have overlapping content. This design effectively connects the context of adjacent knowledge blocks, preventing the omission of cross-block related information during retrieval due to the independent existence of knowledge blocks.

[0030] In steps S202 and S203, for knowledge blocks containing images and text, the images and associated descriptive text are input into the CLIP (Contrastive Language-Image Pre-training) visual language pre-training model. The image encoder extracts features from the image, and the text encoder extracts features from the text. Then, a contrastive learning mechanism maps the feature encodings of both to the same 1024-dimensional vector space, generating a unified multimodal fusion vector. For pure text knowledge blocks without images, the text encoder of the CLIP model directly encodes the text, generating a text vector of the same dimension.

[0031] This enables deep semantic alignment of cross-modal data, allowing the system to truly understand the intrinsic relationships between different modalities, and providing a semantically consistent multimodal knowledge foundation for subsequent accurate retrieval and intelligent question answering.

[0032] Step S3 primarily aims to optimize the efficiency and quality of knowledge vectorization while ensuring data security, thus addressing the challenge of balancing the security of confidential data with the efficiency of processing non-sensitive data.

[0033] See the instruction manual appendix Figure 3 The process of identifying the sensitivity level of the knowledge block, dynamically selecting an embedding model based on the sensitivity level, and storing the vectors output by different embedding models and their corresponding metadata into a vector database, and partitioning and isolating them according to sensitivity level, includes the following steps: S301. Based on business needs, use the Drools rule engine to preset sensitivity judgment rules to determine the three-level standards of public, internal, and confidential. S302. Use the fine-tuned BERT model in conjunction with the rule engine Drools to identify the sensitivity level of the knowledge block; S303. Schedule the corresponding embedded model according to the sensitivity level; among which, the public / internal level schedules the cloud-based intelligent model, and the confidential level schedules the locally deployed lightweight model. S304. After standardizing the dimensions of the vectors output by different embedding models, store them together with the corresponding metadata in the vector database, partition and isolate them according to sensitivity level, and configure differentiated access strategies for different partitions.

[0034] In step S301, Drools is selected as the core engine for sensitive rule configuration. This engine is a lightweight business rule management system that adapts to the sensitivity classification needs of enterprises at different development stages and in different business scenarios. In one embodiment, based on the general security requirements of enterprise knowledge management, three sensitivity levels are preset: public, internal, and confidential. The public level refers to knowledge that can be published externally without security risks; the internal level refers to daily operational knowledge that can only be accessed by internal employees and cannot be disclosed externally; the confidential level refers to core knowledge that can only be accessed by specific authorized personnel (such as the core technical team and management), and whose disclosure would damage the enterprise's technical, economic, and commercial interests. Furthermore, through the Drools rule editing interface, the above classification standards are converted into executable business rules.

[0035] In step S302, this application adopts a hybrid judgment scheme that combines fine-tuned BERT model semantic analysis with precise matching by the Drools rule engine. By combining the semantic understanding capability of deep learning with the precise execution capability of the rule engine, it achieves automated and precise judgment of the sensitivity level of knowledge blocks. This solves the problems of low efficiency, strong subjectivity, and easy omissions in traditional manual classification, and the inability of pure rule classification to identify deep semantically sensitive information.

[0036] The process of selecting a Chinese pre-trained BERT model (such as bert-base-chinese) as the base model and fine-tuning and optimizing it based on the enterprise's proprietary dataset is a technical method well-known to those skilled in the art and will not be elaborated here.

[0037] The knowledge block obtained in step S2 is then input into the Drools rule engine. Explicit information matching is performed using the rule base, and the rule determination result is output. Simultaneously, the knowledge block is input into a fine-tuned BERT model for deep semantic analysis, outputting the model prediction result and confidence level. Finally, cross-validation is performed. If the rule determination result matches the model prediction result, that result is directly adopted as the final sensitivity level of the knowledge block. If the two results are inconsistent, the model prediction result with a confidence level ≥ 85% is prioritized, and this case is included in the rule base update samples for subsequent Drools rule optimization. If the model prediction confidence level is < 85%, manual judgment is performed, and the review result is simultaneously updated to the model training dataset and the rule base, achieving continuous iterative optimization of the model and rules. Finally, a knowledge block sensitivity classification table is generated.

[0038] In step S303, the corresponding embedding model is scheduled based on the sensitivity level of the knowledge block. Public / internal knowledge calls the cloud-based OpenAI text-embedding-3-large model API to ensure processing efficiency; confidential knowledge is switched to the locally deployed Qwen3-Embedding lightweight model to ensure the data remains within the domain. This achieves a differentiated vectorization strategy of "efficient processing of non-sensitive knowledge and continuous processing of confidential knowledge," maximizing processing efficiency while ensuring data security.

[0039] In step S304, the vectors output by different models are standardized using the Python Flask scheduling service to a uniform 1024-dimensional vector, preventing the difference in vector dimensions from affecting subsequent retrieval results. Furthermore, the generated vectors, along with their source, sensitivity level, and other metadata, are stored in the vector database Chroma. These vectors are then partitioned and isolated according to sensitivity levels, with differentiated access policies configured for different partitions (e.g., confidential partitions are accessible only to authorized personnel), balancing storage efficiency and data security.

[0040] Step S4 mainly improves the accuracy of complex queries by integrating and enhancing RAG knowledge retrieval, thereby meeting users' needs for efficiently obtaining accurate knowledge.

[0041] Specifically, upon receiving a user query request, on the one hand, based on a vector database, within the partition corresponding to the user's permissions, the similarity between the query vector and all knowledge block vectors is calculated, sorted in descending order of similarity score, and the Top-N (e.g., Top 30) knowledge blocks with the highest similarity are recalled; on the other hand, based on a knowledge graph database, deep logical connections and multi-hop reasoning relationships between entities are mined to obtain a list of logical relationship retrieval results. For example, when a user asks "How should side effect B of drug A be handled?": Path 1 (vector retrieval) retrieves knowledge blocks semantically similar to the question in the vector database Chroma. Path 2 (knowledge graph retrieval) queries the knowledge graph database Neo4j for the relationship path "Drug A - causes - side effect B - treatment measures".

[0042] Then, the results from the two retrieval paths are merged and reordered to generate an optimized context. Results that simultaneously contain semantic similarity and logical relevance are prioritized. This optimized context is then input into a locally deployed large language model, restricting the model to generate answers solely based on the provided context, avoiding unfounded "illusions," and outputting accurate and traceable query answers.

[0043] Step S5 mainly involves adaptive updates based on user behavior feedback, which solves the problems of knowledge updates relying on manual processes, poor timeliness, and insufficient activity, thereby reducing operation and maintenance costs.

[0044] See the instruction manual appendix Figure 4 The process of evaluating the value of user behavior data to generate a knowledge point lifecycle status table, and predictively updating the work order list based on the knowledge point lifecycle status table and knowledge point relationships, includes the following steps: S501. Capture user behavior data from all dimensions and perform cleaning and aggregation processing; wherein, the behavior data includes interaction data collected through the front end and log records collected through the back end; S502. Based on the aggregated behavioral data, a value assessment model is constructed, and the real-time popularity weight of each knowledge point is calculated by comprehensively considering multi-dimensional behavioral indicators, generating a knowledge point lifecycle status table; among which, multi-dimensional behavioral indicators include page visit frequency, dwell time, and positive feedback ratio. S503. Based on the knowledge point lifecycle status table and the knowledge point association relationship stored in the knowledge graph database, predictively update the work order list; the work order list includes task type and associated knowledge point ID.

[0045] In step S501, a dual-channel data collection scheme combining front-end tracking and back-end logging is used to capture user interaction behavior with the knowledge base from all dimensions, ensuring data comprehensiveness. Channel one involves embedding tracking code at key nodes in the user interaction layer to collect anonymized data of direct user interaction behavior. Core dimensions include: query-related data (such as query keywords, query time, query device type, and query scenario), result interaction data (such as click-through rate (CTR) of search results, the clicked knowledge point number, the duration of page dwell on the knowledge point, and the page scroll depth), explicit feedback data (such as likes / dislikes on answers, feedback reasons, and knowledge correction submissions), and operational behavior data (such as the number of knowledge downloads / favorites, the number of knowledge shares, and follow-up keywords in multi-turn dialogues). Channel Two utilizes a log service comprised of Elasticsearch, Logstash, and Kibana to record complete system-level data. Core dimensions include retrieval chain data (parsed results of user queries, recall lists from dual-path searches, reordered result sequences, and the final returned answer content), system response data (such as retrieval response time, large model generation time, and data call sources), and abnormal behavior data (number of queries with no results, multiple unclicked results for the same keyword, and knowledge point IDs with high-frequency negative feedback). Furthermore, the collected user behavior data undergoes data cleaning and deduplication, data aggregation and structuring, before being written to the Snowflake data warehouse, providing a unified, high-quality data foundation for subsequent analysis.

[0046] In step S502, a value assessment model for knowledge points is constructed based on the aggregated behavioral data. For example, Python is used for data calculation, and indicators such as page visit frequency, dwell time, and positive feedback ratio are combined to calculate the real-time popularity weight of each knowledge point, thereby obtaining a real-time popularity weight table for knowledge points. Among them, the weight of knowledge points with high frequency, long dwell time, and positive reviews is automatically increased.

[0047] Furthermore, combining popularity weights and user feedback, the Drools rule engine defines a status (e.g., Active, Pending Review, Archived) for each knowledge point. Active is defined as high-value, high-demand knowledge point with no negative feedback; Pending Review is defined as knowledge point with questionable value, negative feedback, or data anomalies; Archived is defined as low-value, no-demand, and outdated knowledge point. This generates a knowledge point lifecycle status table and outputs a list of administrator task notifications to drive manual review and subsequent processing actions.

[0048] In step S503, key relationships between knowledge points are mined using the knowledge graph data stored in Neo4j (e.g., API v2.0 is an upgrade of API v1.0). When a change in the state of associated knowledge is detected (e.g., a surge in API v2.0 access), work orders such as "archiving old versions" and "writing migration guides" are automatically created by calling the Apache Airflow API, enabling predictive updates. For example, when a sharp increase in API v2.0 document access is detected, the graph database can quickly traverse and locate its associated API v1.0 node. Task work orders such as "archiving API v1.0 documents" and "writing v1.0 to v2.0 migration guides" are automatically created and assigned to the corresponding content maintenance teams.

[0049] This AI-driven automated workflow transforms knowledge base maintenance from a reactive response to proactive management, significantly reducing operational costs. It dynamically adjusts the value and status of knowledge based on user data, prioritizing high-value content and archiving outdated information, thus significantly improving knowledge activity and timeliness.

[0050] As can be seen, the AI ​​knowledge base construction method provided in this application, based on multimodal fusion and adaptive updating, uses the joint vectorization technology of the visual language pre-trained model CLIP to map multimodal data such as text, images, and audio to a unified semantic vector space, achieving cross-modal deep understanding and semantic alignment. It automatically selects a high-performance cloud model or a local lightweight model based on the sensitivity level of the knowledge block, optimizing the processing efficiency of non-sensitive knowledge while ensuring that confidential data does not leave the domain. It constructs a knowledge point lifecycle model by analyzing user interaction data, automatically evaluating knowledge value and predicting update needs. Combining vector similarity retrieval and knowledge graph logical relationship query, the results are reordered and input into a large language model to improve the accuracy of complex queries.

[0051] Based on the same inventive concept, this application also provides an AI knowledge base construction device based on multimodal fusion and adaptive updating. Since the principle of the device in this application is similar to the AI ​​knowledge base construction method based on multimodal fusion and adaptive updating described above in this application, the implementation of the device can refer to the implementation of the method, and the repeated parts will not be described again.

[0052] As per the instruction manual Figure 5 As shown, this application also provides an AI knowledge base construction device based on multimodal fusion and adaptive updating, the device comprising: The acquisition module 501 is used to acquire multi-source data and parse and clean the acquired multi-source data to obtain preprocessed data; among which, multi-source data includes unstructured documents, audio data, and structured / semi-structured data; The knowledge block segmentation module 502 is used to segment the preprocessed data into multiple knowledge blocks according to semantic logic, and to perform cross-modal semantic alignment of text and images on knowledge blocks containing images. Sensitivity level identification module 503 is used to identify the sensitivity level of the knowledge block, dynamically select the embedding model based on the sensitivity level, and store the vectors output by different embedding models and the corresponding metadata into the vector database, and partition and isolate them according to the sensitivity level. The dual-path retrieval module 504 is used to perform semantic similarity retrieval and entity relationship logic retrieval when receiving user queries, and to fuse and reorder the retrieval results of the two paths to obtain an optimized context, which is then input into the locally deployed large language model. The adaptive update module 505 is used to evaluate the value of user behavior data, generate a knowledge point lifecycle status table, and predictively update the work order list based on the knowledge point lifecycle status table and the knowledge point association relationship.

[0053] In some embodiments, the acquisition module 501 acquires multi-source data and parses and cleans the acquired multi-source data to obtain preprocessed data, including: acquiring multi-source data by connecting to the corresponding database table or API interface; parsing and converting multi-source data of different formats into a unified text format; and using natural language processing tools to perform data cleaning operations such as word segmentation, removal of useless characters, and entity recognition to obtain preprocessed data.

[0054] In some embodiments, the knowledge block segmentation module 502 segments the preprocessed data into multiple knowledge blocks according to semantic logic, and performs cross-modal semantic alignment of text and images on the knowledge blocks containing images and text. This includes: using a recursive character text segmenter to segment the preprocessed data into multiple knowledge blocks of a set length and containing overlapping areas; for knowledge blocks containing images and text, identifying the image and its associated descriptive text; and encoding the image and its associated descriptive text into a unified vector space using a visual language pre-trained model CLIP to obtain cross-modal semantically aligned knowledge blocks. The character length of the knowledge block is 600-2000; the character length of the overlapping area is 100-200.

[0055] In some embodiments, the sensitivity level identification module 503 identifies the sensitivity level of the knowledge block, dynamically selects an embedding model based on the sensitivity level, and stores the vectors output by different embedding models and their corresponding metadata into a vector database, partitioning and isolating them according to sensitivity level. This includes: determining three levels of standards—public, internal, and confidential—based on business needs by using the Drools rule engine to preset sensitivity judgment rules; identifying the sensitivity level of the knowledge block using a fine-tuned BERT model combined with the Drools rule engine; scheduling the corresponding embedding model according to the sensitivity level; wherein, the public / internal level schedules a cloud-based intelligent model, and the confidential level schedules a locally deployed lightweight model; after standardizing the dimensions of the vectors output by different embedding models, they are stored in the vector database along with the corresponding metadata, partitioned and isolated according to sensitivity level, and different access strategies are configured for different partitions.

[0056] In some embodiments, the dual-path retrieval module 504 performs semantic similarity retrieval and entity relationship logic retrieval, and fuses and reorders the retrieval results of the two paths to obtain an optimized context, which is then input into a locally deployed large language model. This includes: when receiving a user query, performing a dual-path hybrid retrieval; wherein, semantic similarity retrieval is performed in a vector database to match knowledge blocks related to the query semantics; and entity relationship logic query is performed in a knowledge graph database to mine the relationships between knowledge points; the results obtained from the two retrieval paths are fused and reordered to obtain an optimized context; wherein, results that simultaneously contain semantic similarity and logical relevance are preferentially retained; and the optimized context is input into the locally deployed large language model to restrict the large language model from generating answers based on the provided context.

[0057] In some embodiments, the adaptive update module 505 performs knowledge point value assessment on user behavior data, generates a knowledge point lifecycle status table, and predictively updates the work order list based on the knowledge point lifecycle status table and knowledge point associations. This includes: capturing user behavior data across all dimensions and performing cleaning and aggregation processing; wherein the behavior data includes interaction data collected from the front end and log records collected from the back end; constructing a value assessment model based on the aggregated behavior data, calculating the real-time popularity weight of each knowledge point by comprehensively considering multi-dimensional behavior indicators, and generating a knowledge point lifecycle status table; wherein the multi-dimensional behavior indicators include page visit frequency, dwell time, and positive feedback ratio; and predictively updating the work order list based on the knowledge point lifecycle status table and knowledge point associations stored in the knowledge graph database; the work order list includes task type and associated knowledge point ID.

[0058] This application provides an AI knowledge base construction device based on multimodal fusion and adaptive updating. The device collects multi-source data through an acquisition module, and parses and cleans the collected data to obtain preprocessed data. A knowledge block segmentation module segments the preprocessed data into multiple knowledge blocks according to semantic logic, and performs cross-modal semantic alignment of text and images in knowledge blocks containing images. A sensitivity level identification module identifies the sensitivity level of the knowledge blocks, dynamically selects an embedding model based on the sensitivity level, and stores the vectors output by different embedding models and their corresponding metadata in a vector database, partitioning and isolating them according to sensitivity level. When a user query is received, a dual-path retrieval module performs semantic similarity retrieval and entity relationship logic retrieval, and fuses and reorders the retrieval results of the two paths to obtain an optimized context, which is then input into a locally deployed large language model. An adaptive updating module evaluates the knowledge point value of user behavior data, generates a knowledge point lifecycle status table, and predictively updates the work order list based on the knowledge point lifecycle status table and knowledge point associations. This achieves deep multimodal data fusion, secure and efficient knowledge processing, accurate response to complex queries, and adaptive updating of the knowledge base.

[0059] Based on the same concept of the present invention, the specification is attached. Figure 6 As shown in the figure, an embodiment of this application provides the structure of an electronic device 600, which includes: at least one processor 601, at least one network interface 604 or other user interface 603, memory 605, and at least one communication bus 602. The communication bus 602 is used to realize the connection and communication between these components. The electronic device 600 may optionally include a user interface 603, including a display (e.g., touch screen, LCD, CRT, holographic imaging, or projector, etc.), a keyboard, or a clicking device (e.g., mouse, trackball, touchpad, or touch screen, etc.).

[0060] Memory 605 may include read-only memory and random access memory, and provides instructions and data to processor 601. A portion of memory 605 may also include non-volatile random access memory (NVRAM).

[0061] In some implementations, memory 605 stores elements that can protect modules or data structures, or subsets thereof, or extended sets thereof: The 6051 operating system contains various system programs used to implement various basic business functions and handle hardware-based tasks. Application module 6052 contains various applications, such as desktop (launcher), media player (MediaPlayer), browser (Browser), etc., to implement various application services.

[0062] In this embodiment of the application, by calling the program or instructions stored in the memory 605, the processor 601 is used to execute the steps in an AI knowledge base construction method based on multimodal fusion and adaptive updating, which can realize deep fusion of multimodal data, secure and efficient processing of knowledge, accurate response to complex queries, and adaptive updating of the knowledge base.

[0063] This application also provides a computer-readable storage medium storing a computer program that is executed by a processor, such as the steps in the AI ​​knowledge base construction method based on multimodal fusion and adaptive updating.

[0064] Specifically, the storage medium can be a general-purpose storage medium, such as a portable disk or hard drive. When the computer program on the storage medium is run, it can execute the above-mentioned AI knowledge base construction method based on multimodal fusion and adaptive updating.

[0065] In the embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. The apparatus embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and there may be other division methods in actual implementation. Furthermore, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Additionally, the coupling or direct coupling or communication connection shown or discussed may be through some communication interface; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0066] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0067] In addition, the functional units in the embodiments provided in this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0068] If a function is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0069] Finally, it should be noted that the above embodiments are merely specific implementations of this application, used to illustrate the technical solutions of this application, and not to limit them. The protection scope of this application is not limited thereto. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that any person skilled in the art can still modify or easily conceive of changes to the technical solutions described in the foregoing embodiments, or make equivalent substitutions for some of the technical features, within the scope of the technology disclosed in this application; and these modifications, changes, or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application. All should be covered within the protection scope of this application. Therefore, the protection scope of this application should be determined by the protection scope of the claims.

Claims

1. A method for constructing an AI knowledge base based on multimodal fusion and adaptive updating, characterized in that, The method includes the following steps: Collect multi-source data, and parse and clean the collected multi-source data to obtain preprocessed data; the multi-source data includes unstructured documents, audio data, and structured / semi-structured data; The preprocessed data is divided into multiple knowledge blocks according to semantic logic, and cross-modal semantic alignment of text and images is performed on the knowledge blocks containing images. The knowledge blocks are identified by sensitivity level, and the embedding model is dynamically selected based on the sensitivity level. The vectors output by different embedding models and the corresponding metadata are stored in the vector database and partitioned and isolated according to sensitivity level. When a user query is received, semantic similarity retrieval and entity relationship logic retrieval are performed, and the retrieval results of the two paths are merged and reordered to obtain an optimized context, which is then input into the locally deployed large language model. The knowledge point value of user behavior data is evaluated, a knowledge point lifecycle status table is generated, and the work order list is predictively updated based on the knowledge point lifecycle status table and the knowledge point association relationship.

2. The AI ​​knowledge base construction method based on multimodal fusion and adaptive updating according to claim 1, characterized in that, The process of collecting multi-source data and parsing and cleaning the collected multi-source data to obtain preprocessed data includes the following steps: Collect data from multiple sources by connecting to the corresponding database tables or API interfaces; Multi-source data in different formats are parsed and converted into a unified text format. Natural language processing tools are then used to perform data cleaning operations such as word segmentation, removal of useless characters, and entity recognition to obtain preprocessed data.

3. The method for constructing an AI knowledge base based on multimodal fusion and adaptive updating according to claim 1, characterized in that, The step of dividing the preprocessed data into multiple knowledge blocks according to semantic logic, and performing cross-modal semantic alignment of text and images in the knowledge blocks containing images and text, includes the following steps: A recursive character text segmenter is used to cut the preprocessed data into multiple knowledge blocks of a set length that contain overlapping areas according to semantic logic. For knowledge blocks containing both images and text, identify the images and their associated descriptive text; The visual language pre-trained model CLIP encodes images and their associated descriptive text into a unified vector space, resulting in cross-modal semantically aligned knowledge blocks.

4. The AI ​​knowledge base construction method based on multimodal fusion and adaptive updating according to claim 3, characterized in that, The process of identifying the sensitivity level of the knowledge block, dynamically selecting the embedding model based on the sensitivity level, and storing the vectors output by different embedding models and their corresponding metadata into a vector database, and partitioning and isolating them according to sensitivity level, includes the following steps: Based on business needs, the Drools rule engine is used to preset sensitivity judgment rules to determine the three levels of standards: public, internal, and confidential. The sensitivity level of the knowledge block is identified by using a fine-tuned BERT model combined with the rule engine Drools. The embedded model is scheduled according to the sensitivity level; among them, the public / internal level schedules the cloud-based intelligent model, and the confidential level schedules the locally deployed lightweight model. After the vectors output by different embedding models are dimensionally standardized, they are stored in the vector database along with the corresponding metadata. They are then partitioned and isolated according to sensitivity levels, and different access strategies are configured for different partitions.

5. The AI ​​knowledge base construction method based on multimodal fusion and adaptive updating according to claim 4, characterized in that, When a user query is received, semantic similarity retrieval and entity relationship logic retrieval are performed. The retrieval results from the two paths are then merged and reordered to obtain an optimized context, which is then input into the locally deployed large language model. This process includes the following steps: When a user query is received, a dual-path hybrid retrieval is performed; that is, semantic similarity retrieval is performed in the vector database to match knowledge blocks that are semantically related to the query; and entity relationship logic query is performed in the knowledge graph database to mine the relationships between knowledge points. The results from the two search paths are merged and reordered to obtain an optimized context; among them, results that simultaneously contain semantic similarity and logical relevance are retained first. The optimized context is input into a locally deployed large language model to restrict the large language model from generating answers based on the provided context.

6. The method for constructing an AI knowledge base based on multimodal fusion and adaptive updating according to claim 5, characterized in that, The process of evaluating the value of user behavior data for knowledge points, generating a knowledge point lifecycle status table, and predictively updating the work order list based on the knowledge point lifecycle status table and knowledge point relationships includes the following steps: The system captures user behavior data from all dimensions and performs cleaning and aggregation processing; the behavior data includes interaction data collected from the front end and log records collected from the back end. A value assessment model is built based on aggregated behavioral data. The real-time popularity weight of each knowledge point is calculated by integrating multi-dimensional behavioral indicators, and a knowledge point lifecycle status table is generated. Among them, multi-dimensional behavioral indicators include page visit frequency, dwell time, and positive feedback ratio. The work order list is predictively updated based on the knowledge point lifecycle status table and the knowledge point associations stored in the knowledge graph database; the work order list includes task type and associated knowledge point ID.

7. The AI ​​knowledge base construction method based on multimodal fusion and adaptive updating according to claim 3, characterized in that, in, When the preprocessed data is divided into multiple knowledge blocks according to semantic logic using a recursive character text segmenter, the character length of the knowledge block is 600-2000; the character length of the overlapping area is 100-200.

8. An AI knowledge base construction device based on multimodal fusion and adaptive updating, characterized in that, The device includes: The acquisition module is used to collect multi-source data and parse and clean the collected multi-source data to obtain preprocessed data; the multi-source data includes unstructured documents, audio data, and structured / semi-structured data; The knowledge block segmentation module is used to segment the preprocessed data into multiple knowledge blocks according to semantic logic, and to perform cross-modal semantic alignment of text and images for knowledge blocks containing images. The sensitivity level identification module is used to identify the sensitivity level of the knowledge block, dynamically select the embedding model based on the sensitivity level, and store the vectors output by different embedding models and the corresponding metadata into the vector database, which is partitioned and isolated according to the sensitivity level. The dual-path retrieval module is used to perform semantic similarity retrieval and entity relationship logic retrieval when receiving user queries, and to fuse and reorder the retrieval results of the two paths to obtain an optimized context, which is then input into the locally deployed large language model. The adaptive update module is used to evaluate the value of knowledge points in user behavior data, generate a knowledge point lifecycle status table, and predictively update the work order list based on the knowledge point lifecycle status table and the knowledge point association relationship.

9. An electronic device, characterized in that, include: The device includes a processor, a memory, and a bus. The memory stores machine-readable instructions executable by the processor. When the electronic device is running, the processor communicates with the memory via the bus. When the machine-readable instructions are executed by the processor, the steps of the AI ​​knowledge base construction method based on multimodal fusion and adaptive updating as described in any one of claims 1 to 7 are performed.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, performs the steps of the AI ​​knowledge base construction method based on multimodal fusion and adaptive updating as described in any one of claims 1 to 7.

Images