A method and system for fully localized hybrid retrieval and knowledge graph construction in the research and production of standard gases.

Abstract

This invention relates to a fully localized hybrid retrieval and knowledge graph construction method and system for standard gas R&D and production, belonging to the interdisciplinary field of artificial intelligence and industrial control. The method includes: acquiring multi-source heterogeneous documents and performing text parsing, mixed Chinese and English word segmentation, and terminology standardization; employing a hybrid retrieval strategy that weights and fuses BM25 keyword retrieval and vector semantic retrieval to obtain fused retrieval results; automatically extracting entities and relations from the fused retrieval results to generate triplet data; constructing a knowledge graph based on the triplet data, with nodes bearing source document identifiers and version tags; and deploying the embedded model, text units, semantic vectors, and knowledge graph on a local server. This invention achieves the fusion of precise matching and semantic recall, ensures core data security, transforms unstructured knowledge into a computable and traceable knowledge network, and significantly improves the efficiency and intelligence level of knowledge retrieval in standard gas R&D and production.

Description

Technical Field

[0001] This invention belongs to the interdisciplinary field of artificial intelligence and industrial control, specifically involving a method and system for fully localized hybrid retrieval and knowledge graph construction of standard gas research and production. Background Technology

[0002] Standard gases, as key substances in metrology standards, environmental monitoring, and instrument calibration, are characterized by high precision, high risk, diverse varieties, and small batches in their research and production. During this process, technical personnel frequently need to consult interdisciplinary resources, including metrology, physical chemistry, safety engineering, and equipment engineering. These resources come in various forms, such as Chinese and English literature, industry standards, internal company reports, and historical process documents, resulting in a dispersed and diverse range of knowledge sources. Traditionally, the acquisition and utilization of this knowledge relies primarily on the following methods, each with significant drawbacks: First, manual retrieval and manual review are inefficient and prone to omissions. Technical personnel need to browse and compare a large number of paper or electronic documents one by one, which consumes a lot of time and energy. Especially when facing R&D and production tasks with multiple varieties, small batches, and frequent changes, manual retrieval is difficult to fully cover all relevant knowledge points, and it is easy to miss key information, affecting the integrity of R&D demonstration and the reliability of process decisions.

[0003] Secondly, single-keyword searches struggle to cover synonyms, abbreviations, and cross-linguistic expressions, resulting in incomplete and inaccurate search results. For example, in the field of standard gases, terms such as "dew point," "explosion limit," and "mass ratio" may be used interchangeably in different documents. Traditional keyword-matching-based search systems cannot identify these semantic equivalence relationships, leading to either missing a large amount of relevant content or returning a large amount of noisy data, making it difficult to balance search precision and recall.

[0004] Furthermore, existing knowledge management systems lack a mechanism to combine "precise terminology matching" with "semantic similarity recall," and struggle to structure search results into a computable and traceable knowledge network. For example, Chinese patent CN118296165A proposes a method for constructing a dynamic process knowledge base for a production line, generating process solutions through a combination of case-based reasoning and rule-based reasoning. However, its retrieval mechanism primarily relies on preset rules and case similarity matching, failing to achieve a deep integration of precise matching and semantic retrieval. Its knowledge organization method also fails to dynamically transform search results into a structured knowledge graph to support multi-round reasoning. Chinese patent CN114647741A discloses a process reasoning method based on temporal knowledge graphs and partially observable Markov decision processes. However, its knowledge graph construction relies on predefined ontology structures and historical data training, lacking dynamic linkage with hybrid retrieval mechanisms, making it difficult to adapt to the rapid retrieval and knowledge extraction needs of multi-source heterogeneous documents in standard gas R&D and production.

[0005] More importantly, the reliance of mainstream large-scale models on external network services poses a risk of leakage of sensitive data such as core formulas and production parameters, making it difficult for enterprises to use them in compliance with regulations. With the development of artificial intelligence technology, some enterprises have begun to try to use large language models to assist in knowledge retrieval and question answering. For example, Chinese patent CN119808931A discloses a method for constructing a large aerospace knowledge model based on progressive knowledge injection and retrieval enhancement generation. Although this method introduces retrieval enhancement generation (RAG) technology, its model deployment relies on cloud services or external API calls. For standard gas R&D and production enterprises involving highly sensitive data such as core formulas and production process parameters, this external network deployment mode poses a compliance risk of data leakage and is difficult to meet the high data security control requirements of enterprises.

[0006] In summary, a common deficiency in existing technologies lies in the lack of an integrated system that, while ensuring core data security, integrates precise terminology matching and semantic similarity recall, and dynamically structures the search results into a computable and traceable knowledge network to directly support R&D validation and process decision-making. How to construct a fully localized deployment method that combines high accuracy and high recall, and can transform unstructured knowledge into a structured knowledge network, is a technical challenge that urgently needs to be addressed by those skilled in the art. Summary of the Invention

[0007] In view of this, the present invention aims to address the technical deficiencies in the existing research and development and production of standard gases: First, manual retrieval and keyword retrieval are inefficient and lack precision, making it difficult to cover synonyms, abbreviations, and cross-language expressions; second, existing AI systems rely on external network deployment, posing compliance risks of leakage of sensitive data such as core formulas and production parameters; third, existing knowledge management systems lack a mechanism that combines "precise terminology matching" and "semantic similarity recall," making it difficult to structure search results into a computable and traceable knowledge network, and thus unable to directly support research and development demonstration and process decision-making.

[0008] To achieve the above objectives, this invention provides a fully localized hybrid retrieval and knowledge graph construction method and system for standard gas research and development and production.

[0009] A fully localized hybrid retrieval and knowledge graph construction method for standard gas R&D and production includes: Data preprocessing: Obtain multi-source heterogeneous documents, and perform text parsing, Chinese-English mixed word segmentation, terminology standardization, and stop word filtering on the multi-source heterogeneous documents to obtain multiple text units; Hybrid retrieval: Upon receiving a user query, the BM25 keyword retrieval algorithm is used to precisely match the text units to obtain a first retrieval result; a locally deployed embedding model is used to convert the user query and the text units into semantic vectors respectively, and semantic retrieval is performed based on vector similarity to obtain a second retrieval result; according to the density of standard gas domain terms involved in the user query, the weighting coefficients of the weighted fusion are dynamically adjusted, and the first retrieval result and the second retrieval result are weighted and fused to obtain a fused retrieval result; Structured extraction: Entities and relations are automatically extracted from the fused retrieval results to generate triplet data. The entities include one or more of the following: gas components, process parameters, safety thresholds, and equipment models. The extraction process is performed based on a combination of a predefined domain entity dictionary and a locally deployed named entity recognition model. Knowledge Graph Construction: A knowledge graph is constructed based on the triple data. The nodes of the knowledge graph are labeled with source document identifiers, paragraph positions, and version tags, while the edges of the knowledge graph are labeled with relation types and source evidence. Through the multidimensional metadata of nodes and edges, this knowledge graph enables bidirectional tracing from any knowledge node to its original document source, specific paragraph position, and effective version of the knowledge, providing a complete chain of evidence to support subsequent process decisions, compliance reviews, and knowledge updates. Fully localized deployment: The embedded model, text units, semantic vectors, and knowledge graph are all deployed on local servers. All core data is stored and processed within the enterprise's internal network environment and is not transmitted to any external network servers or third-party cloud platforms, in order to meet the compliance control requirements of standard gas companies for sensitive data such as core formulas and process parameters.

[0010] Preferably, in the hybrid retrieval step, the weighted fusion calculation formula is:

[0011] Where α is the weighting coefficient of BM25 retrieval. The BM25 score for document d is calculated using the following formula:

[0012]

[0013] in, For words Frequency of occurrence in document d The length of document d, The average length of all documents is given by k1, and k1 and b are adjustable parameters. The inverse document frequency is N, where N is the total number of documents. For the word q i The number of documents.

[0014] The formula for calculating semantic similarity is:

[0015] in, For the semantic vector of the user query, This is the semantic vector of a text unit.

[0016] Preferably, the data preprocessing further includes: for multi-source heterogeneous documents containing images, calling a computer vision model to extract image features and generate image description text, and storing the image feature vectors in a vector database; for multi-source heterogeneous documents containing tables, retaining the row and column structure and combining the table headers and cell contents to generate table description text; for multi-source heterogeneous documents containing formulas, extracting the LaTeX representation of the formulas and generating a natural language description of the formulas; the image description text, table description text, and natural language description of the formulas participate in the hybrid retrieval step.

[0017] Preferably, the structured extraction adopts a strategy that combines rule-based matching and model-based extraction: entity extraction is performed by string matching based on a predefined entity dictionary, and entity extraction is performed based on a locally deployed named entity recognition model; the relation extraction is performed based on dependency parsing and predefined relation templates.

[0018] Preferably, the method further includes adaptive learning: when a user manually corrects the extraction result, the difference before and after the correction is recorded as a training sample, the training sample is used periodically to incrementally fine-tune the named entity recognition model, and the entity dictionary is updated.

[0019] Preferably, the knowledge graph construction further includes: when a new document is added, extracting entities and relations from the new document to obtain new triples; calculating the similarity between the newly extracted entities and existing entities in the knowledge graph, merging them into the same node and updating the node attributes when the similarity exceeds a preset threshold; and recording the conflict and triggering a manual review process when the newly extracted relations contradict existing relations.

[0020] Preferably, the method further includes session state management: assigning a unique session ID to each user session, recording the user's query history, hit nodes, and used evidence; when a new query is received in the same session, concatenating the session state summary with the current query to form an enhanced query, and re-executing the hybrid retrieval step based on the enhanced query; when the similarity between the new query and the historical query exceeds a preset threshold, directly returning the historical retrieval results.

[0021] Preferably, the method further includes proactive recommendation: identifying the user's query intent based on a locally deployed intent classification model; and proactively recommending knowledge content that is related to the user's followed knowledge points and has not been retrieved, based on the knowledge points the user has followed in the session history and through node similarity calculation of the knowledge graph.

[0022] This invention also provides a fully localized hybrid retrieval and knowledge graph construction system for standard gas R&D and production. The system includes: a multi-source document pre-embedding module for performing the data preprocessing step; a hybrid retrieval engine for performing the hybrid retrieval step; a structured extraction module for performing the structured extraction step; a knowledge graph construction module for performing the knowledge graph construction step; a session state management module for performing the session state management step; and a web-based interactive platform for providing a retrieval interface, knowledge graph visualization, and user feedback annotation functions.

[0023] Compared with the prior art, the beneficial effects of the present invention are as follows: (1) This invention uses a hybrid retrieval mechanism that combines BM25 keyword retrieval and vector semantic retrieval with weighted fusion. This mechanism retains the advantage of precise matching at the term level and achieves semantic recall of synonyms, abbreviations and cross-language expressions. It significantly improves the accuracy and coverage of document retrieval in the field of standard gases and solves the problem of incomplete and inaccurate traditional keyword retrieval.

[0024] (2) The present invention adopts a fully localized deployment architecture, which deploys the embedded model, text unit, semantic vector and knowledge graph on the local server. Sensitive data such as core formula and production parameters are not transmitted to the external network, which completely avoids the risk of data leakage caused by external network deployment and meets the enterprise's data security compliance requirements.

[0025] (3) This invention realizes knowledge structuring and traceability. The unstructured text is converted into triple data through the structured extraction module, and a lightweight knowledge graph is constructed. The graph nodes have source documents, paragraph positions and version tags, which support knowledge tracing, updating and reuse, so that the search results can directly serve parameter calculation, process comparison and risk review.

[0026] (4) The present invention records the user's query history, hit nodes and used evidence through the session state management module, realizes context-enhanced retrieval and avoidance of repeated retrieval, and improves the coherence and efficiency of multi-turn dialogue.

[0027] (5) Through adaptive learning mechanism, cross-document association discovery and proactive recommendation function, the present invention enables the system to continuously learn and evolve from user feedback, automatically discover implicit knowledge associations, and proactively push relevant content, which significantly improves user experience and knowledge utilization efficiency.

[0028] Other advantages, objectives, and features of the invention will be set forth in part in the description which follows, and in part will be apparent to those skilled in the art from the following examination, or may be learned from practice of the invention. The objectives and other advantages of the invention can be realized and obtained through the following description. Attached Figure Description

[0029] To make the objectives, technical solutions, and advantages of the present invention clearer, the preferred embodiments of the present invention will be described in detail below with reference to the accompanying drawings, wherein: Figure 1 This is an architectural block diagram of a fully localized hybrid retrieval and knowledge graph construction system for the research and production of standard gases according to the present invention; Figure 2 This is a schematic diagram of the weighted fusion process of the hybrid search engine in an embodiment of the present invention. Detailed Implementation

[0030] The following specific examples illustrate the implementation of the present invention. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and various details in this specification can be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of the present invention. Unless otherwise specified, the following embodiments and features can be combined with each other.

[0031] The accompanying drawings are for illustrative purposes only and are schematic diagrams, not actual pictures. They should not be construed as limiting the invention. To better illustrate the embodiments of the invention, some parts in the drawings may be omitted, enlarged, or reduced, and do not represent the actual product dimensions. It is understandable to those skilled in the art that some well-known structures and their descriptions may be omitted in the drawings.

[0032] Example 1 This embodiment provides a fully localized hybrid retrieval and knowledge graph construction system for the research and production of standard gases. This system aims to solve the technical problems of fragmented knowledge, incomplete retrieval, high data security risks, and difficulty in structured utilization of knowledge in existing technologies.

[0033] The system architecture is as follows Figure 1 As shown, it mainly includes: a multi-source document pre-embedding module, a Chinese and English corpus processing module, a hybrid retrieval engine, a structured extraction module, a knowledge graph construction module, a session state management module, and an optional web-based interactive platform.

[0034] 1. Multi-source document pre-embedding module The multi-source document pre-embedding module serves as the system's data entry point, pre-loading heterogeneous documents scattered across different sources into the local system to provide a foundational corpus for subsequent retrieval and knowledge graph construction. Its data sources include, but are not limited to, technical standards and specifications, Chinese and English literature, internal company reports and process documents (historical R&D records, production formulas, process parameter tables, quality inspection reports, equipment operation manuals, and fault case libraries), as well as professional dictionaries and terminology databases. The multi-source document pre-embedding module employs a hierarchical storage strategy: original documents are stored as files in the local file system, while document metadata, such as document name, type, source, version number, publication date, and author, is extracted and stored in a relational database for easy filtering and sorting during subsequent retrieval.

[0035] 2. Chinese and English Corpus Processing Module The Chinese and English corpus processing module is responsible for performing unified text preprocessing on pre-embedded multi-source heterogeneous documents, laying the foundation for subsequent hybrid retrieval. This module includes the following sub-modules: (1) Document parsing and format conversion: For documents in different formats (PDF, Word, Excel, TXT, etc.), call the corresponding parser to extract the plain text content. For scanned PDF documents, call the optical character recognition (OCR) tool to recognize the text in the image. For tabular data, retain its structured information for subsequent entity extraction. For documents containing mathematical formulas, use OCR or a LaTeX parser to extract the LaTeX representation of the formulas and generate a natural language description of the formulas so that the formula content can participate in subsequent semantic retrieval.

[0036] (2) Chinese-English mixed word segmentation: Use a word segmentation tool that supports Chinese-English mixing, such as jieba segmentation, configured to support English word recognition, and perform word segmentation on the extracted text. For Chinese text, recognize the word boundaries; for English text, split the words by spaces and punctuation marks, and retain proper nouns, such as "CO", "N2", "GB / T 5274.1", etc., as complete units.

[0037] (3) Terminology dictionary expansion: Based on the pre-set professional terminology library, perform terminology recognition and normalization on the word segmentation results. For example, normalize "dew point temperature" to "dew point", and establish a cross-language association between "dew point" and "露点". The terminology dictionary supports user-defined addition to adapt to the specific terms within the enterprise.

[0038] (4) Stop word filtering and stemming: Filter out meaningless stop words, such as "的", "了", "the", "a", etc., and perform stemming on English words to improve the generalization ability of the retrieval.

[0039] The processed text is segmented by paragraphs or sentences, and each text unit (chunk) retains its source document, chapter location, paragraph number and other meta-information, and is stored in the local text database.

[0040] 3. Hybrid retrieval engine The hybrid retrieval engine is the core of the system. It adopts a strategy of weighted fusion of BM25 keyword retrieval and vector semantic retrieval to achieve accurate matching and semantic recall of user queries. The specific steps are as follows: 3.1 Query preprocessing When the user enters a query statement, such as "How to prepare a 10 ppm carbon monoxide standard gas", the system first performs the same preprocessing on the query statement as on the document: Chinese-English mixed word segmentation, terminology normalization, stop word filtering, etc., to obtain the query word list Q = {q1, q2, …, q m}.

[0041] 3.2 BM25 keyword retrieval BM25 is a term frequency-based retrieval algorithm used to calculate a relevance score between a document and a query. For each word in query Q... The formula for calculating the BM25 score is as follows:

[0042] in, For words Frequency of occurrence in document d The length of document d, k1 represents the average length of all documents, and b are adjustable parameters (k1=1.2, b=0.75 in this embodiment). The inverse document frequency is calculated using the following formula:

[0043] Where N is the total number of documents, For the word q i The number of documents.

[0044] The total BM25 score for document d and query Q is the sum of the scores for all terms:

[0045] The system calculates the BM25 score for all documents and sorts them from highest to lowest score, taking the top scores. (like =100 documents were used as keywords for search results.

[0046] 3.3 Vector Semantic Retrieval Vector semantic retrieval is based on a locally deployed embedding model, which converts text into high-dimensional vectors and achieves semantic matching through vector similarity calculation. This embodiment uses a locally deployed lightweight embedding model that supports mixed Chinese and English corpora and can map text fragments to a 384-dimensional or 768-dimensional semantic vector space.

[0047] For each text unit output by the Chinese and English corpus processing module, the embedding model is called to generate the corresponding semantic vector. The preprocessed query statements are then stored in a local vector database, which indexes each vector to support efficient Approximate Nearest Neighbor (ANN) search. The preprocessed query statements are also converted into query vectors using an embedding model. The algorithm retrieves the k text units most similar to the query vector from the vector database, using cosine similarity as the distance metric.

[0048] The cosine similarity value ranges from -1 to 1, with larger values ​​indicating greater semantic similarity. The system returns the top results with the highest similarity. Each text unit serves as a semantic retrieval result.

[0049] 3.4 Weighted Fusion and Denoising Rearrangement Figure 2 The weighted fusion process of the hybrid search engine in this embodiment is illustrated. The hybrid search engine weighted and fused the BM25 search results and semantic search results to obtain the final search result list. For documents that appear in both search results, their comprehensive score is calculated as follows:

[0050] Wherein, α is the weight coefficient of BM25 retrieval (0≤α≤1), and in this embodiment, α=0.6 to retain the advantage of precise term-level matching. and The scores are BM25 score and semantic similarity score, respectively, and need to be normalized before fusion. For documents that appear only in one search result, their original score is directly multiplied by the corresponding weight.

[0051] It's important to note that the weighting coefficient α is not a fixed value, but rather dynamically adjusted based on the user's query content. The system has a built-in standard gas domain terminology database. When it detects that a user's query contains high-density domain terms (such as "dew point," "explosion limits of carbon monoxide," and "mass ratio calculation formula"), the system determines that the user has high precision requirements and automatically increases the weighting coefficient α of the BM25 search to 0.7~0.8 to enhance terminology-level precision matching. Conversely, when the user's query is an open-ended question with relatively vague natural language descriptions, the system appropriately decreases α to 0.4~0.5 to enhance the recall effect of semantic retrieval. This dynamic weighting adjustment mechanism based on query content allows the hybrid retrieval strategy to better adapt to the diverse knowledge query needs in standard gas R&D and production scenarios.

[0052] After merging, the system sorts the scores in descending order and performs noise reduction and reordering: duplicate documents are removed, and when multiple text units from the same document are retrieved, only the unit with the highest score is retained; the sorting is fine-tuned based on factors such as the authority of the document source (e.g., national standards take precedence over internal company reports) and timeliness. Finally, the system returns the Top-K most relevant text units as the search results.

[0053] 4. Structured Extraction Module The structured extraction module receives the search results returned by the hybrid search engine and automatically extracts entities and relationships from them to provide structured data for knowledge graph construction.

[0054] Entity recognition employs a strategy combining rule-based matching and model-based extraction: on one hand, it performs string matching on text based on a predefined entity dictionary, such as a list of gas names, a list of equipment models, and a list of units of measurement; on the other hand, for entities not covered by the rules, a locally deployed Named Entity Recognition (NER) model is used for extraction. This embodiment uses a lightweight BERT model fine-tuned on a standard gas domain corpus, capable of recognizing entity types such as "gas composition," "process parameters," "safety thresholds," and "equipment models."

[0055] Relation extraction employs a method based on dependency parsing and rule templates: First, dependency parsing is performed on the text to identify subject-verb-object structures, modification relations, etc. Then, matching is performed using predefined relation extraction templates, for example: "The concentration of A is B" → "Concentration" relation (A, B), "The dew point of A under pressure B is C" → "Dew point" relation (A, B, C), "A should be prepared using method B" → "Preparation method" relation (A, B), "The standard of A is B" → "Reference standard" relation (A, B). The relation extraction results are represented in the form of triples: (head entity, relation type, tail entity), for example: ("Carbon monoxide", "Concentration", "10ppm"), ("Carbon monoxide", "Lower explosive limit", "12.5%"), ("Carbon monoxide", "Filling pressure", "12.5MPa").

[0056] 5. Knowledge Graph Construction Module The knowledge graph construction module receives triples output by the structured extraction module, constructs a lightweight knowledge graph, and supports the storage, updating, and querying of the graph.

[0057] This embodiment uses a graph database to store the knowledge graph, with each entity as a node and each relationship as an edge. Each node and edge can have attached attributes: node attributes include entity name, entity type, source document ID, source paragraph position, version number, creation time, update time, etc.; edge attributes include relationship type, confidence level, source evidence, etc.

[0058] The knowledge graph supports incremental updates. When a new document is added to the system, the structured extraction module extracts entities and relations from the new document and adds the new triples to the graph database. For existing nodes, the system performs a fusion process: it calculates the similarity between entity names to determine if the newly extracted entities are the same as existing entities. If the similarity exceeds a threshold, they are merged into the same node, and the attributes are updated. If the newly extracted relations contradict existing relations, the system records the conflict and triggers a manual review process, where experts confirm the correct relations.

[0059] In terms of version management, each map update generates a new version number and records the change history. Users can query any historical version, meeting traceability and auditing needs.

[0060] 6. Session State Management Module The session state management module records the user's interaction history with the system, improving the consistency and efficiency of multi-turn question-and-answer sessions. The system assigns a unique session ID to each user session and records the following information: (1) User query history: Each query statement entered by the user and the query time; (2) Hit nodes: Entity nodes in the search results that have been adopted or clicked by the user and their association strength; (3) Evidence already used: Evidence fragments cited by the system when answering user questions, avoiding repeated citation of the same evidence in multiple rounds of dialogue; (4) Session state summary: A summary generated based on historical interactions, used for context enhancement of subsequent queries.

[0061] When a user submits a new query within the same session, the system concatenates the session state summary with the current query to form an enhanced query, which is then used to retrieve the results again via the hybrid search engine. For example, if a user first asks, "What is the explosion limit of carbon monoxide?", and after the system answers, the user follows up with, "Then what precautions should be taken when preparing it?", the system recognizes the entity "carbon monoxide" in the context and automatically enhances the current query to "What precautions should be taken when preparing carbon monoxide?", thus obtaining more precise search results. Simultaneously, the system records previously retrieved queries and accepted evidence. When a user asks a question highly similar to a historical query, the system directly returns historical results, avoiding resource waste caused by duplicate searches.

[0062] 7. Web-based interactive platform The system interacts with users through a web-based operating platform. This platform provides the following functions: (1) Search interface: The user enters a natural language query, and the system returns a list of search results, highlighting the matched keywords and semantically related fragments; (2) Knowledge graph visualization: The knowledge graph subgraphs related to the search results are displayed in a graphical way. Users can click on the nodes to view detailed information or expand adjacent nodes to explore the connections. (3) Conversation history: Displays the interaction history of the current conversation, supporting backtracking and editing; (4) Document Preview: Clicking the document link in the search results allows you to preview the original document and jump to the paragraph where the evidence is located; (5) Feedback and annotation: Users can rate the search results and manually correct incorrect entity identification or relationship extraction. The correction results will be used for continuous optimization of the model.

[0063] Example 2 Based on Example 1, this embodiment expands the Chinese and English corpus processing module and the structured extraction module, enhancing the processing capability and adaptive extraction capability for multimodal data.

[0064] 1. Multimodal data processing extension Building upon Example 1, this example adds image data processing capabilities to the Chinese and English corpus processing module. For documents containing images, the system calls a locally deployed computer vision model to extract image features and generate image description text. The image feature vectors are stored in a vector database and participate in semantic retrieval along with the text vectors. When a user queries about equipment structure or fault diagnosis, the system can recall relevant images as supporting evidence.

[0065] Meanwhile, for tabular data, the system retains its row and column structure while combining the table headers and cell contents into a natural language description. This table description text is used for retrieval and entity extraction, and the table itself is stored in a structured format, supporting conditional queries.

[0066] 2. Adaptive extraction mechanism The structured extraction module in this embodiment introduces an adaptive extraction mechanism that combines rule bases and machine learning to improve the accuracy of entity and relation extraction.

[0067] The rule base is predefined based on domain expert knowledge, covers common entity types and relationship patterns, is stored in JSON format, and supports dynamic addition and modification.

[0068] When users manually correct the extraction results, the system records the differences before and after the correction and uses them as training samples. The newly accumulated samples are periodically used to incrementally fine-tune the NER model and relation extraction model, allowing the models to gradually adapt to domain-specific expressions. For example, if a user repeatedly corrects the identification of "CO" as "carbon monoxide," the system will update the terminology dictionary to improve subsequent recognition accuracy.

[0069] Example 3 This embodiment expands upon Embodiment 1 by adding cross-document association discovery, proactive recommendation, and lightweight deployment optimization functions to the knowledge graph construction module and session state management module.

[0070] 1. Cross-document association discovery This embodiment's knowledge graph construction module adds a graph-based association discovery mechanism using graph algorithms. The system automatically analyzes potential associations in the graph: Indirect correlation discovery: Through path query, it was found that although "Gas A" and "Equipment B" are not directly related, they are indirectly related through "Process C" (A→Process C→B), which helps technicians discover new process possibilities.

[0071] Frequent pattern mining: Analyze the frequently occurring subgraph structures in the spectrum, such as the "gas-concentration-pressure-dew point" quadruple, to discover the common patterns that may exist between different gases.

[0072] Anomaly detection: Identifies deviations from mainstream patterns, such as "the explosion limit data of a certain gas deviates too much from the standard value", prompting the user that there may be data errors or special operating conditions.

[0073] On the web-based interactive platform, the system displays the discovery results in a graphical format. Users can click on nodes to view the connection paths and trace back to the original documents, intuitively understanding the inherent relationships between knowledge points.

[0074] 2. Proactive Recommendation Mechanism This embodiment's session state management module introduces intent recognition and proactive recommendation functions for multi-turn dialogues. Based on a locally deployed intent classification model, user queries are categorized and intent types are identified, such as "knowledge query," "parameter calculation," "process recommendation," and "equipment selection." Different intents trigger different retrieval strategies and processing flows.

[0075] Based on the knowledge points that users have focused on in their conversation history, the system proactively recommends relevant content that has not yet been retrieved. For example, after a user has repeatedly searched for information related to "carbon monoxide," the system proactively recommends information on other common standard gases such as "carbon dioxide" and "methane," or recommends safety regulations and testing methods related to "carbon monoxide." This proactive recommendation is implemented based on similarity calculations using a knowledge graph.

[0076] 3. Lightweight deployment optimization This embodiment optimizes the system for lightweight deployment to adapt to resource-constrained edge computing environments: Model compression: Knowledge distillation is employed to compress large embedding models into lightweight models, reducing memory usage and inference time while maintaining retrieval accuracy. During distillation, the original model is used as the teacher model to train student models to fit the output of the teacher model.

[0077] Index optimization: Quantization and pruning techniques are used to compress the storage space of the vector index. For example, quantizing a 32-bit floating-point vector into an 8-bit integer reduces the index file size by 75% while maintaining over 95% retrieval accuracy.

[0078] Incremental indexing: For newly added documents, an incremental update method is used to build a vector index, avoiding the computational overhead of full reconstruction. The system maintains a queue of new documents and periodically merges the vectors of new documents into the main index.

[0079] Edge deployment: The system can be deployed on local servers, industrial PCs, or even embedded devices to meet the deployment needs of enterprises of different sizes. During deployment, search parameters can be automatically adjusted based on hardware resources.

[0080] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A fully localized hybrid retrieval and knowledge graph construction method for standard gas R&D and production, characterized in that, include: Data preprocessing: Obtain multi-source heterogeneous documents, and perform text parsing, Chinese-English mixed word segmentation, terminology standardization, and stop word filtering on the multi-source heterogeneous documents to obtain multiple text units; Hybrid retrieval: Upon receiving a user query, the BM25 keyword retrieval algorithm is used to precisely match the text units to obtain a first retrieval result; a locally deployed embedding model is used to convert the user query and the text units into semantic vectors respectively, and semantic retrieval is performed based on vector similarity to obtain a second retrieval result; according to the density of standard gas domain terms involved in the user query, the weighting coefficients of the weighted fusion are dynamically adjusted, and the first retrieval result and the second retrieval result are weighted and fused to obtain a fused retrieval result; Structured extraction: Entities and relations are automatically extracted from the fused retrieval results to generate triplet data. The entities include one or more of the following: gas components, process parameters, safety thresholds, and equipment models. The extraction process is performed based on a combination of a predefined domain entity dictionary and a locally deployed named entity recognition model. Knowledge graph construction: A knowledge graph is constructed based on the triple data. The nodes of the knowledge graph are labeled with source document identifiers, paragraph positions, and version tags, and the edges of the knowledge graph are labeled with relation types and source evidence. The knowledge graph supports historical state tracing based on version tags and knowledge verification based on source evidence. Fully localized deployment: The embedded model, text units, semantic vectors, and knowledge graph are all deployed on local servers. All core data is stored and processed within the enterprise's internal network environment and is not transmitted to any external network servers or third-party cloud platforms, in order to meet the compliance control requirements of standard gas companies for sensitive data such as core formulas and process parameters.

2. The method according to claim 1, characterized in that, In the hybrid retrieval step, the weighted fusion calculation formula is as follows: Where α is the weighting coefficient of BM25 retrieval. The BM25 score for document d is calculated using the following formula: in, For words Frequency of occurrence in document d The length of document d, The average length of all documents is given by k1, and k1 and b are adjustable parameters. The inverse document frequency is N, where N is the total number of documents. For the word q i The number of documents.

3. The method according to claim 1, characterized in that, The formula for calculating semantic similarity is: in, For the semantic vector of the user query, This is the semantic vector of a text unit.

4. The method according to claim 1, characterized in that, The data preprocessing also includes: For multi-source heterogeneous documents containing images, a computer vision model is invoked to extract image features and generate image description text, and the image feature vectors are stored in a vector database; For multi-source heterogeneous documents containing tables, preserve the row and column structure and combine the table header and cell content to generate table description text; For multi-source heterogeneous documents containing formulas, extract the LaTeX representation of the formulas and generate a natural language description of the formulas; The natural language descriptions of the images, tables, and formulas are used in the hybrid retrieval step.

5. The method according to claim 1, characterized in that, The structured extraction employs a strategy that combines rule-based matching with model-based extraction. Entities are extracted by string matching based on a predefined entity dictionary, and entities are extracted by a locally deployed named entity recognition model. The relation extraction is based on dependency parsing and predefined relation templates.

6. The method according to claim 5, characterized in that, It also includes adaptive learning: When a user manually corrects the extraction results, the difference before and after the correction is recorded as a training sample. The named entity recognition model is incrementally fine-tuned periodically using the training sample, and the entity dictionary is updated.

7. The method according to claim 1, characterized in that, The knowledge graph construction also includes: When a new document is added, new triples are obtained by extracting entities and relations from the new document; Calculate the similarity between newly extracted entities and existing entities in the knowledge graph. When the similarity exceeds a preset threshold, merge them into the same node and update the node attributes. When a newly extracted relationship conflicts with an existing relationship, the conflict is recorded and a manual review process is triggered.

8. The method according to claim 1, characterized in that, It also includes session state management: Assign a unique session ID to each user session and record the user's query history, hit nodes, and used evidence; When a new query is received in the same session, the session state summary is concatenated with the current query to form an enhanced query, and the hybrid retrieval step is re-executed based on the enhanced query; When the similarity between a new query and a historical query exceeds a preset threshold, the historical search results are returned directly.

9. The method according to claim 8, characterized in that, It also includes proactive recommendations: The intent of user queries is identified based on a locally deployed intent classification model. Based on the knowledge points that users have followed in the conversation history, the system proactively recommends knowledge content that is related to the followed knowledge points but has not been searched, through node similarity calculation of the knowledge graph.

10. A fully localized hybrid retrieval and knowledge graph construction system for the research and development and production of standard gases, characterized in that, The system for implementing the method as described in any one of claims 1 to 9 includes: A multi-source document pre-embedding module is used to perform the data preprocessing steps; A hybrid search engine is used to perform the hybrid search steps; A structured extraction module is used to perform the structured extraction steps; A knowledge graph construction module is used to execute the knowledge graph construction steps; The session state management module is used to execute the session state management steps; And a web-based interactive platform, used to provide a search interface, knowledge graph visualization, and user feedback annotation functions.

Images