Equipment operation manual vector knowledge base construction method

Abstract

The invention provides an equipment operation manual vector knowledge base construction method. According to the method, heterogeneous multi-modal data such as texts, images, videos and three-dimensional models related to equipment are collected, semantic vector coding is conducted on the multi-modal data through a large pre-training model, the multi-modal data are fused into a unified vector space, and vector representation of equipment knowledge is constructed. An efficient index is established based on vector representation, and intelligent retrieval and association services of a semantic level are supported. And an interpretable vector coding technology is also introduced, so that the semantic interpretability of the vector is endowed, and a visual interface is provided for analyzing a vector distribution mode. And continuous expansion and optimization of the knowledge base are realized through active learning and online learning strategies. According to the method, heterogeneous data integration, intelligent semantic service, interpretability enhancement and knowledge automatic updating are realized, intelligent knowledge service support is provided for the fields of equipment operation and maintenance and the like, and the method has important application value.

Description

Technical Field

[0001] The present invention relates to the technical field of knowledge base construction, and in particular to a method for constructing a device operation manual vector knowledge base. Background Art

[0002] Knowledge base technology refers to the technical means of providing users with efficient knowledge services by organizing, storing and utilizing various knowledge resources in a systematic way. The knowledge base gathers various structured and unstructured knowledge sources, establishes a unified knowledge organization framework, supports rapid retrieval and association of knowledge, and is an important foundation for realizing intelligent decision-making and expert systems.

[0003] At present, knowledge base technology has been widely used in many fields such as enterprise management, e-government, and expert systems. Common forms of knowledge bases include ontology, corpus, and question-answer database. Among them, ontology uses an object-oriented approach to formalize domain knowledge modeling; corpus mainly stores natural language text; and question-answer database provides users with knowledge query services in the form of human-computer interaction. All kinds of knowledge bases are based on relational databases, file systems, or NoSQL technologies for knowledge storage and management.

[0004] However, existing knowledge base technologies have some shortcomings when processing heterogeneous multimodal data such as equipment operation manuals. The traditional knowledge organization method based on file storage makes it difficult to efficiently integrate heterogeneous data such as text, images, videos, and three-dimensional models. The knowledge retrieval efficiency is low, and it is difficult to associate knowledge across files. At the same time, most existing knowledge bases are limited to keyword retrieval, lack semantic-level intelligent retrieval and association capabilities, and have a poor user experience. In addition, existing knowledge bases lack knowledge interpretability analysis and visualization methods, making knowledge difficult to understand and apply. Furthermore, the knowledge in existing knowledge bases is updated slowly, and it is difficult to supplement new knowledge in a timely manner, resulting in outdated and missing knowledge. Therefore, it is urgent to propose a new method for constructing an equipment operation manual knowledge base to overcome the above-mentioned shortcomings of the existing technology. Summary of the invention

[0005] In order to solve the problems of the prior art, an embodiment of the present invention provides a method for constructing a device operation manual vector knowledge base. The technical solution is as follows:

[0006] On the one hand, a method for constructing a device operation manual vector knowledge base is provided, comprising the following steps:

[0007] Collect heterogeneous multimodal data related to the equipment, including text manuals, images, videos, and 3D model data;

[0008] Perform corresponding preprocessing operations on different modal data;

[0009] Use a large pre-trained model to encode the semantic vectors of the pre-processed multimodal data, where text knowledge is encoded using a language model based on a self-attention mechanism, image knowledge is encoded using a visual pre-trained model, video knowledge is encoded using a temporal encoding based on a video transformation model, and 3D model knowledge is encoded using a structure-aware encoding based on a graph convolutional neural network.

[0010] The encoded multimodal vectors are integrated into a unified semantic vector space to construct a vector representation of device knowledge.

[0011] Based on the vector representation, an efficient vector retrieval index is established to support semantic-level knowledge retrieval and association services.

[0012] Furthermore, the preprocessing of text data includes natural language processing operations such as word segmentation, lemmatization and stop word removal.

[0013] Furthermore, the preprocessing of the image data includes computer vision processing operations such as image enhancement and object entity labeling.

[0014] Furthermore, the preprocessing of the video data includes video preprocessing operations such as frame extraction and motion estimation.

[0015] Furthermore, the preprocessing of the three-dimensional model data includes three-dimensional data preprocessing operations such as mesh simplification and normal vector calculation.

[0016] Furthermore, the semantic vector encoding module uses contrastive learning and cross-modal attention fusion techniques to fuse vectors of different modalities into a unified semantic space.

[0017] Furthermore, the step of establishing a vector search index includes:

[0018] Construct a vector index based on a generative vector retrieval library or an approximate nearest neighbor library;

[0019] Design partitioning strategies to support batch data insertion and incremental updates;

[0020] Use optimization techniques such as locality-sensitive hashing to improve vector retrieval performance.

[0021] Furthermore, it also includes:

[0022] Provides natural language-based semantic retrieval capabilities, returning the most relevant knowledge based on the semantic similarity between the query and the knowledge vector;

[0023] Analyze association patterns in vector space, automatically associate related knowledge, and enhance knowledge coherence;

[0024] Combined with knowledge graph representation, it supports compound queries based on semantic and structural constraints.

[0025] Furthermore, it also includes:

[0026] Use techniques such as attention visualization and Word2Vec to make vector representations semantically interpretable;

[0027] Build a visualization interface to intuitively display the distribution and clustering rules of vectors in the semantic space;

[0028] Analyze the manifold structure of vectors in semantic space and explain the association patterns between knowledge.

[0029] Furthermore, it also includes:

[0030] Monitor user query logs, identify blind spots in the knowledge base, and proactively initiate annotation requests to experts;

[0031] Using online learning algorithms, model parameters are updated promptly after receiving new samples;

[0032] Design a super decision-making model, weigh the value and cost of new knowledge, and optimize the incremental update strategy.

[0033] The technical solution provided by the embodiment of the present invention has the following beneficial effects:

[0034] 1. It realizes the unified representation of heterogeneous multimodal data, effectively integrates different knowledge sources such as text, images, videos, and three-dimensional models, and breaks through the limitation that existing technologies cannot efficiently process multi-source heterogeneous data.

[0035] 2. Based on semantic vector representation, it provides intelligent semantic retrieval and association services, improves the efficiency and experience of user knowledge retrieval, and overcomes the deficiency of existing technology that only supports keyword retrieval.

[0036] 3. The interpretable vector encoding technology is introduced to give the vector representation semantic interpretability, and the distribution pattern of the vector in the semantic space is intuitively displayed through a visual interface, which enhances the interpretability and comprehensibility of the knowledge base.

[0037] 4. Active learning and online learning strategies were designed to achieve continuous expansion and optimization of the knowledge base, ensure the freshness and completeness of knowledge, and effectively solve the problem of slow knowledge updating in existing technologies.

[0038] 5. By integrating multimodal data into a unified vector space, an intelligent equipment knowledge service system is constructed, which supports integrated services such as intelligent retrieval, association, explainable analysis and online learning, and significantly improves the utilization efficiency of equipment knowledge.

[0039] 6. This method has strong versatility and extensibility. It can not only be applied to the construction of equipment operation manual knowledge base, but also can be extended to the construction of knowledge base in other fields.

[0040] In general, this invention makes full use of the latest research results of deep learning and knowledge graph technology, has significant innovations and improvements over the existing technology, and can provide intelligent knowledge service support for industries such as equipment operation and maintenance and smart manufacturing, and has important theoretical significance and application value. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0042] Figure 1 It is a flow chart of a method for constructing a device operation manual vector knowledge base according to an embodiment of the present invention. DETAILED DESCRIPTION

[0043] In order to make the objectives, technical solutions and advantages of the present invention more clear, the embodiments of the present invention will be further described in detail below with reference to the accompanying drawings.

[0044] The present invention provides a method for constructing a device operation manual vector knowledge base, see Figure 1 , including the following steps:

[0045] 1. Multimodal data acquisition and preprocessing

[0046] Collect heterogeneous multimodal data related to the equipment, including text manuals, images, videos, and 3D model data;

[0047] Perform corresponding preprocessing operations on different modal data:

[0048] Text data preprocessing:

[0049] Word segmentation: Use word segmentation tools to split text into word sequences

[0050] Lemmatization: Use the lemma library to merge word forms into a unified form

[0051] Stop word removal: remove high-frequency meaningless words from the corpus

[0052] Image data preprocessing:

[0053] Image enhancement: perform enhancement transformations such as rotation, flipping, scaling, etc. to expand image data

[0054] Object entity annotation: Use target detection models to annotate object entities in images

[0055] Video data preprocessing:

[0056] Frame extraction: Sample the video at a fixed frame rate and extract key frames

[0057] Motion estimation: Use optical flow or deep models to estimate the motion displacement between frames

[0058] 3D model data preprocessing:

[0059] Mesh simplification: Perform centroid sampling on overly dense 3D meshes to reduce the amount of data

[0060] Normal vector calculation: Calculate the normal vector of each mesh vertex to capture the local geometric structure

[0061] 2. Multi-source heterogeneous knowledge vector encoding

[0062] Use a large pre-trained model to encode semantic vectors of pre-processed multimodal data:

[0063] Text knowledge encoding:

[0064] Encode text using Transformer encoder models such as BERT

[0065] The input is the word sequence after segmentation, and the output is the corresponding vector representation

[0066] Using self-attention mechanism to capture long-range semantic dependencies

[0067] Image knowledge encoding:

[0068] Use a pre-trained vision model such as ViT to linearly project image pixels

[0069] Use multi-head self-attention module to model local features of the image

[0070] The output is the semantic vector representation of the corresponding global image

[0071] Video Knowledge Encoding:

[0072] Encoding video frame sequences based on the Video Transformer model

[0073] Capturing inter-frame motion and contextual dependencies using temporal self-attention

[0074] The output is a video vector representation that incorporates temporal semantics

[0075] 3D model encoding:

[0076] Designing a graph convolutional network to encode 3D mesh data

[0077] Introducing message passing mechanism to capture structural information between grid nodes

[0078] The output is a vector representation of the overall shape and structure of the 3D model.

[0079] Through contrastive learning, cross-modal attention fusion and other methods, vectors of different modalities are fused into the same semantic space.

[0080] 3. Vector Knowledge Fusion and Indexing

[0081] The encoded multi-modal vectors are fused to establish a device knowledge vector library and build an efficient vector retrieval index:

[0082] Use the generative vector search library PGVector to create an index

[0083] Hash partitions the vector by ID to support partition data persistence

[0084] Using the PQ-based IVFPQ indexing algorithm to improve vector query efficiency

[0085] 4. Composite semantic retrieval and association

[0086] Based on the unified vector space, it provides intelligent semantic retrieval and association functions:

[0087] Semantic search:

[0088] Encoding natural language queries into semantic vectors

[0089] Retrieve the most relevant knowledge vectors to the query based on vector similarity

[0090] Return the original data such as text / image / video corresponding to these knowledge vectors

[0091] Knowledge related:

[0092] Analyze the clustering distribution pattern of knowledge vectors in semantic space

[0093] Identify clusters of vectors with similar distances as related knowledge groups

[0094] Automatically associate other relevant knowledge based on the initial knowledge retrieved

[0095] Compound query:

[0096] Constructing the ontology library and knowledge graph of equipment knowledge

[0097] Link knowledge vectors with entities / relationships in the knowledge graph

[0098] Supports compound queries based on semantic vector similarity and structural constraints

[0099] 5. Interpretable Vector Analysis and Visualization

[0100] Endow vector representations with semantic interpretability and explore the distribution patterns of vectors in semantic space:

[0101] Vector Interpretability:

[0102] Use attention visualization technology to explain the semantics of each attention head

[0103] Based on Word2Vec technology, each dimension of the vector is associated with a specific semantic concept

[0104] Build an interpretable model based on vector decomposition to generate interpretable vector representation

[0105] Visual interface:

[0106] Use dimensionality reduction techniques to map high-dimensional vectors to a two-dimensional plane

[0107] Display the distribution of vectors on the plane based on the visualization framework

[0108] Label vector clustering to highlight the positional features of vectors of different categories in the semantic space

[0109] Correlation pattern analysis:

[0110] Fitting low-dimensional manifolds in vector semantic space to analyze the local neighborhood structure of vectors

[0111] Discover the association patterns and paths between knowledge along the direction of the manifold

[0112] Interpret the associated path as the corresponding semantic transition and explain the cause of the association

[0113] 6. Active learning and online knowledge base update

[0114] Through active learning and online learning strategies, the knowledge base is continuously expanded and optimized:

[0115] Active Learning:

[0116] Cluster user query logs to find query-intensive areas

[0117] Identify blind spots and missing knowledge in the knowledge base in this area

[0118] Actively request domain experts to annotate the missing knowledge

[0119] Online Learning:

[0120] Use online learning algorithms such as Online ELM, ONS, etc.

[0121] When receiving new samples, the model parameters are updated incrementally.

[0122] Ensure that the knowledge base always reflects the latest data distribution

[0123] Incremental update optimization:

[0124] Building a super decision-making model based on reinforcement learning

[0125] The value of the new knowledge and the annotation cost are used as rewards

[0126] Automatically weigh the pros and cons and provide the optimal incremental update strategy

[0127] The above are only preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for constructing a device operation manual vector knowledge base, characterized in that: The following steps are involved: Collect heterogeneous multimodal data related to the equipment, including text manuals, images, videos, and 3D model data; Perform corresponding preprocessing operations on different modal data; Use a large pre-trained model to encode the semantic vectors of the pre-processed multimodal data, where text knowledge is encoded using a language model based on a self-attention mechanism, image knowledge is encoded using a visual pre-trained model, video knowledge is encoded using a temporal encoding based on a video transformation model, and 3D model knowledge is encoded using a structure-aware encoding based on a graph convolutional neural network. The encoded multimodal vectors are integrated into a unified semantic vector space to construct a vector representation of device knowledge. Based on the vector representation, an efficient vector retrieval index is established to support semantic-level knowledge retrieval and association services.

2. The method according to claim 1, characterized in that The preprocessing of text data includes natural language processing operations such as word segmentation, lemmatization, and removal of stop words.

3. The method according to claim 1, characterized in that The preprocessing of image data includes computer vision processing operations such as image enhancement and object entity labeling.

4. The method according to claim 1, characterized in that: The preprocessing of video data includes video preprocessing operations of frame extraction and motion estimation.

5. The method according to claim 1, characterized in that The preprocessing of the 3D model data includes 3D data preprocessing operations of mesh simplification and normal vector calculation.

6. The method according to claim 1, characterized in that The semantic vector encoding module uses contrastive learning and cross-modal attention fusion techniques to fuse vectors of different modalities into a unified semantic space.

7. The method according to claim 1, characterized in that The steps to create a vector retrieval index include: Construct a vector index based on a generative vector retrieval library or an approximate nearest neighbor library; Design partitioning strategies to support batch data insertion and incremental updates; The locality-sensitive hashing optimization technology is used to improve vector retrieval performance.

8. The method according to claim 1, characterized in that Also includes: Provides natural language-based semantic retrieval capabilities, returning the most relevant knowledge based on the semantic similarity between the query and the knowledge vector; Analyze association patterns in vector space, automatically associate related knowledge, and enhance knowledge coherence; Combined with knowledge graph representation, it supports compound queries based on semantic and structural constraints.

9. The method according to claim 1, characterized in that: Also includes: Use attention visualization and Word2Vec technology to make vector representations semantically interpretable; Build a visualization interface to intuitively display the distribution and clustering rules of vectors in the semantic space; Analyze the manifold structure of vectors in semantic space and explain the association patterns between knowledge.

10. The method according to claim 1, characterized in that Also includes: Monitor user query logs, identify blind spots in the knowledge base, and proactively initiate annotation requests to experts; Using online learning algorithms, model parameters are updated promptly after receiving new samples; Design a super decision-making model, weigh the value and cost of new knowledge, and optimize the incremental update strategy.

Images