Intelligent question and answer auxiliary method for fault diagnosis
By combining multimodal input with a vector knowledge base, an intelligent question-answering method is used to solve the problems of single information input and low cross-source retrieval efficiency in equipment fault diagnosis. This method enables efficient and accurate fault diagnosis and maintenance guidance, and is suitable for complex field environments.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- CHINESE FLIGHT TEST ESTAB
- Filing Date
- 2025-12-27
- Publication Date
- 2026-05-08
AI Technical Summary
Existing equipment fault diagnosis relies on the experience of maintenance personnel, has limited information input, is difficult to adapt to complex field environments, has low efficiency in cross-source retrieval, and lacks intuitive output results, thus failing to provide effective assistance in handling new types of faults that have not been recorded.
An intelligent question-answering method combining multimodal input processing, vector knowledge base construction, and generative large model reasoning is adopted. It supports text and voice input, builds a vector knowledge base, realizes efficient collection and accurate matching of fault information, and provides intuitive maintenance guidance through multimodal feedback.
It improves the flexibility and accuracy of information collection, enhances retrieval efficiency and accuracy, possesses the reasoning ability for complex problems, optimizes the maintenance interaction experience, and adapts to various complex on-site working conditions.
Smart Images
Figure CN121996676A_ABST
Abstract
Description
Technical Field
[0001] This application belongs to the field of interdisciplinary technology of intelligent diagnosis and artificial intelligence, and in particular relates to an intelligent question-answering assistance method for fault diagnosis. Background Technology
[0002] Currently, equipment fault diagnosis mainly relies on the experience of maintenance personnel and manual data search, which presents the following typical problems: (1) The information input is simple and difficult to adapt to complex on-site environments (such as noisy, obstructed, etc.). (2) Maintenance knowledge is scattered across heterogeneous carriers such as manuals, databases, and historical cases, resulting in low efficiency in cross-source retrieval and difficulty in matching complex patterns; (3) The output results are mostly in text form, which lacks intuitiveness and interactivity, seriously affecting the efficiency and accuracy of maintenance; (4) It lacks reasoning ability for new types of faults that are not recorded and cannot provide effective assistance.
[0003] Current technologies lack a set of intelligent question-answering methods that integrate multimodal input, semantic understanding, cross-source knowledge fusion, and generative reasoning, making it difficult to meet the needs of modern complex equipment for rapid, accurate, and efficient maintenance and support. Summary of the Invention
[0004] The purpose of this invention is to propose an intelligent question-answering assistance method for fault diagnosis. By combining multimodal input processing, vector knowledge base construction, and generative large model reasoning, this method achieves efficient input, accurate matching, and multi-form feedback of fault information, assisting maintenance personnel in quickly completing fault diagnosis and repair work.
[0005] This application provides an intelligent question-answering assistance method for fault diagnosis, the method comprising: Step 1: Construct a multimodal input processing module. The multimodal input processing module supports both text and voice input methods, enabling flexible collection of fault information and providing comprehensive and accurate initial data for subsequent intelligent retrieval and question answering. Step 2: Based on maintenance domain knowledge, construct a vector knowledge base that supports efficient retrieval, providing a solid knowledge foundation for intelligent question answering; Step 3: Construct an intelligent agent decision-making module, and perform user intent recognition, task scheduling, and data retrieval based on the intelligent agent decision-making module; Step 4: Combining semantic retrieval with generative question answering models yields a retrieval-enhanced generative question answering module, enabling multi-round reasoning and knowledge deduction capabilities for complex questions; Step 5: Build a multimodal feedback module. The multimodal feedback module converts the intelligent question-and-answer results into text or voice feedback to adapt to the needs of maintenance scenarios.
[0006] Preferably, step 1 includes: Text input processing: Receives fault description text directly entered by the user; Voice input processing: Integrates automatic speech recognition technology to convert user-spoken fault information into text; Unified conversion of input intent: The fault information input by text or voice is standardized into a structured query statement, which includes core fields such as "equipment type", "fault phenomenon", "fault location" and "operating status". This provides a unified and standardized format for subsequent vector knowledge base retrieval, ensuring the accuracy and efficiency of the retrieval.
[0007] Preferably, step 2 includes: Knowledge Acquisition and Cleaning: Collect equipment maintenance manuals, historical failure cases, manufacturer technical documents, maintenance procedures, and industry standards; perform deduplication on the collected data; standardize the format by converting documents of different formats into a unified text format and performing structured processing on tabular data; and standardize professional terminology to ensure that the same concept has a consistent expression in the knowledge base. Document segmentation and vectorization: The processed document is segmented into segments of 500-1000 characters using the sliding window method, and each segment is converted into a 768-dimensional vector using a pre-trained model; Vector storage and optimization: Vector data is stored in the Chroma vector database and deployed using the ONNX Runtime, supporting GPU batch inference and significantly improving the model's retrieval accuracy and throughput; Knowledge base dynamic updates: A regular update mechanism is set up to automatically incorporate new maintenance cases, technical documents, and industry standards, and the vector model is updated through incremental training.
[0008] Preferably, step 3 includes: Intent recognition engine: Based on a fine-tuned BERT / RoBERTa model, it performs multi-class intent judgment on the input query to identify whether it is a knowledge retrieval, data retrieval or flow control request; Semantic judgment engine: Combines context, entity labels and semantic vector alignment information to deconstruct and map complex semantic structures; Data Request and Knowledge Matching Engine: When the intent is determined to be "data request", the Agent will automatically generate a structured query statement and dynamically schedule equipment operating parameters, maintenance logs, and fault statistics business data; when it is a knowledge request, a query vector will be constructed and a vector knowledge base will be called to perform semantic similarity retrieval. Unified abstract data interface: Resolves differences in semantic mapping and table structure between multiple data sources, and improves the universality of data access.
[0009] Preferably, step 4 includes: Multi-stage retrieval mechanism: First, keyword + semantic hybrid retrieval is performed through Elasticsearch with a confidence score > 0.75. Then, semantic nearest neighbor search is performed through a vector database with a top-k=2 ratio. The dual retrieval results are then merged and re-ranked using embedding, retaining the top-3 best paragraphs. LLM reasoning enhancement: Input the above paragraphs as contextual prompts into the generative large model, perform multiple rounds of question-answering generation, and output structured content covering the dimensions of "cause analysis - maintenance suggestions - precautions"; Unknown fault handling mechanism: When the vector library fails to match a highly similar paragraph, the system falls back to the knowledge generation module inside the large model, autonomously constructs a speculative answer, and adds a "reference suggestion" label to prompt manual review.
[0010] Preferably, step 5 includes: Text feedback: The answers are displayed in a structured card format, highlighting key steps and important information, and supporting the combination of text and images; Voice feedback: Integrates text-to-speech technology to convert text responses into natural speech, supports speech rate adjustment and emphasis, and adapts to the needs of loudspeaker use in noisy environments; Adaptive feedback format: Automatically recommends feedback format based on input method, while also supporting manual switching by the user.
[0011] Preferably, the text input processing further includes: By performing word segmentation, entity recognition, and semantic disambiguation, redundant information is removed, and the core content of the fault description is clarified.
[0012] Preferably, the voice input processing further includes: Considering the noisy environment at the maintenance site, a noise suppression algorithm is used to optimize the recognition accuracy, while also supporting adaptive recognition of dialects and professional terms.
[0013] The beneficial technical effects of this application are as follows: This application enhances the flexibility of information collection, adapting to various complex on-site conditions; achieves high-precision semantic understanding and cross-source knowledge fusion, significantly improving retrieval efficiency and accuracy; possesses the ability to reason about complex problems, and can provide reasonable suggestions even for unknown faults; multimodal feedback optimizes the maintenance interaction experience and reduces operational burden; the system architecture supports edge deployment and has good adaptability and scalability. Attached Figure Description
[0014] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. The drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0015] Figure 1 This is an input processing flowchart of a multimodal input processing module provided in an embodiment of this application; Figure 2 This is a flowchart of a smart agent decision-making module provided in an embodiment of this application; Figure 3 This is a flowchart of a multimodal feedback module provided in an embodiment of this application; Figure 4 This is a flowchart of an intelligent question-and-answer assistance method for fault diagnosis provided in an embodiment of this application. Detailed Implementation
[0016] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0017] The features and illustrative embodiments of various aspects of the present invention will now be described in detail. Numerous specific details are set forth in the following detailed description to provide a thorough understanding of the invention. However, it will be apparent to those skilled in the art that the invention may be practiced without requiring some of these specific details. The following description of embodiments is merely intended to provide a better understanding of the invention by illustrating examples of the invention. The invention is by no means limited to any specific setups and methods set forth below, but covers any improvements, substitutions, and modifications to structures, methods, and devices without departing from the spirit of the invention. Well-known structures and techniques are not shown in the drawings and the following description to avoid unnecessarily obscuring the invention.
[0018] In the description of this invention, it should be noted that the directions or positional relationships indicated by terms such as "center," "upper," "lower," "left," "right," "vertical," "horizontal," "inner," and "outer" are based on the directions or positional relationships shown in the accompanying drawings and are only for the convenience of describing and simplifying the invention, and should not be construed as limiting the invention. Furthermore, the use of ordinal numbers (e.g., "first and second," etc.) is for distinguishing objects and is not limited to this order, and should not be construed as indicating or implying relative importance.
[0019] In the description of this invention, it should be noted that, unless otherwise explicitly specified and limited, the terms "installation," "connection," and "linking" should be interpreted broadly, encompassing both direct connection and indirect connection via an intermediate medium. Those skilled in the art can understand the specific meaning of these terms in this invention based on the specific circumstances.
[0020] It should be noted that, unless otherwise specified, the embodiments of the present invention and the features thereof can be combined with each other, and the various embodiments can be referenced and cited in each other. The present invention will now be described in detail with reference to the accompanying drawings and embodiments.
[0021] The following describes the embodiments and appendices. Figure 1 -Appendix Figure 4 The present invention will be described in further detail, but the embodiments of the present invention are not limited thereto.
[0022] This invention proposes an intelligent question-answering assistance method for fault diagnosis. By combining multimodal input processing, vector knowledge base construction, and generative large-scale model reasoning, it achieves efficient input, accurate matching, and multi-form feedback of fault information, assisting maintenance personnel in quickly completing fault diagnosis and repair tasks. The specific technical solution is as follows: 1. Construction of Multimodal Input Processing Module This module supports both text and voice input methods, enabling flexible collection of fault information and providing comprehensive and accurate initial data for subsequent intelligent retrieval and question-and-answer processes. • Text Input Processing: The system receives user-inputted fault description text (e.g., "engine abnormal noise," "screen not displaying") and performs preprocessing operations such as word segmentation, entity recognition (extracting key information such as equipment model, fault location, and fault characteristics), and semantic disambiguation to remove redundant information and clarify the core content of the fault description. For example, when a user inputs "A certain model of machine tool spindle vibrates violently and is accompanied by abnormal noise when running at high speed," the system can extract key information such as "certain model of machine tool," "spindle," "high-speed operation," "violent vibration," and "abnormal noise."
[0023] • Voice Input Processing: Integrates Automatic Speech Recognition (ASR) technology to convert user-spoken fault information into text. Considering the potentially noisy environment at the repair site, noise suppression algorithms (such as a deep learning-based noise cancellation model) are used to optimize recognition accuracy, while also supporting adaptive recognition of dialects and technical terms. For example, if a repairman in a factory workshop says, "The heating element of this injection molding machine isn't working; the temperature isn't rising," the system can accurately convert the speech into corresponding text information and recognize key terms such as "injection molding machine," "heating element," "not working," and "temperature not rising."
[0024] • Unified Conversion of Input Intent: The fault information input by text or voice is standardized into structured query statements, including core fields such as "equipment type", "fault phenomenon", "fault location" and "operating status", providing a unified and standardized format for subsequent vector knowledge base retrieval, ensuring the accuracy and efficiency of the retrieval.
[0025] 2. Construction of the Vector Knowledge Base (RAG Library) Following the construction method of the RAG library, and with maintenance domain knowledge as the core, a vector knowledge base supporting efficient retrieval is built to provide a solid knowledge foundation for intelligent question answering. • Knowledge Acquisition and Cleaning: Extensive data collection from multiple sources, including equipment maintenance manuals, historical fault cases, manufacturer technical documents, maintenance procedures, and industry standards. The collected data undergoes deduplication to prevent redundant information from occupying storage space; format standardization is implemented by converting documents of different formats such as PDF, Word, and Excel into a unified text format, and tabular data is structured; professional terminology is standardized to ensure that the same concept has a consistent expression in the knowledge base.
[0026] • Document Segmentation and Vectorization: The processed document is segmented into fragments of 500-1000 characters using a sliding window method. This segmentation method can improve the accuracy of subsequent retrieval while preserving the semantic connections between texts. Each fragment is converted into a 768-dimensional vector using a pre-trained model (such as all-mpnet-base-v2). This process can transform the semantic information of the text into a numerical form that computers can understand, thereby realizing semantic-based similarity calculation.
[0027] • Vector Storage and Optimization: Vector data is stored in the Chroma vector database, which has efficient vector storage and retrieval capabilities. Simultaneously, deployment using the ONNX Runtime supports GPU batch inference, significantly improving the model's retrieval accuracy and throughput, ensuring the system maintains a fast response time (≤500ms) even when facing a large number of concurrent queries.
[0028] • Dynamic knowledge base updates: A regular update mechanism is set up to automatically incorporate new maintenance cases, technical documents, and industry standards. By incrementally training and updating the vector model, it is ensured that the knowledge base can reflect the latest maintenance technologies and failure modes in a timely manner, thus guaranteeing the timeliness and practicality of the knowledge.
[0029] 3. Intelligent Agent Decision Module This module is the core of the system, responsible for functions such as user intent recognition, task scheduling, and data retrieval. • Intent recognition engine: Based on a fine-tuned BERT / RoBERTa model, it performs multi-class intent judgment on the input query to identify whether it is a knowledge retrieval, data retrieval or flow control request; • Semantic judgment engine: Combines context, entity labels and semantic vector alignment information to deconstruct and map complex semantic structures; • Data Invocation and Knowledge Matching Engine: When the intent is determined to be "data invocation", the Agent will automatically generate a structured query statement and dynamically schedule business data such as equipment operating parameters, maintenance logs, and fault statistics; when it is a knowledge-based request, a query vector will be constructed and a vector knowledge base will be called to perform semantic similarity retrieval.
[0030] • Unified Abstract Data Interface (UDAI): Resolves differences in semantic mapping and table structure between multiple data sources, improving the universality of data retrieval.
[0031] For example, when a user asks "the failure rate of a certain model of equipment in the past six months", the system automatically identifies the business data query intent, constructs query conditions "equipment model + time window + failure count field", performs aggregation calculation in the background database, and sends it to the large model to generate the final natural language answer.
[0032] 4. Retrieval Enhancement Generative Question Answering Module (RAG Model Fusion) This module combines semantic retrieval with generative question answering models to enable multi-round reasoning and knowledge deduction capabilities for complex questions: • Multi-stage retrieval mechanism: First, keyword + semantic hybrid retrieval is performed through Elasticsearch (confidence > 0.75), then semantic nearest neighbor search is performed through a vector database (top-k=2), and the dual retrieval results are merged for embedding re-ranking, retaining the top-3 best paragraphs; • LLM reasoning enhancement: Input the above knowledge paragraphs as contextual prompts into the generative large model (such as Qwen, Deepseek, etc.), perform multi-round question-answering generation, and output structured content covering dimensions such as "cause analysis - maintenance suggestions - precautions"; • Unknown fault handling mechanism: When the vector library fails to match a highly similar paragraph, the system falls back to the knowledge generation module inside the large model, autonomously constructs a speculative answer, and adds a "reference suggestion" label to prompt manual review.
[0033] 5. Construction of Multimodal Feedback Module Converting intelligent Q&A results into text or voice feedback adapts to maintenance scenarios and improves the convenience for maintenance personnel to obtain information. • Text feedback: The answers are presented in a structured card format, highlighting key steps and important information (such as "Step 1: Turn off the power to the equipment to ensure safety"), and supporting the combination of text and images (embedding equipment disassembly diagrams, component location labels, fault feature images, etc.), so that maintenance personnel can intuitively and clearly understand the maintenance plan.
[0034] • Voice feedback: Integrates text-to-speech (TTS) technology to convert text responses into natural speech, supports adjustable speech rate (default 1.2x speed) and emphasis (such as "Note: When replacing parts, tighten the bolts to the specified torque"), adapts to the external speaker requirements of noisy environments, and allows maintenance personnel to obtain information without looking at the screen during operation.
[0035] • Adaptive feedback format: Automatically recommends feedback format based on input method (voice input defaults to voice feedback, text input defaults to text feedback), while also supporting manual switching by the user to meet the usage habits and scenario needs of different maintenance personnel.
[0036] 6. Intelligent Question Answering Process The overall process architecture describes the system as following a closed-loop chain of "multimodal input → semantic standardization → intent classification → data / knowledge retrieval → generative question answering → multimodal feedback" to achieve a highly robust and scalable intelligent assistance system.
[0037] Response latency: The average system response time is ≤800ms, and ≤1.2s for complex business data aggregation scenarios; Intent recognition accuracy: over 95%; knowledge matching accuracy: top-3 coverage ≥ 90%; The system supports expansion to edge deployment devices such as vehicle terminals and industrial tablets to meet on-site deployment needs.
[0038] Compared with existing technologies, the present invention has the following advantages: Enhance the flexibility of information collection and adapt to various complex on-site conditions; Achieve high-precision semantic understanding and cross-source knowledge fusion, significantly improving retrieval efficiency and accuracy; Possesses the ability to reason about complex problems and can provide reasonable suggestions for unknown faults; Multimodal feedback optimizes the maintenance interaction experience and reduces operational burden; The system architecture supports edge deployment and has good adaptability and scalability.
[0039] This invention integrates multimodal input recognition, vectorized knowledge representation, intelligent agent control mechanism, and generative language model to construct an intelligent question-and-answer assistance system for equipment fault diagnosis. It significantly improves the flexibility of fault information collection, the accuracy of knowledge acquisition, the intelligence of system reasoning, and the convenience of user interaction. It is applicable to multiple high-reliability equipment operation and maintenance scenarios such as aerospace, high-end manufacturing, and intelligent transportation, and has broad engineering application prospects and industrial promotion value.
[0040] The above detailed embodiments are a description of the present invention. It should not be considered that the specific embodiments of the present invention are limited to these descriptions. For those skilled in the art, several simple deductions and substitutions can be made without departing from the concept of the present invention, and all of these should be considered to fall within the protection scope of the present invention.
Claims
1. An intelligent question-answering assistance method for fault diagnosis, characterized in that, The method includes: Step 1: Construct a multimodal input processing module. The multimodal input processing module supports both text and voice input methods, enabling flexible collection of fault information and providing comprehensive and accurate initial data for subsequent intelligent retrieval and question answering. Step 2: Based on maintenance domain knowledge, construct a vector knowledge base that supports efficient retrieval, providing a solid knowledge foundation for intelligent question answering; Step 3: Construct an intelligent agent decision-making module, and perform user intent recognition, task scheduling, and data retrieval based on the intelligent agent decision-making module; Step 4: Combining semantic retrieval with generative question answering models yields a retrieval-enhanced generative question answering module, enabling multi-round reasoning and knowledge deduction capabilities for complex questions; Step 5: Build a multimodal feedback module. The multimodal feedback module converts the intelligent question-and-answer results into text or voice feedback to adapt to the needs of maintenance scenarios.
2. The method according to claim 1, characterized in that, Step 1 includes: Text input processing: Receives fault description text directly entered by the user; Voice input processing: Integrates automatic speech recognition technology to convert user-spoken fault information into text; Unified conversion of input intent: The fault information input by text or voice is standardized into a structured query statement, which includes core fields such as "equipment type", "fault phenomenon", "fault location" and "operating status". This provides a unified and standardized format for subsequent vector knowledge base retrieval, ensuring the accuracy and efficiency of the retrieval.
3. The method according to claim 1, characterized in that, Step 2 includes: Knowledge Acquisition and Cleaning: Collect equipment maintenance manuals, historical failure cases, manufacturer technical documents, maintenance procedures, and industry standards; perform deduplication on the collected data; standardize the format by converting documents of different formats into a unified text format and performing structured processing on tabular data; and standardize professional terminology to ensure that the same concept has a consistent expression in the knowledge base. Document segmentation and vectorization: The processed document is segmented into segments of 500-1000 characters using the sliding window method, and each segment is converted into a 768-dimensional vector using a pre-trained model; Vector storage and optimization: Vector data is stored in the Chroma vector database and deployed using the ONNX Runtime, supporting GPU batch inference and significantly improving the model's retrieval accuracy and throughput; Knowledge base dynamic updates: A regular update mechanism is set up to automatically incorporate new maintenance cases, technical documents, and industry standards, and the vector model is updated through incremental training.
4. The method according to claim 1, characterized in that, Step 3 includes: Intent recognition engine: Based on a fine-tuned BERT / RoBERTa model, it performs multi-class intent judgment on the input query to identify whether it is a knowledge retrieval, data retrieval or flow control request; Semantic judgment engine: Combines context, entity labels and semantic vector alignment information to deconstruct and map complex semantic structures; Data Request and Knowledge Matching Engine: When the intent is determined to be "data request", the Agent will automatically generate a structured query statement and dynamically schedule equipment operating parameters, maintenance logs, and fault statistics business data; when it is a knowledge request, a query vector will be constructed and a vector knowledge base will be called to perform semantic similarity retrieval. Unified abstract data interface: Resolves differences in semantic mapping and table structure between multiple data sources, and improves the universality of data access.
5. The method according to claim 1, characterized in that, Step 4 includes: Multi-stage retrieval mechanism: First, keyword + semantic hybrid retrieval is performed through Elasticsearch with a confidence score > 0.
75. Then, semantic nearest neighbor search (top-k=2) is performed through a vector database. The dual retrieval results are then merged and re-ranked using embedding, retaining the top-3 best paragraphs. LLM reasoning enhancement: Input the above paragraphs as contextual prompts into the generative large model, perform multiple rounds of question-answering generation, and output structured content covering the dimensions of "cause analysis - maintenance suggestions - precautions"; Unknown fault handling mechanism: When the vector library fails to match a highly similar paragraph, the system falls back to the knowledge generation module inside the large model, autonomously constructs a speculative answer, and adds a "reference suggestion" label to prompt manual review.
6. The method according to claim 1, characterized in that, Step 5 includes: Text feedback: The answers are displayed in a structured card format, highlighting key steps and important information, and supporting the combination of text and images; Voice feedback: Integrates text-to-speech technology to convert text responses into natural speech, supports speech rate adjustment and emphasis, and adapts to the needs of loudspeaker use in noisy environments; Adaptive feedback format: Automatically recommends feedback format based on input method, while also supporting manual switching by the user.
7. The method according to claim 2, characterized in that, The text input processing also includes: By performing word segmentation, entity recognition, and semantic disambiguation, redundant information is removed, and the core content of the fault description is clarified.
8. The method according to claim 2, characterized in that, The voice input processing also includes: Considering the noisy environment at the maintenance site, a noise suppression algorithm is used to optimize the recognition accuracy, while also supporting adaptive recognition of dialects and professional terms.