An engine fault diagnosis method, system and device based on multi-modal data

Abstract

The application discloses an engine fault diagnosis method, system and device based on multi-modal data, relates to the technical field of engine fault diagnosis, and comprises the following steps: through correlation of multi-source data and analysis into evidence items, the evidence table is written. Based on the consistency evaluation result, the running time sequence and maintenance image data are dynamically supplemented and collected, and the evidence table is updated. The time sequence data is windowed and sliced and features are extracted, and is converted into engineering semantic text according to a binning rule table. Evidence conflicts are identified and managed, and according to this, support evidence, conflict evidence and mechanism constraints are combined to infer root cause components, a candidate list with confidence is obtained and sorted. A verification path for each candidate root cause component is generated to guide on-site diagnosis and maintenance operation, and a closed loop is formed. The application converts time sequence data into engineering semantic evidence, fuses conflict management and mechanism constraint inference root cause, outputs a verification path to form a closed loop, enhances diagnosis interpretability, and improves on-site maintenance efficiency.

Description

Technical Field

[0001] This application relates to the field of engine fault diagnosis technology, and in particular to an engine fault diagnosis method, system and equipment based on multimodal data. Background Technology

[0002] Currently, the core process for diagnosing engine faults in the market mainly involves receiving fault description information, obtaining equipment operating information, and then determining fault diagnosis information and solutions through pre-trained models or large language models. Additionally, current data-driven methods input structured information such as work orders, basic engine information, and maintenance reports into the model, outputting frequently occurring fault-causing components and their probabilities.

[0003] The aforementioned existing technologies present numerous problems when applied to engine repair field operations. On-site verbal descriptions are highly conversational, leading to information gaps and biases; directly inputting these into the model results in insufficient evidence credibility and a high risk of misjudgment. Time-series operational data is often coarsely compressed, causing information distortion and making it difficult to support root cause localization. Furthermore, repair process evidence is dynamically incremental, and traditional one-off reasoning cannot form a closed loop of "diagnosis, evidence collection, re-diagnosis, and verification." Summary of the Invention

[0004] To address the aforementioned issues, this application proposes an engine fault diagnosis method based on multimodal data, comprising: acquiring multi-source engine data; associating the multi-source data according to pre-set identifiers; parsing the associated multi-source data into unified evidence items; writing the evidence items into a pre-set evidence table, the evidence table including an evidence master table, an attachment index table, a conflict relationship table, and a binning rule table, the evidence master table being associated with the attachment index table, the conflict relationship table, and the binning rule table respectively; reading a set of evidence items from the evidence master table; performing a consistency evaluation on the set of evidence items to determine the evaluation result; determining a supplementary acquisition instruction based on the evaluation result; acquiring newly added runtime sequence data and maintenance image data according to the supplementary acquisition instruction; and updating the evidence table; and processing the runtime sequence data according to... A pre-set time window is used for slicing. Corresponding feature values ​​are determined based on the signals within the time window. Engineering semantic grading is performed on the feature values ​​according to the binning rule table to obtain feature level and engineering semantic level text. Conflict evidence items are determined based on the evaluation results, and corresponding conflict relationships and causes are identified. The conflict relationships and causes are written into the conflict relationship table. Candidate root causes are determined based on the evidence item set. Root cause inference is performed on the candidate root causes to determine the confidence level of each candidate root cause based on the set of candidate root causes, supporting evidence, conflict evidence, and mechanistic constraint information. The candidate root causes are then sorted according to the confidence level to obtain a candidate root cause list. Verification paths for the candidate root causes are determined, and diagnosis and repair are performed based on these verification paths.

[0005] In one example, the runtime sequence data is sliced ​​according to a pre-set time window, and corresponding feature values ​​are determined based on the signals within the time window. Specifically, this includes: slicing the runtime sequence data according to a pre-set time window, where each time window corresponds to a continuous segment of runtime sequence data; for the signal data within each time window, calculating time-domain features, frequency-domain features, time-frequency-domain features, and trend features based on a pre-set feature set; wherein the time-domain features include mean, fluctuation amplitude, standard deviation, root mean square, and coefficient of variation; the frequency-domain features include dominant frequency, bandwidth energy, and spectral energy percentage; the time-frequency-domain features are obtained through short-time Fourier transform or wavelet transform; and the trend features include slope of change, rate of change, and stability index.

[0006] In one example, the feature value is subjected to engineering semantic classification according to the binning rule table to obtain feature level and engineering semantic level text. Specifically, this includes: determining the corresponding binning rule according to the signal name, engine type and operating conditions corresponding to the feature value; comparing the feature value with the threshold interval in the binning rule to determine the interval in which the feature value is located, and mapping it to obtain the corresponding engineering semantic level text, wherein the engineering semantic level text includes at least one level of extremely low, low, medium, high and extremely high; and writing the feature level, the engineering semantic level text and the corresponding binning rule identifier into the evidence master table.

[0007] In one example, the method further includes: acquiring a maintenance image; performing target region detection, region segmentation, and key part localization on the maintenance image to obtain a target region; inputting the target region and the maintenance image into a pre-set visual language model to output a natural language description and semantic label through the visual language model; writing the generated natural language description as image evidence into the main evidence table, and writing the semantic label into the attachment index table.

[0008] In one example, a consistency assessment is performed on the set of evidence items to determine the assessment result. Specifically, this includes: retrieving multiple types of evidence items corresponding to the same case identifier from the main evidence table; these multiple types of evidence items include fault code evidence, operational characteristic evidence, image evidence, and textual description evidence; determining multiple pre-set consistency relationships; performing consistency analysis based on these multiple consistency relationships to determine a consistency support score; these consistency relationships include temporal consistency, semantic consistency, and mechanistic consistency; and calculating a consistency score using a weighted scoring method. The formula for calculating the consistency score is as follows:

[0009] in, Represents the consistency score. This represents the weight corresponding to the i-th type of evidence. The consistency support score represents the i-th type of evidence.

[0010] In one example, identifying candidate conflicting evidence items and determining the corresponding conflict relationships and causes, and writing the conflict relationships and causes into the conflict relationship table, specifically includes: performing evidence consistency assessment on multiple evidence items corresponding to the same case identifier to determine the consistency score between different evidence items; when the consistency score between different evidence items is lower than a preset threshold, marking the corresponding evidence item as a candidate conflicting evidence item; identifying conflicts in the candidate conflicting evidence items based on evidence type, standardized text, feature level, engineering semantic level text, and engine mechanism constraint rules; and writing the conflict relationships and causes into the conflict relationship table.

[0011] In one example, candidate root causes are determined based on the set of evidence items, and root cause inference is performed on the candidate root causes. Specifically, this includes: reading the set of evidence items corresponding to the case identifier, as well as a preset engine component tree, the correspondence between fault codes and components, the correspondence between fault phenomena and components, and the correspondence between characteristic anomalies and components; determining an initial set of candidate root causes based on fault code evidence, text description evidence, and component tree relationships; matching various types of evidence with candidate root causes to determine the evidence support for each candidate root cause; determining the candidate probability result of each candidate root cause based on the candidate root cause set, supporting evidence corresponding to each candidate root cause, conflicting evidence, and mechanistic constraint information, and using the candidate probability result as the confidence level of the candidate root cause; sorting the candidate root causes according to the confidence level to determine the candidate root cause list, and simultaneously outputting the key supporting evidence and conflict prompts corresponding to each candidate root cause.

[0012] In one example, determining the verification path for the candidate root cause specifically includes: generating a corresponding verification path based on the candidate root cause, wherein the verification path includes the target component, operation action, judgment condition, and type of supplementary evidence; outputting the verification path to the maintenance technician for execution; when new evidence or updated evidence is written back to the main evidence table, determining the re-inference condition to trigger incremental re-inference based on the re-inference condition, and re-executing the steps of root cause inference and verification path generation after triggering; the re-inference condition includes adding a new evidence item, the engineering semantic level text change of the same evidence item exceeding a preset threshold, the conflict state changing, and the consistency score crossing a preset threshold range.

[0013] On the other hand, this application also proposes an engine fault diagnosis system based on multimodal data, including: a data acquisition module for acquiring multi-source data of the engine to be diagnosed, the multi-source data including engine basic data, fault code information, work order information, sales records, maintenance records, running sequence data, and maintenance image data; an evidence table construction module for parsing the acquired multi-source data into unified evidence items and writing them into an evidence master table, the evidence master table serving as a unified intermediate representation in the diagnosis process, including an identifier and association field group, a source and time field group, a content and semantic field group, and a credibility, conflict, and audit field group; the evidence table construction module is also used to write the attachment index information of the original data file into an attachment index table, write the conflict relationship between evidence items into a conflict relationship table, and write the engineering semantic classification rules into a binning rule table; a representation generation module, including a time series representation submodule, for slicing the acquired time series data according to a preset time window and extracting feature values, based on the binning rule table. The binning rule performs engineering semantic grading on the feature values ​​to obtain the engineering semantic level, and then generates standardized text based on the constraint template and writes it into the main evidence table; the diagnostic agent module includes an evidence manager, an evidence consistency evaluator, a collection decision-maker, a root cause inferencer, and a verification path generator; the evidence manager is used to read evidence items from the main evidence table and perform aggregation; the evidence consistency evaluator is used to perform consistency evaluation in combination with the conflict relationships in the conflict relationship table; the collection decision-maker is used to generate supplementary collection instructions when there is insufficient evidence, conflict exists, or insufficient confidence; the root cause inferencer is used to output candidate root causes and confidence levels by combining the evidence content in the main evidence table with the component tree constraint information; the verification path generator is used to generate corresponding investigation steps and verification actions for candidate root causes; the output and interaction module is used to display candidate root causes, confidence levels, evidence consistency results, conflict prompts, and verification paths to maintenance technicians, and supports users to supplement evidence to trigger the diagnostic agent to perform incremental re-inference.

[0014] On the other hand, this application also proposes an engine fault diagnosis device based on multimodal data, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the engine fault diagnosis device based on multimodal data to perform: the method described in any of the examples above.

[0015] On the other hand, this application also proposes a non-volatile computer storage medium storing computer-executable instructions, wherein the computer-executable instructions are configured to be the method described in any of the examples above.

[0016] This application achieves a unified evidentiary representation of multi-source heterogeneous data by constructing an evidence table structure containing a main table and related tables, providing a clear and traceable data foundation for subsequent reasoning. A dynamic data supplementation mechanism based on consistency assessment results can proactively trigger data supplementation when evidence is insufficient or contradictory, effectively avoiding misjudgments due to missing information. Windowing, multi-dimensional feature extraction, and engineering semantic grading based on binning rule tables transform raw numerical signals into hierarchical text with clear engineering meaning, preserving key diagnostic information while enhancing feature interpretability and cross-condition robustness. Explicit recording and management of evidence conflict relationships, combined with mechanistic constraints based on fault code-component tree mapping and physical logic constraints, significantly compresses the candidate root cause search space, making the reasoning process engineering logically interpretable and avoiding the black-box uncertainty of purely data-driven models. The final output of candidate root causes not only includes confidence ranking but also includes executable verification paths, such as target components, operational actions, judgment conditions, and types of supplementary evidence. It also supports incremental evidence-triggered re-reasoning, forming a closed-loop mechanism of "diagnosis-verification-feedback-re-diagnosis". This effectively reduces the risk of incorrect component replacement and blind disassembly and inspection, and improves on-site maintenance efficiency and the credibility of diagnostic conclusions. Attached Figure Description

[0017] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings: Figure 1 This is a flowchart illustrating an engine fault diagnosis method based on multimodal data in an embodiment of this application. Figure 2 This is a schematic diagram of the data structure of the evidence table in an embodiment of this application; Figure 3 This is a schematic diagram of an engine fault diagnosis device based on multimodal data in an embodiment of this application. Detailed Implementation

[0018] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0019] The technical solutions provided by the various embodiments of this application are described in detail below with reference to the accompanying drawings.

[0020] Current engine fault diagnosis methods typically follow a core process of receiving fault description information, acquiring equipment operating information, and then determining fault diagnosis information and solutions through pre-trained models or large language models. In existing technologies, data-driven methods input structured information such as work orders, basic engine information, and maintenance reports into the model, outputting frequently occurring fault-causing components and their probabilities.

[0021] However, the aforementioned existing technologies face numerous challenges when applied to engine repair field operations. Verbal descriptions from the field are highly conversational, often resulting in missing information and biases. Directly inputting these descriptions into the model leads to insufficient evidence credibility and a high risk of misjudgment. Time-series operational data is frequently compressed with coarse granularity, causing information distortion and hindering root cause localization. Existing technologies lack defined engineering representation methods for time-series data before it enters the model, often compressing it into a small number of statistics or unconstrained textual representation. Since engine fault diagnosis relies on dynamic and multi-dimensional feature combinations, coarse-grained summarization leads to the loss of key information, resulting in indistinguishability between similar faults and unstable boundary conditions, leading to unreliable root cause candidate ranking and confidence levels. There is a lack of mechanisms to explicitly map operational features to engineering semantic levels as computable constraints, as well as a lack of candidate domain pruning rules based on engine component tree / fault code mapping. Consequently, under conditions of noise, missing data, batch differences, or measurement errors, the inference output fluctuates significantly and lacks interpretable engineering semantic support. Furthermore, existing technologies do not provide an evidence table structure to record source, confidence level, and conflict fields, nor do they provide consistency indicators or conflict warning outputs. When fault codes contradict operational characteristics and imaging phenomena, the system fails to proactively suggest what additional evidence is needed to resolve the conflict, leading to a risk of misjudgment spilling over into the maintenance process. Evidence during maintenance is dynamically incremental, and traditional one-off reasoning cannot form a closed loop of "diagnosis, evidence collection, re-diagnosis, and verification." While existing technologies can support multi-round dialogue, they lack mechanisms for evidence status maintenance, incremental evidence-triggered re-reasoning, verification path generation, and execution feedback. Output remains at the suggestion level, lacking executable troubleshooting instructions, resulting in limited improvements in maintenance efficiency.

[0022] Therefore, existing technologies cannot solve the technical problem that this application aims to address: to achieve root cause diagnosis and guidance that is "evidence credible, mechanism-constrained, and verifiable closed-loop" at the engine repair site.

[0023] To address the aforementioned issues, this application proposes an engine fault diagnosis method, system, and device based on multimodal data. Its core concept involves: uniformly converting multi-source data into evidence items in an evidence table; generating stable and controllable operational evidence through multi-feature binning of time-series data and constrained D2T; generating traceable image evidence through a vision-language (VL) model; and performing evidence consistency assessment and conflict management before and after inference. When consistency is insufficient or conflicts occur, on-demand data collection and incremental re-inference are triggered. Finally, root cause candidates and verification paths are output, forming a verifiable closed loop between the diagnostic process and maintenance operations.

[0024] In one embodiment, the evidence table is the core data structure for achieving unified evidence representation and traceable diagnosis. It includes the Evidence Master Table, the Evidence Attachment Index Table, the Evidence Conflict Table, and the Dict Binrule Table. The Evidence Master Table is associated with the Evidence Attachment Index Table, the Evidence Conflict Table, and the Dict Binrule Table, respectively.

[0025] The Evidence master table serves as a unified intermediate representation in the diagnostic process, storing primary key identifiers, case identifiers, evidence types, source identifiers, collection time or effective time window, evidence content or standardized text, engineering semantic level, binning rule identifiers, confidence levels, quality identifiers, conflict identifiers, and audit information for each evidence item. Specifically, the Evidence master table is divided into four field groups: The Identification & Association field group records the unique identifier of each evidence item, the identifier of the fault case to which it belongs, the evidence type identifier, and the association key. Specifically, `evidence_id` is the unique identifier for the evidence item; `case_id` is the fault case identifier, used to associate multiple pieces of evidence under the same diagnostic case; `evidence_type` is the evidence type identifier, used to distinguish fault code evidence, operational characteristic evidence, image evidence, textual description evidence, etc.; and `association_key` is the association key, used to achieve cross-evidence, cross-component, cross-work order, or cross-time window association aggregation.

[0026] The Source & Time field group is used to record the source identifier, collection time, and effective time range of the evidence. Specifically, source_ref is the evidence source identifier, indicating the source system, device, or entity; captured_at is the evidence collection time, representing the actual time the evidence was acquired or generated; and effective_time_range is the effective time range, indicating the effective operating window or time interval corresponding to the evidence.

[0027] The Content & Semantic field group records structured evidence content, normalized text content, feature level, engineering semantic level text, and binning rule identifiers. Specifically, `content` stores structured evidence content, used to store structured data or feature results; `text_normalized` stores normalized text content, used to store constraint-based D2T (Data-to-Text) generated text or standardized descriptive text; `feature_level` represents the feature level, used to indicate the engineering semantic level encoding corresponding to the feature; `bin_level_text` represents the engineering semantic level text, used to indicate readable semantic levels such as "low," "medium," "high," and "abnormal"; and `bin_rule_id` is the binning rule identifier, used to identify the rule version used for the current semantic level.

[0028] The Reliability, Conflict & Audit field group records the evidence's confidence level, quality flag, conflict flag, creation time, and update time. Specifically, `confidence` indicates the current level of credibility or support of the evidence; `quality_flag` indicates the evidence's quality status, such as normal, noisy, missing, or low quality; `conflict_flag` indicates whether the evidence conflicts with other evidence; `created_at` records the first time the evidence item was written; and `updated_at` records the most recent update or revision time of the evidence item.

[0029] The main evidence table and the attachment index table have a one-to-many relationship, the main evidence table and the conflict relationship table have a one-to-many relationship, and the main evidence table and the binning rule table have a many-to-one relationship.

[0030] The Evidence_Attachment index table is used to hold the index relationships of large objects such as time-series raw files, photos, and videos. Large volumes of raw data are not directly written to the main table, but are associated with it through an attachment mechanism. The attachment index table includes at least the following fields: attachment_id (unique identifier for the attachment); evidence_id (identifier for the associated evidence item); attachment_type (identifier for the attachment type, indicating whether it's a time-series raw file, image, or video); attachment_uri (attachment storage address, indicating the access path of the attachment file in external storage); hash (attachment digest hash value, used to verify attachment integrity and uniqueness); and captured_at (attachment acquisition time, indicating the time when the corresponding data for the attachment was obtained).

[0031] The Conflict table (Evidence_Conflict) expresses the contradictory relationships between pieces of evidence and the reasons for the conflicts. This highlights the core difference between this application's approach and the general approach of "directly feeding the description and operational data to the model"—this application does not simply perform reasoning, but first manages evidence, then identifies conflicts, and then decides whether to supplement evidence and proceed with further reasoning. The Conflict table includes at least the following fields: conflict_id (unique identifier of the conflict relationship); evidence_id (identifier of the current evidence item, used to indicate the evidence of the conflicting party); conflict_evidence_id (identifier of the conflicting evidence item, used to indicate the other evidence item that conflicts with the current evidence); conflict_flag (conflict status identifier, used to indicate whether the current conflict is valid); conflict_reason (description of the conflict reason, used to indicate the specific reason for the evidence conflict); detected_by (source of conflict detection, used to indicate the rule, model, or module used to detect the conflict); and detected_at (time of conflict detection, used to indicate the time when the conflict was identified).

[0032] The binning rule table `Dict_BinRule` expresses the rule source behind engineering semantic hierarchies. Engineering semantic binning is not an ad-hoc interpretation, but a reproducible mechanism with versions, rules, and unit constraints. The binning rule table includes at least the following fields: `bin_rule_id` is a unique identifier for the binning rule, used to identify the specific binning rule and its version; `signal_name` is the signal name, used to represent the corresponding operating parameter or feature name; `rule_version` is the rule version number, used to represent the binning rule version; `bins_json` is the binning rule content, used to define the threshold range corresponding to each level; and `unit` is the unit identifier, used to represent the engineering unit corresponding to the signal or feature.

[0033] like Figure 2 As shown, the evidence table adopts a structure of "main table + three necessary association tables". The left side of the figure is the Evidence main table, the upper right is the Evidence_Attachment attachment index table, the middle right is the Evidence_Conflict conflict table, and the lower right is the Dict_BinRule binning rule table.

[0034] like Figure 1 As shown, in order to solve the above problems, this application provides an engine fault diagnosis method based on multimodal data, the method comprising: S101. Obtain multi-source data of the engine, associate the multi-source data with a pre-set identifier, parse the associated multi-source data into a unified evidence item, and write the evidence item into a pre-set evidence table. The evidence table includes a main evidence table, an attachment index table, a conflict relationship table, and a binning rule table. The main evidence table is associated with the attachment index table, the conflict relationship table, and the binning rule table, respectively.

[0035] Acquire multi-source data for the engine to be diagnosed. This multi-source data includes at least: basic engine data, such as engine serial number, engine series, manufacturer, production date, manufacturing date, and equipment purpose; fault code information, such as fault code number, fault description, fault cause, and faulty component; work order information; sales records; maintenance records, including maintenance reports; runtime sequence data; and maintenance image data. This multi-source data can be obtained from the OBD interface, vehicle networking platform, maintenance system, or human-machine interface terminal.

[0036] The runtime timing data can be sensor timing signals collected under idling, loading, or specified operating conditions, and may include engine operating data such as speed, torque, temperature, pressure, vibration, and noise. Maintenance image data may include photos or videos taken during the maintenance process.

[0037] After acquiring multi-source data, the multi-source data is associated based on the case identifier (case_id). The associated data is then parsed into unified evidence items and written to the main evidence table (Evidence). Different types of evidence are distinguished by evidence_type, such as fault code evidence, operational characteristic evidence, image evidence, and text description evidence, and cross-evidence association relationships are established using association_key.

[0038] The main evidence table is written as follows: For structured or semi-structured data such as engine basic data, fault code information, work order information, sales records, and maintenance reports, these are parsed into corresponding evidence items and populated with the corresponding fields in the main evidence table. Textual description evidence originates from work orders, maintenance reports, technician input text, voice transcription results, or historical maintenance records. Large-volume raw data files, such as complete time-series data files, photos, and videos, are not directly written to the main Evidence table but are stored in an external storage system, such as a distributed file system or object storage. The corresponding attachment index information is written to the attachment index table Evidence_Attachment, which includes at least attachment_id, evidence_id, attachment_type, attachment_uri, hash, and captured_at.

[0039] When there are contradictions between different pieces of evidence at the temporal, semantic, or physical constraint levels, the conflict relationship is written into the conflict relationship table Evidence_Conflict. For the engineering semantic hierarchical rules of operational features, a binning rule table Dict_BinRule is used for management to achieve versioned management and traceable reproducibility of engineering semantic rules.

[0040] By using the above method, multi-source heterogeneous data is transformed into unified evidence items and written into the evidence table, laying a data foundation for subsequent consistency assessment, conflict detection, and root cause reasoning.

[0041] S102. Read the set of evidence items from the main evidence table, perform a consistency assessment on the set of evidence items to determine the assessment result, and determine a supplementary collection instruction based on the assessment result to obtain the newly added runtime sequence data and maintenance image data according to the supplementary collection instruction, and update the evidence table.

[0042] First, the set of evidence items corresponding to the fault diagnosis case with the current case_id is read from the Evidence master table. Then, the evidence consistency assessment result corresponding to the current set of evidence items is obtained or retrieved, and combined with missing item checks or candidate confidence analysis, an on-demand data collection decision is generated. The assessment result may include consistency score, missing evidence type, conflicting evidence pair, and other information. Specifically, a consistency assessment is performed on the set of evidence items to determine the assessment result. Multiple types of evidence items corresponding to the same case identifier (case_id) are read from the evidence master table. These multiple types of evidence items include fault code evidence, operational characteristic evidence, image evidence, and textual description evidence. Then, a consistency analysis is performed based on at least one of the following relationships: Time consistency analysis determines whether there is time overlap, proximity, or logical sequence relationship between the captured_at or effective_time_range of different evidence items. If the difference between the time windows corresponding to different pieces of evidence exceeds a preset threshold, the consistency score is reduced. For example, if the difference between the fault code acquisition time and the operational feature acquisition time exceeds a preset time threshold, such as 10 seconds, the time consistency support score is reduced.

[0043] Semantic consistency analysis determines whether different pieces of evidence point to the same fault phenomenon, the same component, or the same fault mode. For example, if fault code evidence points to "fire" and operational characteristic evidence also shows abnormal speed fluctuations, the consistency score is increased; if video evidence shows a loose connector, but there is no corresponding abnormality in the operational characteristics, the consistency score is decreased.

[0044] Mechanism consistency analysis, based on the engine component tree, fault code mapping relationships, or physical constraint rules, determines whether different pieces of evidence satisfy the engine mechanism logic. For example, if abnormal oil pressure and lubrication system-related faults have a mechanistic correlation, the consistency score is increased; if there is a combination of evidence that is physically difficult to establish simultaneously, such as the simultaneous presence of evidence of "extremely high oil pressure" and "serious leakage in the lubrication system," a conflict is determined.

[0045] The consistency score can be calculated using a weighted scoring method:

[0046] in, Represents the consistency score. This represents the weight corresponding to the i-th type of evidence. This represents the consistency support score for the i-th type of evidence. The consistency support score can be quantified based on whether it is supportive, weakly supportive, neutral, weakly conflicting, or strongly conflicting. For example, supportive can be quantified as +1, weakly supportive as +0.5, neutral as 0, weakly conflicting as -0.5, and strongly conflicting as -1.

[0047] Then, based on the evaluation results, including consistency score, missing evidence, conflict indicators, and candidate root cause confidence levels, it is determined whether supplementary data collection is needed. Specifically, a supplementary data collection instruction is generated when any of the following conditions are met: The evidence consistency score is below a preset threshold. That is, if the consistency_score is lower than the preset consistency score threshold, such as 0.6, it indicates that there are contradictions or insufficient support among the current evidence, and additional evidence is needed to resolve the contradictions.

[0048] Key evidence types are missing. Key evidence types can be determined by a pre-defined key evidence rule base. The system pre-establishes a "fault mode - key evidence type" mapping rule base, which defines the types of evidence that must be present or prioritized for collection under different fault scenarios. For example, when the fault code points to a misfire fault, key evidence types include at least: idling operating characteristics, cylinder pressure, ignition coil voltage, etc. If the above key evidence types are missing from the current evidence set, supplementary collection is triggered.

[0049] The conflict flag `conflict_flag=1` indicates that there are evidence items marked as conflicting in the main evidence table. This suggests that there are irreconcilable contradictions among the current evidence, and additional evidence needs to be collected to resolve the conflict.

[0050] The confidence level of a candidate root cause is below a threshold. The confidence level of a candidate root cause essentially originates from the candidate probability results output by the diagnostic agent. The diagnostic agent is a large language model or multimodal fusion model based on the Transformer architecture. During inference, it outputs corresponding probability values ​​for different candidate root causes, and these probability values ​​can be used as the confidence level of the candidate root cause. In one implementation, the confidence level can be derived from the probability value of the model output layer logits after Softmax normalization; in another implementation, it can also be derived from the candidate probability scores corresponding to the results generated by a ranking model, classification model, or large language model. When the probability corresponding to the largest candidate root cause is below a preset threshold, such as 0.5, it can be determined that the confidence level of the current diagnostic result is insufficient, thereby triggering a supplementary data collection and incremental re-inference process.

[0051] When data collection is required, the engine is controlled to enter a preset operating condition, such as idling or loading, and corresponding runtime sequence data or maintenance image data is collected. The raw collected data is stored in an external storage system, and the raw collected data includes runtime sequence data or maintenance image data, with corresponding attachment index information written to Evidence_Attachment.

[0052] S103. The runtime sequence data is sliced ​​according to a pre-set time window, the corresponding feature value is determined according to the signal within the time window, and the feature value is subjected to engineering semantic classification according to the binning rule table to obtain feature level and engineering semantic level text.

[0053] The acquired runtime sequence data is transformed into computable evidence with engineering semantics, including window slicing of time-series data, multi-feature extraction, and engineering semantic grading based on binning rules. Specifically, the runtime sequence data is sliced ​​according to pre-set time windows. First, the acquired runtime sequence data is sliced ​​according to preset time windows, with each time window corresponding to a segment of continuous runtime data. The window strategy can be a fixed time window, such as a window every 120 seconds; or an event-triggered window, such as one triggered by crankshaft rotation or operating condition switching events, but the core mechanisms of "multi-feature, binning, constraint text, and writing to the evidence table" should be retained. Subsequently, feature calculations are performed on the signal data within each time window. Specifically, the system determines the feature items to be calculated according to the preset feature_set_id, and performs feature calculations on the corresponding signals within each time window, calculating various features such as time-domain features, frequency-domain features, time-frequency-domain features, and trend features for the data within the window. Among them, time-domain features are used to characterize the statistical properties of a signal in the time dimension, and may include representative features such as mean, amplitude, standard deviation, root mean square (RMS), and coefficient of variation. Frequency-domain features are used to characterize the energy distribution and periodicity of a signal in the frequency dimension, and may include representative features such as dominant frequency, band energy, and spectral energy ratio. Time-frequency domain features are used to characterize the frequency characteristics of non-stationary signals over time, and can be obtained through short-time Fourier transform (STFT), wavelet transform, or other time-frequency analysis methods. Trend features are used to characterize the trend of a signal over time, and may include representative features such as slope, rate of change, and stability index. It should be noted that there are many feature indicators that can be extracted from time-series data. Different engine types, different fault modes, different sampled signals, and different diagnostic scenarios may require different feature indicators, and this application does not limit the specific types of feature indicators. Those skilled in the art can flexibly select the corresponding feature set for calculation based on actual diagnostic needs, target fault type, operating conditions, or model input requirements.

[0054] After feature calculation, the system associates the feature values ​​with corresponding time windows and signal names, and uses this association for subsequent engineering semantic classification and evidence generation. Then, it performs engineering semantic classification on the feature values ​​according to the binning rules in the Dict_BinRule table. The system first queries the Dict_BinRule table for the corresponding bin_rule_id and the corresponding binning rule based on the signal name, engine type, and operating conditions corresponding to the current feature. The binning rules define the mapping relationship between different numerical ranges and engineering semantic levels. The binning rule table uses fields such as bin_rule_id, signal_name, rule_version, bins_json, and unit to achieve version management and traceable reproducibility of engineering semantic rules. Subsequently, the system compares the currently calculated feature value with the threshold range in the binning rules to determine the range in which the feature value belongs and maps it to the corresponding engineering semantic level text bin_level_text. For example, when a feature value is in the first threshold range, it can be mapped to "low"; when it is in the second threshold range, it can be mapped to "medium"; and when it is in the third threshold range, it can be mapped to "high".

[0055] In one embodiment, `bin_level_text` represents the engineering semantic level text, which is essentially a discrete semantic representation of continuous numerical features. `bin_level_text` can adopt the following level system: extremely low, low, medium, high, and extremely high. This embodiment does not limit the specific number of levels or level names, and can be configured according to engine type, fault scenario, operating parameter type, and actual engineering needs. The number of levels can be three, five, or more; the threshold can be configured based on calibration, statistics, or regulatory sources, but `bin_rule_id` / version management should be retained to ensure traceability and reproducibility. After mapping, the system writes the feature level code `feature_level`, the engineering semantic level text `bin_level_text`, and the corresponding `bin_rule_id` into the Evidence main table for subsequent consistency evaluation, conflict detection, and root cause reasoning.

[0056] In the above manner, continuous numerical operational characteristics can be transformed into discrete semantic evidence that has engineering implications, is interpretable, and can be used for reasoning.

[0057] In one embodiment, after completing the engineering semantic classification, this embodiment further includes the step of generating operational evidence text using constraint-based D2T and writing it into the evidence table. Specifically, based on a preset template, fields such as feature name, unit, level, time window, confidence level, and bin_rule_id are generated into normalized text text_normalized. For example, normalized text in the following format can be generated: "Idle window (120s, TW-IDLE-001): Average RPM [Medium] (BIN-RPM-MEAN-v2); Average Coolant Temperature [High] (BIN-ECT-MEAN-v1); Average Oil Pressure [Low] (BIN-OILP-MEAN-v1); All generated according to binning rules, units are normalized." Then, the constraint text is written into the Evidence table, where the evidence type evidence_type is marked as operational feature evidence, and the corresponding original time-series data index is written into Evidence_Attachment.

[0058] Constrained D2T can be implemented using rule templates or a dedicated text generation model, but it must be subject to field and format constraints. The output should include at least "features, units, levels, windows, confidence, and rule IDs" to ensure the stability and reproducibility of the generated text.

[0059] S104. Based on the evaluation results, determine conflicting evidence items and corresponding conflict relationships and causes. Write the conflict relationships and causes into the conflict relationship table. Based on the set of evidence items, determine candidate root causes. Perform root cause inference on the candidate root causes. Determine the confidence level of the candidate root causes based on the set of candidate root causes, supporting evidence, conflicting evidence, and mechanistic constraint information. Sort the candidate root causes according to the confidence level to obtain a candidate root cause list. Determine the verification path of the candidate root causes. Perform diagnosis and repair according to the verification path.

[0060] In one embodiment, conflict identification and management are performed. First, candidate conflict evidence items are identified, and the corresponding conflict relationships and causes are determined. The conflict relationships and causes are then written into a conflict relationship table.

[0061] The system first performs an evidence consistency assessment on multiple evidence items corresponding to the same case_id, calculating the consistency score or consistency level between different evidence items. In one possible implementation, during the consistency assessment, the system combines the temporal relationship, semantic relationship, and mechanistic constraint relationship between evidence items to determine whether different pieces of evidence support each other or are contradictory. For example, if the fault code evidence points to a misfire, and the operational characteristic evidence shows abnormal speed fluctuations, then the consistency between the corresponding evidence is high; if the fault code evidence points to a lubrication system abnormality, while the operational characteristic evidence shows that the oil pressure is always normal, then the consistency between the corresponding evidence is low.

[0062] When the consistency between different evidence items falls below a preset threshold, the system marks the corresponding evidence item as a candidate conflicting evidence item. Subsequently, it further combines evidence_type, text_normalized, feature_level, bin_level_text, and engine mechanism constraint rules to identify conflicts among the candidate conflicting evidence items. Finally, the conflict relationship conflict_with and the conflict reason conflict_reason are written into Evidence_Conflict.

[0063] In one embodiment, root cause inference is performed. Root cause inference is performed under mechanistic constraints. Specifically, the system first reads the set of evidence items corresponding to the current case_id, including fault code evidence, operational feature evidence, image evidence, and textual description evidence. It also reads the preset engine component tree, i.e., the component hierarchy and assembly relationships, the correspondence between fault codes and components, the correspondence between fault phenomena and components, and the correspondence between abnormal features and components. Subsequently, the system determines an initial set of candidate root causes based on the fault code evidence, textual description evidence, and component tree relationships. This initial set of candidate root causes is not an unconstrained search across all engine components, but rather generated within the scope of components that are related to the current fault phenomenon, fault code, or abnormal feature, thereby reducing interference from irrelevant components. The mechanistic constraints include at least those based on fault code-component tree mapping relationships, operational feature constraints, or physical-logical constraints. Based on these constraints, candidate domain pruning and evidence support ranking are performed.

[0064] After obtaining the initial set of candidate root causes, the system matches various types of evidence with the candidate root causes and calculates the evidence support for each candidate root cause. If a fault code, abnormal feature, image description, or text description of a certain evidence item has a preset correlation with a candidate root cause, then that evidence item is used as supporting evidence for that candidate root cause; if a certain evidence item has no clear correlation with a candidate root cause, then that evidence item is not used as primary supporting evidence; if a certain evidence item contradicts the mechanism corresponding to a candidate root cause, or has been marked as conflicting evidence in the Evidence_Conflict table, then the support strength of that candidate root cause is reduced.

[0065] The aforementioned mechanistic constraints are primarily used to limit the reasonable range of candidate root causes and to adjust the strength of supporting evidence. These constraints may include the correspondence between fault codes and components, the correspondence between fault phenomena and components, the correspondence between abnormal operating characteristics and components, upstream and downstream relationships of components, and the basic physical and logical relationships during engine operation. After evidence matching is completed, the diagnostic agent takes the set of candidate root causes, supporting evidence for each candidate root cause, conflicting evidence, and mechanistic constraint information as input, and outputs the candidate probability results for each candidate root cause. These candidate probability results serve as the confidence level of the candidate root cause. The diagnostic agent is a large language model or multimodal fusion model based on the Transformer architecture, which outputs corresponding probability values ​​for different candidate root causes during inference. In one implementation, the confidence level may be derived from the probability value of the model output layer logits after Softmax normalization; in another implementation, it may also be derived from the candidate probability scores corresponding to the results generated by a ranking model, classification model, or large language model. Finally, the system sorts the candidate root causes according to their probability or confidence level, outputs a list of candidate root causes, and simultaneously outputs the key supporting evidence and necessary conflict warnings for each candidate root cause.

[0066] Through the above process, the root cause inference does not generate all possible components in an open manner, but sorts and outputs them within a candidate range jointly defined by fault codes, operating characteristics, image descriptions, text descriptions, and engine mechanism constraints, thereby improving the stability, interpretability, and verifiability of the inference results.

[0067] In one embodiment, verification path generation and incremental closure are performed. Determining the verification path for candidate root causes specifically includes: generating a corresponding verification path based on the candidate root causes. The verification path includes the target component, operational action, judgment conditions, and the type of supplementary evidence required. The verification path is an executable instruction containing specific troubleshooting steps and judgment conditions, rather than merely providing conclusive suggestions. The verification path is output to the maintenance technician for execution. The maintenance technician performs diagnostic and maintenance operations according to the verification path, and can supplement text descriptions, upload maintenance images, perform data collection under specified operating conditions, and return new evidence. When new or updated evidence is written back to the main evidence table, incremental re-inference is triggered when any of the following conditions are met, i.e., the steps of root cause inference and verification path generation are re-executed: New evidence items. This means that the repair technician has added new evidence, such as new video evidence, textual description evidence, or runtime sequence evidence, and added it to the main evidence table.

[0068] The engineering semantic level text of the same evidence item changes beyond a threshold. Specifically, in R2, "the bin_level_text of the same evidence item changes beyond a threshold" means that the engineering semantic level corresponding to the same evidence_id changes beyond a preset range after adding new data, recalculating feature values, or re-executing engineering semantic classification. First, the corresponding feature values ​​are recalculated based on the new runtime sequence data. Then, the engineering semantic classification is re-executed based on the bin_rule_id in Dict_BinRule to obtain a new bin_level_text. The new level result is then compared with the level result corresponding to the previous version of the evidence_id. bin_level_text corresponds to the discrete level code feature_level, for example: very low: 1; low: 2; medium: 3; high: 4; very high: 5. At this time, the system determines whether the threshold is exceeded by comparing the difference between the old and new feature_levels. Fixed level difference thresholds, for example, triggering re-inference when the level change is ≥1 or ≥2; different thresholds for different features, for example, different level change thresholds can be used for oil pressure, coolant temperature, and vibration features; thresholds are determined based on historical statistical results, for example, the level change sensitivity corresponding to different signals is determined based on historical normal fluctuation range, calibration data, or empirical rules.

[0069] The conflict status has changed. That is, the conflict status (conflict_flag) in the conflict relation table has changed, such as a new conflict being added or an existing conflict being resolved.

[0070] The consistency score crosses a preset threshold range. That is, if the consistency score, after adding new evidence, jumps from a high range to a low range, or from a low range to a high range, it indicates that the overall consistency of the evidence has changed significantly.

[0071] After being triggered, the steps of root cause inference and verification path generation are re-executed, thereby enabling the diagnostic conclusion to gradually converge with incremental evidence. After new evidence is written back to the Evidence main table, the diagnostic agent can be triggered to perform incremental re-inference, thereby achieving closed-loop update of the diagnosis.

[0072] In one embodiment, this application also provides an engine fault diagnosis system based on multimodal data, including: The data acquisition module is used to acquire multi-source data from the engine to be diagnosed. This multi-source data includes engine basic data, fault code information, work order information, sales records, maintenance records, runtime sequence data, and maintenance image data. The data acquisition module can acquire this multi-source data from the OBD interface, vehicle networking platform, maintenance system, or human-machine interface terminal. The runtime sequence data can be sensor timing signals collected under idling, loading, or specified operating conditions, and the maintenance image data can include photos or videos.

[0073] The evidence table construction module is used to parse the collected multi-source data into unified evidence items and write them into the main evidence table, Evidence. This main evidence table serves as a unified intermediate representation in the diagnostic process and includes identification and association field groups, source and time field groups, content and semantic field groups, and confidence, conflict, and audit field groups. Specifically, the identification and association field group includes evidence_id, case_id, evidence_type, and association_key; the source and time field group includes source_ref, captured_at, and effective_time_range; the content and semantic field group includes content, text_normalized, feature_level, bin_level_text, and bin_rule_id; and the confidence, conflict, and audit field group includes confidence, quality_flag, conflict_flag, created_at, and updated_at.

[0074] The evidence table construction module is also used to write the attachment index information of the original data file into the attachment index table `Evidence_Attachment`, which includes at least `attachment_id`, `evidence_id`, `attachment_type`, `attachment_uri`, `hash`, and `captured_at`. The evidence table construction module is also used to write the conflict relationships between evidence items into the conflict relationship table `Evidence_Conflict`, which includes at least `conflict_id`, `evidence_id`, `conflict_evidence_id`, `conflict_flag`, `conflict_reason`, `detected_by`, and `detected_at`. The evidence table construction module is also used to write the engineering semantic classification rules into the binning rule table `Dict_BinRule`, which includes at least `bin_rule_id`, `signal_name`, `rule_version`, `bins_json`, and `unit`.

[0075] The representation generation module includes a temporal representation submodule and an image representation submodule (optional). The temporal representation submodule slices the acquired temporal data into preset time windows and extracts feature values. Based on the binning rules in the Dict_BinRule table, it performs engineering semantic grading on the feature values ​​to obtain the engineering semantic level. Then, it generates normalized text (text_normalized) based on a constraint template and writes it to the Evidence main table. The image representation submodule (optional) performs visual preprocessing on maintenance photos or videos, including target area detection, region segmentation, or key component localization. It inputs the processed target area into a VL visual language model to generate text descriptions and semantic labels. The generated description text is written as image evidence to the Evidence table, while the original image index information is written to Evidence_Attachment.

[0076] The diagnostic agent module includes an evidence manager, an evidence consistency evaluator, a collection decision-maker, a root cause inferencer, and a verification path generator. The evidence manager reads evidence items from the main Evidence table and performs aggregation; the evidence consistency evaluator performs consistency assessments based on conflict relationships in Evidence_Conflict; the collection decision-maker generates supplementary collection instructions when evidence is insufficient, conflicts exist, or confidence levels are insufficient; the root cause inferencer combines the evidence content and component tree constraint information in Evidence to output candidate root causes and their confidence levels; and the verification path generator generates corresponding investigation steps and verification actions for each candidate root cause.

[0077] The output and interaction module displays candidate root causes, confidence levels, evidence consistency results, conflict alerts, and verification paths to maintenance technicians. It also supports users in adding text descriptions, uploading maintenance images, performing data collection under specified operating conditions, and transmitting new evidence. After new evidence is written back to the Evidence main table, it triggers the diagnostic agent to perform incremental re-inference, thereby achieving a closed-loop update of the diagnosis.

[0078] like Figure 3 As shown in the illustration, this application also provides an engine fault diagnosis device based on multimodal data, comprising: At least one processor; and, A memory that is communicatively connected to at least one processor; wherein, The memory stores instructions that can be executed by at least one processor to enable an engine fault diagnosis device based on multimodal data to perform the method as described in any of the embodiments above.

[0079] This application also provides a non-volatile computer storage medium storing computer-executable instructions, which are configured as described in any of the above embodiments.

[0080] In the 1990s, improvements to a technology could be clearly distinguished as either hardware improvements (e.g., improvements to the circuit structure of diodes, transistors, switches, etc.) or software improvements (improvements to the methodology). However, with technological advancements, many methodological improvements today can be considered direct improvements to the hardware circuit structure. Designers almost always obtain the corresponding hardware circuit structure by programming the improved methodology into the hardware circuit. Therefore, it cannot be said that a methodological improvement cannot be implemented using hardware physical modules. For example, a Programmable Logic Device (PLD) (such as a Field Programmable Gate Array (FPGA)) is such an integrated circuit whose logic function is determined by the user programming the device. Designers can program and "integrate" a digital system onto a PLD themselves, without needing chip manufacturers to design and manufacture dedicated integrated circuit chips. Furthermore, nowadays, instead of manually manufacturing integrated circuit chips, this programming is mostly implemented using "logic compiler" software. Similar to the software compiler used in program development, the original code before compilation must also be written in a specific programming language, called a Hardware Description Language (HDL). There are many HDLs, such as ABEL (Advanced Boolean Expression Language), AHDL (Altera Hardware Description Language), Confluence, CUPL (Cornell University Programming Language), HDCal, JHDL (Java Hardware Description Language), Lava, Lola, MyHDL, PALASM, and RHDL (Ruby Hardware Description Language). Currently, the most commonly used are VHDL (Very-High-Speed ​​Integrated Circuit Hardware Description Language) and Verilog. Those skilled in the art should also understand that by simply performing some logic programming on the method flow using one of these hardware description languages ​​and programming it into an integrated circuit, the hardware circuit implementing the logical method flow can be easily obtained.

[0081] The controller can be implemented in any suitable manner. For example, it can take the form of a microprocessor or processor and a computer-readable medium storing computer-readable program code (e.g., software or firmware) executable by the (micro)processor, logic gates, switches, application-specific integrated circuits (ASICs), programmable logic controllers, and embedded microcontrollers. Examples of controllers include, but are not limited to, the following microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20, and Silicon Labs C8051F320. A memory controller can also be implemented as part of the control logic of the memory. Those skilled in the art will also recognize that, in addition to implementing the controller in purely computer-readable program code form, the same functionality can be achieved by logically programming the method steps to make the controller take the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers. Therefore, such a controller can be considered a hardware component, and the means included therein for implementing various functions can also be considered as structures within the hardware component. Alternatively, the means for implementing various functions can be considered as both software modules implementing the method and structures within the hardware component.

[0082] The systems, devices, modules, or units described in the above embodiments can be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, a computer can be, for example, a personal computer, laptop computer, cellular phone, camera phone, smartphone, personal digital assistant, media player, navigation device, email device, game console, tablet computer, wearable device, or any combination of these devices.

[0083] For ease of description, the above devices are described in terms of function, divided into various units. Of course, in implementing this specification, the functions of each unit can be implemented in one or more software and / or hardware components.

[0084] The various embodiments in this application are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the device and medium embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the description of the method embodiments.

[0085] The devices and media provided in this application are one-to-one with the methods. Therefore, the devices and media also have similar beneficial technical effects as their corresponding methods. Since the beneficial technical effects of the methods have been described in detail above, the beneficial technical effects of the devices and media will not be repeated here.

[0086] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0087] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0088] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0089] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0090] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0091] Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0092] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0093] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0094] The above are merely embodiments of this application and are not intended to limit the scope of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of the claims of this application.

Claims

1. An engine fault diagnosis method based on multimodal data, characterized in that, include: The engine acquires multi-source data, associates the multi-source data with a pre-set identifier, parses the associated multi-source data into a unified evidence item, and writes the evidence item into a pre-set evidence table. The evidence table includes an evidence master table, an attachment index table, a conflict relationship table, and a binning rule table. The evidence master table is associated with the attachment index table, the conflict relationship table, and the binning rule table, respectively. The evidence item set is read from the main evidence table, and a consistency assessment is performed on the evidence item set to determine the assessment result. A supplementary collection instruction is determined based on the assessment result to obtain newly added runtime sequence data and maintenance image data according to the supplementary collection instruction, and the evidence table is updated. The runtime sequence data is sliced ​​according to a pre-set time window, and the corresponding feature value is determined according to the signal within the time window. The feature value is then subjected to engineering semantic classification according to the binning rule table to obtain feature level and engineering semantic level text. Based on the evaluation results, conflicting evidence items are determined, and corresponding conflict relationships and causes are identified. The conflict relationships and causes are written into the conflict relationship table. Candidate root causes are determined based on the set of evidence items. Root cause inference is performed on the candidate root causes. The confidence level of the candidate root causes is determined based on the set of candidate root causes, supporting evidence, conflicting evidence, and mechanistic constraint information. The candidate root causes are sorted according to the confidence level to obtain a candidate root cause list. The verification path of the candidate root causes is determined, and diagnosis and repair are performed based on the verification path.

2. The method according to claim 1, characterized in that, The runtime sequence data is sliced ​​according to a pre-set time window, and corresponding feature values ​​are determined based on the signals within the time window, specifically including: The runtime sequence data is sliced ​​according to a pre-set time window, wherein each time window corresponds to a continuous segment of runtime sequence data; For the signal data within each time window, time-domain features, frequency-domain features, time-frequency-domain features, and trend features are calculated based on a pre-set feature set. The time-domain features include mean, fluctuation amplitude, standard deviation, root mean square, and coefficient of variation; the frequency-domain features include dominant frequency, frequency band energy, and spectral energy ratio; the time-frequency domain features are obtained through short-time Fourier transform or wavelet transform; and the trend features include slope of change, rate of change, and stability index.

3. The method according to claim 1, characterized in that, According to the binning rule table, the feature values ​​are subjected to engineering semantic classification to obtain feature level and engineering semantic level text, specifically including: The corresponding compartmentation rules are determined based on the signal name, engine type, and operating conditions corresponding to the feature values. The feature value is compared with the threshold range in the binning rule to determine the range in which the feature value is located, and the corresponding engineering semantic level text is mapped to obtain the corresponding engineering semantic level text, wherein the engineering semantic level text includes at least one level of extremely low, low, medium, high and extremely high. The feature level, the engineering semantic level text, and the corresponding binning rule identifier are written together into the evidence master table.

4. The method according to claim 1, characterized in that, The method further includes: Acquire maintenance images, and perform target area detection, region segmentation, and key part localization on the maintenance images to obtain the target area; The target area and the maintenance image are input into a pre-set visual language model to output natural language descriptions and semantic labels through the visual language model; The generated natural language description is written into the main evidence table as image evidence, and the semantic tag is written into the attachment index table.

5. The method according to claim 1, characterized in that, A consistency assessment is performed on the set of evidence items to determine the assessment result, specifically including: Read multiple types of evidence items corresponding to the same case identifier from the main evidence table. These multiple types of evidence items include fault code evidence, operational characteristic evidence, image evidence, and text description evidence. Multiple pre-defined consistency relationships are identified, and consistency analysis is performed based on these relationships to determine a consistency support score. The consistency relationships include temporal consistency, semantic consistency, and mechanistic consistency. A weighted scoring method is used to calculate the consistency score, and the formula for calculating the consistency score is as follows: in, Indicates the consistency score. This represents the weight corresponding to the i-th type of evidence. The consistency support score represents the i-th type of evidence.

6. The method according to claim 1, characterized in that, Identify candidate conflicting evidence items and determine the corresponding conflict relationships and causes. Write the conflict relationships and causes into the conflict relationship table, specifically including: Perform an evidence consistency assessment on multiple evidence items corresponding to the same case identifier to determine the consistency score between different evidence items; When the consistency score between different evidence items is lower than a preset threshold, the corresponding evidence item is marked as a candidate conflicting evidence item; Based on the evidence type, normalized text, feature level, engineering semantic level text, and engine mechanism constraint rules, conflict identification is performed on candidate conflict evidence items. Write the conflict relationships and causes into the conflict relationship table.

7. The method according to claim 1, characterized in that, Candidate root causes are determined based on the set of evidence items, and root cause inference is performed on the candidate root causes, specifically including: Read the set of evidence items corresponding to the case identifier, as well as the preset engine component tree, the correspondence between fault codes and components, the correspondence between fault phenomena and components, and the correspondence between characteristic anomalies and components; The initial set of candidate root causes is determined based on fault code evidence, text description evidence, and component tree relationships. Match various types of evidence with candidate root causes to determine the evidence support for each candidate root cause. The candidate probability result of each candidate root cause is determined based on the candidate root cause set, supporting evidence, conflicting evidence, and mechanism constraint information corresponding to each candidate root cause, and the candidate probability result is used as the confidence level of the candidate root cause. The candidate root causes are sorted according to the confidence level to determine the candidate root cause list, and the key supporting evidence and conflict hints corresponding to each candidate root cause are output simultaneously.

8. The method according to claim 1, characterized in that, Determining the verification path for the candidate root causes specifically includes: A corresponding verification path is generated based on the candidate root cause, and the verification path includes the target component, operation action, judgment condition, and type of supplementary evidence required. The verification path is then output to the maintenance technician for execution. When new or updated evidence is written back to the main evidence table, a re-inference condition is determined to trigger incremental re-inference based on the re-inference condition. After triggering, the steps of root cause inference and verification path generation are re-executed. The re-inference conditions include adding new evidence items, changes in the engineering semantic level text of the same evidence item exceeding a preset threshold, changes in the conflict state, and consistency scores crossing a preset threshold range.

9. An engine fault diagnosis system based on multimodal data, characterized in that, include: The data acquisition module is used to acquire multi-source data of the engine to be diagnosed. The multi-source data includes engine basic data, fault code information, work order information, sales records, maintenance records, running sequence data, and maintenance image data. The evidence table construction module is used to parse the collected multi-source data into unified evidence items and write them into the evidence master table. The evidence master table serves as a unified intermediate representation in the diagnostic process and includes identification and association field groups, source and time field groups, content and semantic field groups, and credibility, conflict and audit field groups. The evidence table construction module is also used to write the attachment index information of the original data file into the attachment index table, write the conflict relationship between evidence items into the conflict relationship table, and write the engineering semantic classification rules into the binning rule table. The characterization generation module includes a time series characterization submodule, which is used to slice the collected time series data according to a preset time window and extract feature values, perform engineering semantic classification on the feature values ​​according to the binning rules in the binning rule table to obtain the engineering semantic level, and then generate standardized text based on the constraint template and write it into the evidence master table. The diagnostic intelligent agent module includes an evidence manager, an evidence consistency evaluator, a collection decision-maker, a root cause inferencer, and a verification path generator. The evidence manager reads evidence items from the main evidence table and performs aggregation. The evidence consistency evaluator performs consistency evaluation based on conflict relationships in the conflict relationship table. The collection decision-maker generates supplementary collection instructions when evidence is insufficient, conflicts exist, or confidence levels are insufficient. The root cause inferencer combines the evidence content and component tree constraint information from the main evidence table to output candidate root causes and their confidence levels. The verification path generator generates corresponding investigation steps and verification actions for each candidate root cause. The output and interaction module is used to display candidate root causes, confidence levels, evidence consistency results, conflict alerts, and verification paths to maintenance technicians, and supports users to supplement evidence to trigger the diagnostic agent to perform incremental re-inference.

10. An engine fault diagnosis device based on multimodal data, characterized in that, include: At least one processor; as well as, A memory communicatively connected to the at least one processor; wherein, The memory stores instructions executable by the at least one processor, which, when executed by the at least one processor, enable the engine fault diagnosis device based on multimodal data to perform the method described in any one of claims 1-8.

Images