Multi-modal diagnostic collaboration system and method based on model context protocol

Abstract

The application discloses a multi-modal diagnostic collaboration system and method based on a model context protocol, relates to the technical field of multi-modal diagnosis of industrial equipment, and forms a diagnostic context package by acquiring equipment operation data, multi-modal detection data and historical diagnostic records of the industrial equipment within a diagnostic cycle, performing time alignment, equipment identification matching, working condition state association and fault clue extraction; the vibration diagnostic model, the image recognition model and the text analysis model are packaged into standardized diagnostic services according to the model context protocol to establish a service capability set, and a collaborative calling strategy is generated according to data modalities, working condition states, fault clues and diagnostic targets; the abnormal categories, fault positions, confidence levels, evidence segments and disposal suggestions output by the diagnostic services are uniformly mapped, associated and verified, conflicts are rearranged, and the reliability is corrected to generate a comprehensive diagnostic result, thereby improving the standardized access capability, the collaborative scheduling capability and the reliability of diagnostic conclusions of the multi-modal diagnostic services.

Description

Technical Field

[0001] This invention relates to the field of multimodal diagnostic technology for industrial equipment, and more specifically to a multimodal diagnostic collaborative system and method based on model context protocol. Background Technology

[0002] The operating status of industrial equipment is influenced by a variety of factors, including load, speed, temperature, vibration, image defects, and maintenance history. A single diagnostic model can typically only assess a specific mode or fault range, making it difficult to fully utilize the complementary relationships between different data. While existing multimodal diagnostic solutions can incorporate models such as vibration analysis, image recognition, or text analysis, the input fields, output structures, applicable operating conditions, and fault labeling systems of these models are often inconsistent. This leads to reliance on manual adaptation for model integration, difficulties in expanding diagnostic services, and a lack of system-wide ability to automatically select the appropriate model based on the current equipment status and data availability.

[0003] Meanwhile, existing multi-model diagnostic result fusion methods often rely on simple voting, confidence weighting, or manual experience-based judgment, lacking comprehensive measurements of fault location consistency, fault correlation, source service reliability, and the degree of evidence conflict. When different diagnostic models output conflicting fault categories, fault locations, or confidence levels, the system struggles to determine the supporting relationships and sources of conflict between the evidence, easily leading to unstable diagnostic conclusions, unfounded treatment recommendations, and insufficient utilization of historical fault clues. Therefore, there is an urgent need for a multi-modal diagnostic collaboration solution that can unify model protocol interfaces, dynamically schedule multi-modal diagnostic services, and reliably fuse heterogeneous diagnostic evidence. Summary of the Invention

[0004] This invention addresses the shortcomings of existing technologies by proposing a multimodal diagnostic collaborative system and method based on model context protocols.

[0005] The technical solution to achieve the objective of this invention is as follows:

[0006] On the one hand, the multimodal diagnostic collaboration method based on model context protocol includes the following steps:

[0007] Acquire equipment operation data, multimodal detection data, and historical diagnostic records, and form a diagnostic context package by time alignment, equipment identifier matching, operating condition association, and fault clue extraction;

[0008] The vibration diagnosis model, image recognition model, and text analysis model are encapsulated into a diagnostic service according to the model context protocol, and a service capability set is established.

[0009] Based on the diagnostic context package and service capability set, select matching diagnostic services and generate collaborative invocation strategies according to data integrity, service response status and fault correlation.

[0010] The diagnostic service is invoked according to the collaborative invocation strategy. The anomaly category, fault location, confidence level, evidence fragments and handling suggestions output by each diagnostic service are mapped into a unified format to form a diagnostic evidence set.

[0011] Based on the diagnostic evidence set, the location consistency, fault correlation and confidence contribution of each piece of diagnostic evidence are calculated. Conflicting diagnostic evidence is rearranged and its credibility is corrected to generate a comprehensive diagnostic result.

[0012] Furthermore, a comprehensive diagnostic result is generated, including:

[0013] Perform standardized preprocessing on the valid diagnostic evidence in the diagnostic evidence set to extract the basic feature set;

[0014] Based on the set of basic features, the consistency of location, the degree of correlation of faults, and the overall similarity among different diagnostic evidence are calculated to obtain the overall similarity matrix;

[0015] The confidence contribution of diagnostic evidence is calculated based on the comprehensive similarity matrix, conflicting diagnostic evidence is identified and dynamic confidence correction is performed to generate an evidence priority sequence.

[0016] Candidate fault hypotheses are formed by combining different abnormal categories and fault locations in the diagnostic evidence set. The comprehensive confidence level is calculated based on the evidence priority sequence to generate a sequence of candidate fault hypotheses. After verification by historical fault clues and current operating conditions, the comprehensive diagnostic results are output.

[0017] Furthermore, generate collaborative invocation strategies, including:

[0018] The available modal set, dominant operating condition state, historical fault label set, and diagnostic target priority are parsed from the diagnostic context package to construct a structured diagnostic requirement feature vector;

[0019] Based on the structured diagnostic requirement feature vector, modality matching verification, operational condition adaptability verification, and service availability verification are performed on the service capability set to form a candidate service set;

[0020] Calculate a comprehensive matching score for the diagnostic services in the candidate service set, and filter to obtain the final service set to be invoked;

[0021] Priority scores are calculated based on the final set of called services, a service call priority sequence is generated, and an adaptive fully parallel collaborative calling strategy is generated from the service call priority sequence.

[0022] Furthermore, establish a service capability set, including:

[0023] Define the service identification layer, input / output interface layer, capability description layer, and runtime state layer according to the model context protocol;

[0024] The vibration diagnosis model, image recognition model, and text analysis model are encapsulated into standardized diagnostic services, forming vibration diagnosis services, image recognition services, and text analysis services.

[0025] The standardized diagnostic service performs automated parsing of diagnostic service metadata to obtain an input field description table, an applicable modality set, a set of diagnosable fault labels, an output structure description table, and historical call status.

[0026] The standardized diagnostic service capabilities are quantitatively calibrated and verified, and metadata information and quantitative capability indicators are integrated into a service capability set.

[0027] Furthermore, a diagnostic context package is formed, including:

[0028] Simultaneous acquisition and standardization conversion of equipment operation data, multimodal detection data, and historical diagnostic records are performed to obtain standardized data;

[0029] Based on a unified time benchmark, time alignment is performed on standardized data to generate time-synchronized multimodal data sequences. Based on a global device coding system, device identifier matching is performed on the multimodal data sequences to remove data with abnormal identifiers.

[0030] The operating condition labels are determined by clustering based on operating parameters. The operating condition labels are then associated with the multimodal data sequence at each time point to form a multimodal dataset with operating condition labels.

[0031] The fault labels, fault occurrence time, fault location, fault severity, fault cause, and maintenance measures are extracted from historical diagnostic records to form a structured fault history database. This database is then integrated with a multimodal dataset labeled with operating conditions to form a diagnostic context package.

[0032] Further diagnostic services include:

[0033] Vibration diagnostic services preprocess vibration acceleration signals, extract time-domain and frequency-domain features, construct feature vectors and input them into a random forest classifier, and output anomaly category, fault location, confidence level, evidence fragments and treatment suggestions;

[0034] The image recognition service performs preprocessing on infrared thermal imaging images and visible light images, uses feature extraction and template matching to identify suspected fault areas, and performs secondary confirmation through a lightweight convolutional neural network, outputting anomaly category, fault location, confidence level, evidence fragments, and handling suggestions;

[0035] The text analysis service performs preprocessing on structured text data and uses a combination of exact and fuzzy matching to match it with an industrial fault structured knowledge base, outputting anomaly category, fault location, confidence level, evidence fragments, and handling suggestions.

[0036] Furthermore, a diagnostic evidence set is formed, including:

[0037] Receive the raw output data returned by the diagnostic service, perform field integrity checks, field validity checks, and logical consistency checks on the raw output data, and retain the valid raw output;

[0038] Establish a structured diagnostic evidence data structure compatible with the industrial fault structured knowledge base, and map the effective raw output into diagnostic evidence containing a unique evidence identifier, source service identifier, diagnostic timestamp, anomaly category, fault location, confidence level, evidence fragments, and handling suggestions;

[0039] Multimodal differentiated evidence fragment structure extraction is performed on diagnostic evidence. The extraction results are written into the evidence fragment field, and empty set verification, correlation verification and redundancy verification are performed to form a diagnostic evidence set.

[0040] Furthermore, multimodal differential evidence fragment extraction is performed, including:

[0041] Extract time-domain feature values, frequency-domain feature values, characteristic frequency peaks and their corresponding amplitudes from the evidence fragments output by the vibration diagnostic service to construct a vibration feature evidence vector;

[0042] Extract feature point coordinates, descriptors, template matching similarity, bounding box coordinates of suspected fault areas, and network feature vectors from the evidence fragments output by the image recognition service to construct image feature evidence vectors;

[0043] Extract the text content, start position, end position, and matching similarity value of matching keywords from the evidence fragments output by the text analysis service to construct a text feature evidence vector;

[0044] The feature evidence vector is associated and bound with the corresponding anomaly category and fault location to establish a ternary association relationship between the feature evidence vector, the fault location, and the anomaly category.

[0045] Furthermore, dynamic credibility correction is performed on the conflict diagnosis evidence to generate an evidence priority sequence, including:

[0046] Based on the inclusion relationship of fault locations and the division of component groups in the industrial fault structured knowledge base, the consistency of location among diagnostic evidence is calculated;

[0047] Based on the set of associated fault labels corresponding to anomaly categories in the industrial fault structured knowledge base, the fault correlation degree between diagnostic evidence is calculated.

[0048] By weighting the consistency of the parts with the correlation with the fault, the overall similarity between diagnostic evidence is obtained.

[0049] An evidence conflict coefficient matrix is ​​constructed based on comprehensive similarity. The average conflict coefficient is calculated and conflict diagnosis evidence is identified. Dynamic credibility correction is performed on the conflict diagnosis evidence to generate an evidence priority sequence.

[0050] Secondly, the multimodal diagnostic collaboration system based on the model context protocol includes a data acquisition module, a service registration module, a policy generation module, an evidence collection module, and a fusion scheduling module.

[0051] The data acquisition module obtains equipment operation data, multimodal detection data, and historical diagnostic records. After time alignment, equipment identifier matching, operating condition association, and fault clue extraction, a diagnostic context package is formed.

[0052] The service registration module encapsulates the vibration diagnosis model, image recognition model, and text analysis model into a diagnostic service according to the model context protocol, and establishes a service capability set.

[0053] The strategy generation module filters matching diagnostic services based on the diagnostic context package and service capability set, and generates collaborative invocation strategies according to data integrity, service response status and fault correlation.

[0054] The evidence collection module invokes the diagnostic service according to the collaborative invocation strategy, and receives the anomaly type, fault location, confidence level, evidence fragments and handling suggestions output by each diagnostic service and maps them into a unified format to form a diagnostic evidence set;

[0055] The fusion scheduling module calculates the location consistency, fault correlation, and confidence contribution among various diagnostic evidences based on the diagnostic evidence set, performs evidence rearrangement and confidence correction on conflicting diagnostic evidences, and generates a comprehensive diagnostic result.

[0056] Compared with the prior art, the advantages of this invention are as follows:

[0057] 1. Vibration diagnostic models, image recognition models, and text analysis models are encapsulated into standardized diagnostic services through a model context protocol. The input fields, applicable modes, fault ranges, output structures, and historical call status of each diagnostic service are centrally maintained within the service capabilities. This allows diagnostic models from different sources, with different structures, and deployed methods to participate in collaborative diagnosis through a unified interface. Based on the data modes, operating conditions, fault clues, and diagnostic targets in the diagnostic context package, the system can generate collaborative call strategies by combining data integrity, service response status, and fault correlation. This avoids the problems of complex model access, fixed call rules, and disconnect between service selection and field data status in traditional multi-model diagnostics, improving the scalability and field adaptability of the multi-modal diagnostic system.

[0058] 2. The anomaly categories, fault locations, confidence levels, evidence fragments, and handling suggestions output by each diagnostic service are mapped to a unified diagnostic evidence set. Furthermore, the consistency of location, fault correlation, and confidence contribution among diagnostic evidence are calculated. Conflicting diagnostic evidence is rearranged and its credibility corrected, ensuring that the comprehensive diagnostic results no longer rely on a single model output or simple confidence level aggregation. By jointly constraining multimodal evidence, service history reliability, and an industrial fault structured knowledge base, mutually supporting evidence and conflicting evidence can be more accurately distinguished, improving the reliability of comprehensive fault location, anomaly category determination, and handling suggestion generation. Attached Figure Description

[0059] Figure 1 Here is a flowchart of a multimodal diagnostic collaborative method based on model context protocol;

[0060] Figure 2 This is a flowchart illustrating the formation of the diagnostic context packet in this invention.

[0061] Figure 3 This is a flowchart illustrating the process of generating comprehensive diagnostic results through evidence fusion in this invention.

[0062] Figure 4 This is a structural diagram of the multimodal diagnostic collaborative system based on the model context protocol in this invention. Detailed Implementation

[0063] The present invention will be further described in detail below with reference to the accompanying drawings and embodiments.

[0064] Example 1

[0065] This invention discloses a multimodal diagnostic collaborative method based on a model context protocol, comprising the following steps:

[0066] S1: Acquire equipment operation data, multimodal detection data and historical diagnostic records of industrial equipment during the diagnostic cycle, perform time alignment, equipment identifier matching and operating status association on the equipment operation data and multimodal detection data, and extract fault tags and maintenance clues corresponding to the current equipment from the historical diagnostic records to form a diagnostic context package.

[0067] S2: Encapsulate the vibration diagnosis model, image recognition model, and text analysis model into a diagnosis service according to the model context protocol, parse the input fields, applicable modes, fault range, output structure, and historical call status of each diagnosis service, and establish a service capability set.

[0068] S3: Based on the data modality, operating status, fault clues and diagnostic objectives in the diagnostic context package, select matching diagnostic services from the service capability set, and generate a collaborative invocation strategy according to data integrity, service response status and fault correlation.

[0069] S4: Send diagnostic context packets to the corresponding diagnostic services according to the collaborative invocation strategy, receive the anomaly categories, fault locations, confidence levels, evidence fragments and handling suggestions output by each diagnostic service, and map them into a unified format to form a diagnostic evidence set.

[0070] S5: Calculate the location consistency, fault correlation and confidence contribution among the diagnostic evidences based on the diagnostic evidence set, perform evidence rearrangement and confidence correction on conflicting diagnostic evidence, and generate a comprehensive diagnostic result.

[0071] refer to Figure 1 , Figure 1 This is a flowchart of a multimodal diagnostic collaboration method based on a model context protocol.

[0072] In step S1, the equipment operation data, multimodal detection data, and historical diagnostic records of the industrial equipment during the diagnostic cycle are acquired. Time alignment, equipment identifier matching, and operating condition status association are performed. Fault tags and maintenance clues are extracted to form a diagnostic context package, including:

[0073] S101: Synchronous acquisition and standardized conversion of core multi-source data.

[0074] According to the preset 1-hour diagnostic cycle, three types of core data are collected synchronously from the equipment control system, edge computing nodes, intelligent detection terminals, and equipment management platform: The first type is equipment operation data, including three core status parameters: equipment speed, load rate, and drive-end bearing temperature. The sampling frequency is uniformly set to 1Hz. The 1-hour diagnostic cycle is determined by considering the speed of industrial equipment fault evolution, edge storage capacity, and real-time diagnostic requirements. The 1Hz low-speed sampling can reduce the data storage overhead of conventional operation. The second type is multimodal detection data, including three high diagnostic value modes: vibration acceleration signal, infrared thermal imaging image, and visible light image. The sampling frequency of the vibration acceleration signal is set to 10kHz. This sampling frequency can completely capture the harmonics of bearing and gear fault characteristics and meet the minimum sampling standard for vibration fault spectrum analysis of rotating equipment. The resolution of the infrared thermal imaging image is fixed at 320×240 pixels, and the resolution of the visible light image is fixed at 1280×720 pixels to balance image fault details and edge transmission bandwidth. The third type is historical diagnostic records, including fault reports, maintenance work orders, and expert diagnostic records of the current equipment in the past two years. All collected data were uniformly converted into JSON format, and a unique data identifier, collection timestamp, and data source identifier were added to each data item to complete the standardized processing of multi-source heterogeneous data.

[0075] S102: Multimodal data time alignment based on a unified time base.

[0076] Using UTC time provided by the Global Positioning System (GPS) as the global unified time reference, time alignment is performed on all standardized data. For time-series data, including equipment operation data and vibration acceleration signals, cubic spline interpolation is used to resample them onto a unified time axis of 1Hz. For non-time-series data, including infrared thermal imaging images and visible light images, their acquisition timestamps are mapped to the corresponding time points on the unified time axis, and a 3-second time-series data association window is added before and after each image to capture instantaneous fluctuations in equipment operating conditions before and after image capture, avoiding misjudgments caused by a lack of operating condition context in a single frame image. The time deviation of each data item is calculated, and data with a time deviation exceeding ±10ms is marked as time anomaly data and removed. The ±10ms deviation threshold is adapted to the upper limit of industrial bus transmission delay; time-series data exceeding this range will exhibit operating condition mismatch. Finally, a multimodal data sequence with fully synchronized time is generated.

[0077] S103: Device identifier matching based on the global device coding system.

[0078] A globally unique equipment coding system covering all industrial equipment is established. Each equipment code consists of 16 characters, with the following structure: the first two digits are the equipment type code, the middle four digits are the production line code, the next four digits are the installation location code, and the last six digits are the factory serial number. Three types of equipment identifiers—equipment serial number, asset serial number, and IP address—are extracted from equipment operation data, multimodal detection data, and historical diagnostic records. An equipment identifier mapping table is established to map various heterogeneous equipment identifiers to their corresponding globally unique equipment codes. The consistency of equipment identifiers for all data items is verified one by one. Data that cannot be matched with a valid globally unique equipment code is marked as abnormal data and removed.

[0079] S104: Working condition status association and labeling based on operating parameter clustering.

[0080] Two core operating condition feature parameters, speed and load rate, were extracted from the time-aligned equipment operation data. The K-means clustering algorithm was used to classify the equipment operating status into five standard operating conditions: shutdown, no-load, light load, medium load, and heavy load. The convergence condition of K-means clustering was set as follows: the number of iterations ≤ 100 and the change in cluster centers ≤ 1e-4. The Euclidean distance between the operating condition feature parameter at each time point and the center of each standard operating condition was calculated, and that time point was marked as the nearest standard operating condition. The operating condition status label was associated with the multimodal data sequence time-by-time, and a corresponding operating condition status label was added to each multimodal data item, forming a multimodal dataset with operating condition labels.

[0081] S105: Extraction of historical diagnostic records and encapsulation of diagnostic context.

[0082] A pre-built structured knowledge base for industrial faults is constructed, comprising five sub-bases: fault tag database, fault location database, fault severity database, fault cause database, and maintenance measure database. Each sub-base stores standardized terms and their synonyms, abbreviations, and mappings to commonly used industry expressions. The fault tag database contains 128 common fault tags for industrial equipment, and the fault location database contains 64 typical equipment component names. Text preprocessing is performed on historical diagnostic records, including special character removal, case unification, word segmentation, and stop word filtering. A combination of exact and fuzzy matching is used to match the preprocessed text with the terms in the industrial fault structured knowledge base, extracting six core information categories: fault tag, fault occurrence time, fault location, fault severity, fault cause, and maintenance measures corresponding to the current equipment. A fuzzy matching similarity threshold of 0.85 is set. This threshold balances terminology tolerance with matching accuracy; too low a threshold may result in irrelevant matches, while too high a threshold may result in the loss of abbreviations / colloquial fault records. Matches with similarity below this threshold are marked as pending confirmation and discarded. The six extracted core information categories are subjected to integrity verification. Historical diagnostic records lacking two or more categories of core information are marked as invalid and removed, ultimately forming a structured fault history database. Fault records with an interval of ≤180 days from the current diagnostic cycle are selected from the fault history database as historical fault clues and integrated with the multimodal dataset labeled with operating conditions. Finally, a diagnostic context package is formed, which includes five parts: basic equipment information, time-synchronized multimodal dataset, operating condition sequence, historical fault labels, and historical maintenance clues.

[0083] refer to Figure 2 , Figure 2 A flowchart is generated for the diagnostic context package.

[0084] In step S2, the vibration diagnosis model, image recognition model, and text analysis model are encapsulated into diagnostic services according to the model context protocol. The input fields, applicable modes, fault ranges, output structures, and historical call statuses of each diagnostic service are parsed to establish a service capability set, including:

[0085] S201: Model Context Protocol Specification Definition.

[0086] A predefined model context protocol is adopted, employing a layered architecture with four core layers: a service identification layer, an input / output interface layer, a capability description layer, and a runtime status layer. The service identification layer defines the service's unique identifier, service name, model version number, and deployment node address fields. The input / output interface layer defines the input field set, output field set, data transmission format, and interface call method fields. The capability description layer defines the applicable modalities, supported device types, diagnosable fault range, and operational adaptability fields. The runtime status layer defines the deployment status, historical call statistics, and performance metrics fields. The protocol stipulates that all diagnostic services communicate using a RESTful interface, and the data transmission format is uniformly JSON, consistent with the data format of the diagnostic context package.

[0087] S202: Standardized service encapsulation for multimodal diagnostic models.

[0088] The vibration diagnosis model, image recognition model, and text analysis model are encapsulated as independent diagnostic service instances conforming to the model context protocol specification, generating three standardized diagnostic services: vibration diagnosis service, image recognition service, and text analysis service. Each diagnostic service is assigned a globally unique 128-bit UUID service identifier, configured with corresponding input / output interfaces, embedded with the capability description metadata specified in the protocol, and integrated with a runtime monitoring module. All diagnostic services are deployed using a containerized approach, supporting independent start / stop and horizontal scaling.

[0089] The vibration diagnosis model is a random forest fault classification model based on time-domain and frequency-domain feature fusion. This model first preprocesses the input vibration acceleration signal, extracting 16-dimensional time-domain features and 32-dimensional frequency-domain features to construct a 48-dimensional feature vector. The time-domain features include RMS value, peak value, kurtosis, margin, impulse factor, and waveform factor; the frequency-domain features include fundamental frequency amplitude, harmonic amplitude, sideband energy, and characteristic frequency energy percentage. A random forest classifier containing 200 decision trees is used for fault identification, with a maximum tree depth of 15 and a minimum number of splits of 5. The model is pre-trained using a standard vibration dataset containing 128 common industrial equipment faults. This dataset integrates publicly available vibration samples from CWRU international testing platforms with self-developed fault test benches and production line measured signals. The data is uniformly sampled and labeled with 128 standard fault tags by industry experts, covering all mainstream fault conditions of rotating equipment. The training set to test set ratio is 8:2, and the cross-validation accuracy is ≥92%.

[0090] The image recognition model is a hybrid fault identification model based on ORB feature extraction and template matching combined with a lightweight convolutional neural network. The model first preprocesses the input infrared thermal imaging image and visible light image, including grayscale conversion, histogram equalization, and noise filtering. It then uses the ORB algorithm to extract key feature points from the image and performs feature matching with a pre-built template library containing 64 types of typical equipment fault images. This template library is expanded from real-world infrared and visible light fault images captured on-site, and the 64 types of equipment defect standard features are manually labeled. Imaging parameters are standardized to adapt to industrial on-site image matching scenarios. For suspected fault areas with a similarity between 0.6 and 0.85, the lightweight convolutional neural network is input for secondary confirmation. This lightweight convolutional neural network addresses three core pain points in industrial fault images: low proportion of small targets, strong background interference, and limited edge computing power. While strictly maintaining a minimalist architecture of 3 convolutional layers, 2 pooling layers, and 1 fully connected layer, targeted structural optimizations are performed, achieving a significant improvement in diagnostic performance while keeping the increase in parameter count controllable.

[0091] The network input size is 112×112×3, and the pixel values ​​are normalized to the [0,1] interval. The first convolutional layer uses a 3×3 convolutional kernel, stride 1, SAME padding, ReLU6 activation function and no bias term, and outputs 16 channels, followed by a batch normalization layer; the first pooling layer uses 2×2 max pooling, stride 2 and no padding, and outputs a 56×56×16 feature map.

[0092] The second and third convolutional layers employ a multi-scale depthwise separable convolutional structure, with each layer simultaneously integrating depthwise separable convolutions with 1×1, 3×3, and 5×5 branches. The output features of each branch are concatenated along the channel dimension. This design, with only a slight increase in the number of parameters, can simultaneously capture three different scales of industrial fault features: point defects, line cracks, and surface wear, effectively solving the problem of high false negative rates for small target faults in traditional single convolutional kernels. The second convolutional layer has 32 output channels, and the third convolutional layer has 64 output channels, both followed by batch normalization layers and ReLU6 activation functions.

[0093] The second pooling layer uses 2×2 max pooling with a stride of 2 and no padding, outputting a 28×28×64 feature map. Following the third convolutional layer, a parameter-free fault-aware attention module is implemented. This module directly calculates the local variance of each spatial location in the feature map as the attention weight, enhancing fault regions with high variance and suppressing background regions with low variance. This module requires no additional training parameters or computational overhead, improving the model's accuracy in identifying minute faults accounting for less than 5%.

[0094] The network ends with global average pooling instead of traditional flattening layers, compressing the 28×28×64 feature map into a 64-dimensional feature vector; the fully connected layer contains 128 neurons, uses the ReLU6 activation function, and sets the dropout rate to 0.3 to prevent overfitting; the output layer is a Softmax layer with 64 neurons, corresponding to 64 typical equipment component faults in the fault location library of the industrial fault structured knowledge base.

[0095] The network was pre-trained using 12,000 labeled fault images, employing the Adam optimizer with an initial learning rate of 1e-4, a batch size of 32, 50 training epochs, and an early stopping strategy (stopping the training if the validation set loss does not decrease for 5 consecutive epochs). The loss function was cross-entropy loss, and the final test set accuracy was ≥91%. Its output categories correspond one-to-one with the fault location database of the industrial fault structured knowledge base.

[0096] The text analysis model is a rule-based text semantic matching model based on an industrial fault structured knowledge base. This model does not rely on any deep learning pre-trained models and operates entirely based on the industrial fault structured knowledge base constructed in step S105. Its core logic is as follows: Preprocessing operations identical to those in step S105 are performed on the input text data, namely, removing special characters, standardizing capitalization, word segmentation, and stop word filtering. Then, a combination of exact matching and fuzzy matching is used to map the words in the text to standardized terms, synonyms, abbreviations, and commonly used industry expressions in the knowledge base, extracting structured fault information. The fuzzy matching similarity threshold of this model remains consistent with step S105, set at 0.85.

[0097] S203: Automated parsing of diagnostic service metadata.

[0098] For each encapsulated diagnostic service, perform automated metadata parsing to extract the five core metadata information specified in the protocol:

[0099] Input field parsing: Extract the name, data type, required fields, value range, and format requirements of each diagnostic service's input fields to form an input field description table;

[0100] Applicable modal parsing: Extract the data modal types supported by each diagnostic service. Among them, the vibration diagnostic service only supports the vibration acceleration signal mode, the image recognition service supports two modes: infrared thermal imaging image and visible light image, and the text analysis service only supports the structured text data mode.

[0101] Fault Scope Analysis: Extract the set of fault tags that each diagnostic service can diagnose, and this set corresponds one-to-one with the fault tag library in the industrial fault structured knowledge base;

[0102] Output structure parsing: Extract the output field name, data type, meaning, and value range of each diagnostic service to form an output structure description table;

[0103] Historical call status analysis: Extract historical statistics for each diagnostic service, including cumulative call count, average response time, most recent call time, call success rate, and fault diagnosis accuracy.

[0104] S204: Quantitative calibration and verification of diagnostic service capabilities.

[0105] A quantitative indicator system for diagnostic service capabilities was established, encompassing five dimensions: diagnostic accuracy, fault location accuracy, average response time, operating condition adaptability, and data integrity requirements. This system quantifies and verifies the core capabilities of each diagnostic service. Standard industrial equipment fault datasets were used to test each diagnostic service, with unified calibration thresholds set as follows: diagnostic accuracy ≥ 90%, fault location accuracy down to the component level, average response time ≤ 500ms, operating condition adaptability covering five standard operating conditions (shutdown, no-load, light load, medium load, and heavy load), and normal diagnostic results output when input data integrity ≥ 80%. Diagnostic services that did not meet the calibration thresholds were marked as services requiring optimization and temporarily excluded from the service capability set. Monthly capability verification of the diagnostic services already included in the service capability set was conducted using newly added actual fault data, updating their quantitative indicator data.

[0106] S205: Construction and dynamic updating of service capability sets.

[0107] The metadata and quantitative capability metrics of all quantified diagnostic services are integrated to construct a structured service capability set. This set is stored in a relational database, with each record corresponding to a diagnostic service and containing four parts: service identifier, metadata information, quantitative capability metrics, and operational status information. A dynamic update mechanism for the service capability set is established: when a new diagnostic service is added, the metadata parsing and capability calibration process is automatically executed, and the service capability set is automatically added after verification; when the model version of a diagnostic service is updated, the metadata parsing and capability calibration process is re-executed, and the corresponding record is updated; the historical call status data of each diagnostic service is automatically updated hourly; when a diagnostic service's call success rate is below 80% for seven consecutive days, it is automatically marked as a failed service and removed from the service capability set.

[0108] In step S3, based on the data modality, operating status, fault clues, and diagnostic objectives in the diagnostic context package, matching diagnostic services are selected from the service capability set. A collaborative invocation strategy is generated according to data integrity, service response status, and fault correlation, including:

[0109] S301: Quantitative extraction of multi-dimensional requirement features from diagnostic context packages.

[0110] Four core diagnostic requirement features are parsed and extracted from the diagnostic context package, and a structured diagnostic requirement feature vector is constructed after standardized and quantified processing. .in, This is the modal availability vector. , The availability status corresponds to three modalities: vibration acceleration signal, infrared thermal imaging and visible light image, and structured text. When the proportion of valid data points in a certain modality to the total number of data points to be collected in the diagnostic cycle is not less than 0.8, the availability value of that modality is 1; otherwise, it is 0. The threshold of 0.8 is determined in conjunction with the actual packet loss conditions of industrial sensors. Data integrity of 80% or more can retain valid fault characteristics. Below this proportion, modal information is severely missing and has no diagnostic reference value. Based on the modal availability vector, a set of available modalities for the diagnostic context packet is generated. This set contains all modal types with an availability value of 1, specifically one or more of vibration acceleration signal, infrared thermal imaging image, visible light image, and structured text. The dominant operating condition during the diagnostic cycle is determined by statistically analyzing the cumulative duration percentage of each standard operating condition during the diagnostic cycle. The operating condition with the highest percentage is taken as the dominant operating condition. If there are two or more operating conditions with a percentage difference of less than 0.1, the operating condition with a higher fault correlation is taken as the dominant operating condition. This is a historical fault tag set, containing all fault tags that occurred on the equipment in the past 180 days, along with the corresponding fault occurrence time and severity information. This time window is determined based on the statistical regularity of the fault recurrence cycle of industrial equipment. The priority of the diagnostic target is set to 1, 2, or 3, which correspond to the three priority levels of anomaly detection, fault location, and fault cause analysis, respectively. The default value is 2, which can be adjusted according to user input.

[0111] S302: Initial screening of diagnostic services based on the triple constraints of modality, operating condition, and availability.

[0112] Using structured diagnostic requirement feature vectors As input, a progressive initial screening operation is performed on all diagnostic services stored in the service capability set. First, modality matching verification is performed. The applicable modality set for each diagnostic service is pre-extracted from the service's capability description layer metadata and stored in the metadata information module of the service capability set, associated with the service identifier. When the model version of a diagnostic service is updated or its capability scope is adjusted, the applicable modality set is automatically updated synchronously. The specific execution process of modality matching verification is as follows: traverse each diagnostic service in the service capability set, extract its corresponding applicable modality set, and check whether this set and the available modality set of the diagnostic context package have at least one common modality type. If they exist, the diagnostic service passes the modality matching verification; otherwise, it is directly excluded.

[0113] Next, a working condition adaptability check is performed to filter out the set of working condition adaptability ranges. Includes dominant operating condition status The diagnostic service for this operating condition is formed as follows: During the S204 service capability quantification and calibration, five standard operating condition datasets (stop, idle, light load, medium load, and heavy load) are used to test the single-service diagnostic accuracy. All operating conditions with a historical diagnostic accuracy of not less than 0.85 are collected to form the operating condition's adaptability set. This set is persistently stored in the service capability set metadata, and is recalculated and updated after model iteration. The 0.85 accuracy threshold is set based on the requirements for implementing industrial fault diagnosis. If it is lower than this value, the model has too high a risk of misjudging or missing faults under the corresponding operating conditions, and therefore lacks reliable diagnostic value. Finally, service availability verification is performed, selecting those with normal deployment status and a high success rate of calls within the last 24 hours. Diagnostic services For diagnostic services The ratio of the number of calls that successfully returned valid diagnostic results to the total number of calls in the past 24 hours. This parameter is obtained in real time from the runtime status layer of the service capability set. Diagnostic services that pass the above triple constraint verification constitute a candidate service set. Diagnostic services that fail to pass any of the constraints are directly excluded.

[0114] S303: Candidate service screening based on weighted comprehensive matching degree.

[0115] For candidate service set Each diagnostic service Calculate its comprehensive matching score with the current diagnostic needs. The calculation formula is:

[0116] ,

[0117] in, The fault correlation weight is set to 0.6. This weight was determined through orthogonal experimental analysis of 1,000 industrial equipment fault diagnosis cases covering different equipment types and fault modes. It can achieve the optimal balance between the guiding role of historical fault clues and the supporting role of current data quality. Fault correlation is defined as the set of historical fault labels. The ratio of the size of the intersection of the diagnostic service's diagnosable fault tag set and the size of the historical fault tag set, i.e.:

[0118] ,

[0119] Among them, each diagnostic service has a set of diagnosable fault labels. All fault tags are pre-parsed and extracted from the service's capability description layer metadata, and stored in the service capability set's metadata information module in association with the service identifier. All fault tags in this set come from the fault tag library in the industrial fault structured knowledge base constructed in step S105. When the diagnostic service model version is updated or the diagnosable fault range is adjusted, the diagnosable fault tag set is automatically updated synchronously. ,but ; Data fit is defined as the ratio of the number of valid fields provided by the diagnostic context package to the total number of input fields required by the diagnostic service, where required fields have a weight of 1 and optional fields have a weight of 0.5. A comprehensive matching score threshold is set. Filter out The diagnostic services constitute the final set of services invoked. Meanwhile, the comprehensive matching score of each service is retained as the core basis for subsequent priority ranking. Among them, the required fields are the core input for the model to complete the diagnosis. If they are missing, the model will fail directly, so the weight is set to 1. The optional fields are only used to help improve accuracy and have a limited impact. Therefore, the weight is halved to 0.5. The comprehensive matching score threshold of 0.6 removes services with extremely low matching degree and that cannot provide effective diagnostic gains.

[0120] S304: Service call priority sorting based on multi-dimensional performance fusion.

[0121] Establish a priority scoring system that includes three core dimensions: fault correlation, call success rate, and average response time, to evaluate the final set of called services. Each diagnostic service Calculate its priority score The calculation formula is:

[0122] ,

[0123] in, , , These are the weighting coefficients for each dimension. This set of coefficients was determined by combining the analytic hierarchy process with the needs of industrial field diagnosis, prioritizing diagnostic accuracy while also taking into account diagnostic efficiency and system reliability. For diagnostic services Average response time This represents the maximum average response time across all diagnostic services in the final service call set. Diagnostic services are then sorted from highest to lowest priority score to generate a service call priority sequence. When the priority score difference between two diagnostic services is less than 0.05, the service with the higher overall matching score is ranked higher.

[0124] S305: Generation and validity verification of collaborative invocation strategies.

[0125] Based on service call priority sequence An adaptive, fully parallel collaborative invocation strategy is generated, with all diagnostic services employing a dependency-free parallel invocation approach. The overall diagnostic response time is determined by the slowest-responding service. A dedicated invocation parameter package is generated for each diagnostic service, containing only the required and optional fields specified in the service's input field description table. All parameters are directly extracted from the diagnostic context package, avoiding the transmission of redundant data. A dynamic invocation timeout is set for each diagnostic service, with a base timeout of 2000ms. For diagnostic services with an average response time exceeding 1000ms, the timeout is extended to twice their average response time. For diagnostic services that fail to invoke the service, differentiated processing is performed based on the failure type: if the failure is due to a network timeout, it is automatically retried once; if the failure is due to parameter errors or internal service errors, the diagnostic result for that service is discarded. The generated collaborative invocation strategy undergoes validity verification to ensure the final set of invoked services is valid. If the value is not empty and all call parameters meet the format requirements and value range of the corresponding service, a structured collaborative call strategy will be output after the verification is passed.

[0126] In step S4, a diagnostic context packet is sent to the corresponding diagnostic service according to the collaborative invocation strategy. The anomaly category, fault location, confidence level, evidence fragments, and handling suggestions output by each diagnostic service are received and mapped into a unified format to form a diagnostic evidence set, including:

[0127] S401: Parallel distribution and service execution of diagnostic context packages based on dynamic parameter pruning.

[0128] Based on the service call priority sequence in the collaborative invocation strategy, dedicated invocation parameter packets are simultaneously sent to all diagnostic services in the final invocation service set. The invocation parameter packets adopt the same JSON format as the diagnostic context packets, and only contain the required and optional fields specified in the input field description table of the corresponding diagnostic service. All parameters are directly extracted from the diagnostic context packets and pre-validated for format validity, including data type, value range, and encoding format.

[0129] A strict dynamic timeout control mechanism is implemented during the call process. The base timeout is 2000ms. For diagnostic services with an average response time exceeding 1000ms, the timeout is extended to twice the average response time, with a maximum of 5000ms. The call status of each diagnostic service is monitored in real time, recording the call initiation time, service response time, and call result status. Diagnostic services that fail due to network timeouts are automatically retried once, with the retry timeout being 1.5 times the original timeout. Diagnostic services with parameter errors or internal service errors are terminated directly, and a detailed error log containing error codes and descriptions is recorded.

[0130] Each diagnostic service performs independent diagnostic calculations based on the received call parameter package, generating raw output containing five core fields: anomaly category, fault location, confidence level, evidence fragments, and treatment recommendations. The specific generation logic is as follows:

[0131] Vibration diagnostic service: Based on the input vibration acceleration signal, perform detrending term and high-pass filtering preprocessing to extract 16-dimensional time-domain features, including RMS value, peak value, peak-to-peak value, kurtosis, margin, impulse factor, waveform factor, peak factor, kurtosis factor, margin factor, skewness, variance, standard deviation, root mean square, peak energy, and average amplitude. It also includes 32-dimensional frequency-domain features, including fundamental frequency amplitude, 2nd-10th harmonic amplitude, 1st-5th frequency shift sideband energy, 1st-10th frequency characteristic frequency energy ratio, spectral centroid, spectral variance, and spectral kurtosis. A 48-dimensional feature vector is then constructed as input to a random forest classifier containing 200 decision trees. The fault category with the highest voting ratio output by the classifier is taken as the anomaly category. The standard component name of the industrial fault structured knowledge base corresponding to the fault category is taken as the fault location. The voting ratio of the fault category is taken as the confidence level. All 48-dimensional feature values ​​extracted and the peak values ​​of the top 5 largest feature frequencies and their corresponding amplitudes are taken as evidence fragments. Based on the anomaly category and the fault location, the highest priority standard handling suggestions are matched from the maintenance measure library of the industrial fault structured knowledge base.

[0132] Image recognition service: Based on the input infrared thermal imaging image or visible light image, grayscale conversion, histogram equalization, and 3×3 Gaussian filtering preprocessing are performed. The ORB algorithm is used to extract up to 500 key feature points and their 32-dimensional binary descriptors, which are then brute-force matched against a template library containing 64 types of typical equipment fault images. For suspected fault regions with a similarity between 0.6 and 0.85, they are automatically cropped into 112×112 pixel square regions and normalized to the [0,1] interval, then input into a lightweight convolutional neural network for secondary confirmation. The fault category with the highest Softmax probability output by the network is taken as the anomaly category, the corresponding standard component name from the industrial fault structured knowledge base is taken as the fault location, and the Softmax probability value of the fault category is taken as the confidence level. The coordinates of the top 100 ORB feature points with the highest matching degree, the template matching similarity, the coordinates of the suspected fault region bounding box, and the 128-dimensional feature vector of the last layer of the network are taken as evidence fragments. Based on the anomaly category and fault location, the highest priority standard handling suggestion is matched from the maintenance measure library of the industrial fault structured knowledge base.

[0133] Text Analysis Service: Based on the input structured text data, perform preprocessing operations identical to step S105, using a combination of exact and fuzzy matching to match against the industrial fault structured knowledge base. Exact matching is used to match standard terms, while fuzzy matching is used to match synonyms, abbreviations, and commonly used industry expressions. The fault category with the highest matching degree is designated as the anomaly category, the corresponding standard component name from the industrial fault structured knowledge base is designated as the fault location, the highest matching similarity is used as the confidence level, and the text content of all matching keywords, their start and end positions in the original text, and the matching similarity value of each keyword are used as evidence fragments. Based on the anomaly category and fault location, the highest priority standard handling recommendations are matched from the maintenance measures library of the industrial fault structured knowledge base.

[0134] S402: Validation of the original output of the three-tiered progressive diagnostic service.

[0135] Receive the raw output data returned by all successfully invoked diagnostic services, and perform progressive validity checks on the raw output data.

[0136] Field integrity verification: Traverse each raw output data and check whether it contains the five core fields: anomaly category, fault location, confidence level, evidence fragment, and handling recommendation, and whether each field is not empty. Output data missing any core field or with empty field content is marked as invalid output and discarded, and the field missing type is recorded.

[0137] Field validity verification: The anomaly category is precisely matched against the fault tag library in the industrial fault structured knowledge base; anomaly categories that cannot be matched are marked as invalid. The fault location is precisely matched against the fault location library in the industrial fault structured knowledge base; fault locations that cannot be matched are marked as invalid. The confidence level is checked to ensure it falls within the [0,1] range and retains four decimal places; confidence levels outside this range or with insufficient precision are marked as invalid. The data structure of the evidence fragment conforms to the output specifications of the corresponding diagnostic service; evidence fragments with mismatched structures are marked as invalid. Outputs containing any invalid field are marked as invalid and removed, and the type of the invalid field is recorded.

[0138] Logical consistency check: Query the pre-stored association table between fault tags and fault locations in the industrial fault structured knowledge base. This table contains all valid association combinations of 128 fault tags and 64 fault locations. Check if the anomaly category and fault location in the original output exist in this association table. Outputs without valid associations are marked as invalid and removed, and the type of logical contradiction is recorded. Only valid original outputs that pass all three levels of validity checks are retained and proceed to subsequent processing.

[0139] S403: Standardized format mapping for heterogeneous diagnostic outputs that is fully compatible with the knowledge base.

[0140] A standardized diagnostic evidence data structure was established, fully compatible with the industrial fault structured knowledge base. This structure includes eight fields: unique evidence identifier, source service identifier, diagnostic timestamp, anomaly category, fault location, confidence level, evidence fragment, and treatment recommendation. A unified format mapping operation was performed on each valid original output to eliminate terminology and format differences between different diagnostic service outputs.

[0141] Unique Identifier for Evidence: A globally unique 128-bit UUID (Universally Unique Identifier) ​​is used to ensure that each piece of diagnostic evidence has a unique identifier throughout the entire system lifecycle;

[0142] Source Service Identifier: Directly taken from the globally unique 128-bit UUID service identifier of the corresponding diagnostic service, establishing a traceable relationship between diagnostic evidence and the source service;

[0143] Diagnostic timestamp: The UTC time when the diagnostic service completes the calculation and returns the result, accurate to the millisecond level, in the format "YYYY-MM-DDTHH:MM:SS.sssZ";

[0144] Anomaly Category and Fault Location: Establish a terminology mapping table to uniformly map non-standard terms, synonyms, and abbreviations output by different diagnostic services to standard terms in the industrial fault structured knowledge base. The terminology mapping table is updated synchronously with the industrial fault structured knowledge base.

[0145] Confidence level: Keep the original output values ​​in the range [0,1] unchanged and retain four decimal places of precision;

[0146] Evidence fragment: The original data structure of the original output is preserved, and data type identifiers and data version numbers are added. The data types include vibration features, image features, and text features.

[0147] Recommended action: Convert to a structured text format and label it according to the severity of the fault as "Immediate Shutdown", "Planned Maintenance" and "Continuous Monitoring", with each level corresponding to a different response time requirement.

[0148] S404: Structured extraction and ternary association binding of multimodal differentiated evidence fragments.

[0149] To address the characteristics of different modal evidence, a dedicated structured extraction method is designed to construct corresponding feature evidence vectors and establish a ternary association relationship of "feature evidence vector - fault location - abnormality category" to maximize the retention of diagnostic information for each modality.

[0150] Vibration Feature Evidence Vector Extraction: From the evidence fragments output by the vibration diagnostic service, 16-dimensional time-domain feature values, 32-dimensional frequency-domain feature values, and the top 5 largest characteristic frequency peaks and their corresponding amplitudes are extracted according to a preset vibration feature sorting order, constructing a 53-dimensional vibration feature evidence vector. This sorting order is pre-fixed based on the common feature priority in the rotating machinery fault diagnosis industry, prioritizing core features sensitive to faults such as kurtosis and harmonic amplitude, ensuring that the vibration feature vectors of different equipment are dimensionally aligned and directly comparable. The position of each element in the vector is fixed, corresponding to a specific feature parameter value, ensuring the comparability of feature vectors output by different vibration diagnostic services.

[0151] Image Feature Evidence Vector Extraction: From the evidence fragments output by the image recognition service, the 2D coordinates and 32D descriptors of the first 100 ORB feature points, template matching similarity values, the top-left and bottom-right corner coordinates of the suspected fault region bounding box, and the 128-dimensional feature vector of the last layer output by the lightweight convolutional neural network are extracted according to a preset image feature sorting order to construct the image feature evidence vector. This sorting order is set according to the image fault recognition logic, first the bottom-level matching features, then the high-level semantic features of the network, and a unified feature splicing rule to facilitate subsequent batch similarity calculation. For cases where the number of feature points is less than 100, the remaining positions are filled with 0.

[0152] Text Feature Evidence Vector Extraction: From the evidence fragments output by the text analysis service, extract the text content of all matching keywords, their start and end positions in the original text, and the matching similarity value of each keyword, in descending order of matching similarity, to construct a text feature evidence vector. The vector length dynamically changes with the number of matching keywords, and each keyword corresponds to a triple: text content, start position, and similarity value.

[0153] The extracted feature evidence vectors are associated and bound with the corresponding anomaly categories and fault locations to establish a ternary association relationship of "feature evidence vector - fault location - anomaly category". This relationship is stored in the evidence fragment field of the diagnostic evidence, providing a structured data foundation for subsequent evidence fusion.

[0154] S405: Construction and integrity verification of multi-dimensional diagnostic evidence sets.

[0155] All diagnostic evidence that has undergone standardized format mapping and structured processing is integrated into a diagnostic evidence set, stored in JSON format, with each record corresponding to an independent piece of diagnostic evidence. Multi-dimensional integrity checks are performed on the diagnostic evidence set to ensure its quality and reliability.

[0156] Empty set verification: If the number of valid evidence in the diagnostic evidence set is 0, the downgraded diagnostic process is triggered. The downgraded diagnostic process generates default diagnostic evidence based on historical fault clues and the current operating status in the diagnostic context package: If historical fault clues exist, the anomaly category of the default diagnostic evidence is the most recent fault category, the fault location is the corresponding fault location, the confidence level is 0.6, and the handling suggestion is "continuous monitoring"; if no historical fault clues exist, the anomaly category of the default diagnostic evidence is "unknown anomaly", the fault location is "whole machine", the confidence level is 0.5, and the handling suggestion is "continuous monitoring".

[0157] Association verification: Traverse each piece of evidence in the diagnostic evidence set and check the completeness of its "feature evidence vector-fault location-abnormal category" ternary association relationship. Evidence that is missing any association element or whose association element is invalid is marked as incomplete evidence and removed.

[0158] Redundancy check: Check if there are duplicate pieces of evidence in the diagnostic evidence set that have the same source service identifier, anomaly category, and fault location. If duplicate evidence exists, retain the evidence with the highest confidence level and remove the rest; if the confidence levels are the same, retain the evidence with the most recent diagnostic timestamp.

[0159] After successful verification, the final diagnostic evidence set is output for subsequent evidence fusion and comprehensive diagnostic result generation.

[0160] In step S5, based on the diagnostic evidence set, the location consistency, fault correlation, and confidence contribution among each piece of diagnostic evidence are calculated. Conflicting diagnostic evidence is rearranged and its credibility is corrected to generate a comprehensive diagnostic result, including:

[0161] S501: Standardized preprocessing and basic feature extraction of diagnostic evidence.

[0162] Standardized preprocessing is performed on all valid diagnostic evidence in the diagnostic evidence set to extract a basic feature set for evidence fusion calculation. The basic feature set includes seven dimensions: unique evidence identifier, source service identifier, anomaly category, fault location, original confidence level, evidence generation timestamp, and historical diagnostic accuracy of the source service. The historical diagnostic accuracy of the source service is obtained in real-time from the operational status layer of the service capability set, with a value range of [0,1], retaining four decimal places. Consistency checks are performed on the extracted basic features to ensure that all feature values ​​conform to the specifications of the industrial fault structured knowledge base; evidence that fails the check is discarded.

[0163] S502: Quantitative calculation of multidimensional evidence association.

[0164] Based on the preprocessed set of basic features, three core quantitative indicators are calculated for every two pieces of diagnostic evidence: location consistency, fault correlation, and overall similarity.

[0165] Locational consistency calculation: for any two pieces of diagnostic evidence and Consistency of parts Defined as the degree of matching between the faulty parts of two pieces of evidence; if the faulty parts of the two pieces of evidence are exactly the same, then... If the fault locations of the two pieces of evidence are related, such as "drive-end bearing" and "bearing", then If the faulty parts of the two pieces of evidence belong to the same component group, such as "drive-end bearing" and "non-drive-end bearing", then ;otherwise The inclusion relationships of fault locations and the division of component groups are pre-stored in the fault location library of the industrial fault structured knowledge base.

[0166] Fault correlation calculation: For any two pieces of diagnostic evidence and Fault correlation Defined as the degree of correlation between two categories of abnormal evidence, the formula is as follows:

[0167] ,

[0168] in, For diagnostic evidence The set of associated fault tags corresponding to the anomaly category is pre-stored in the fault tag library of the industrial fault structured knowledge base, containing all fault tags that have a causal or concurrent relationship with the anomaly category.

[0169] Comprehensive similarity calculation: for any two pieces of diagnostic evidence and Overall similarity Defined as the weighted sum of part consistency and fault correlation, the calculation formula is:

[0170] ,

[0171] in, , The weighting coefficients were quantitatively calibrated through 500 sets of multimodal fault diagnosis test cases covering gear, bearing, and motor equipment. Different weighting ratios were used by traversing the 0~1 interval with control variables. The comprehensive diagnostic accuracy and fault location deviation rate were used as quantitative evaluation indicators. The fusion output effect under all ratios was compared, and finally, 0.6 and 0.4, which had the highest diagnostic accuracy and the smallest fault location deviation, were selected as fixed weights.

[0172] S503: Calculation of confidence contribution based on evidence support.

[0173] For each piece of diagnostic evidence Calculate its confidence contribution to the entire diagnostic evidence set. The confidence contribution factor comprehensively considers the original confidence level of the evidence itself, the historical reliability of the source service, and the degree of support from other evidence. The calculation formula is:

[0174] ,

[0175] in, The total number of valid pieces of evidence in the diagnostic evidence set; For diagnostic evidence The original confidence level; For diagnostic evidence Historical diagnostic accuracy of the source service; , , The weighting coefficients are determined using the analytic hierarchy process (AHP). Multiple equipment maintenance and fault diagnosis experts scored the importance of each indicator pairwise. After consistency verification to eliminate subjective bias, the coefficients were normalized to obtain the optimal weighting ratio. This prioritizes the credibility of the evidence itself while also considering service reliability and mutual support between pieces of evidence. The confidence contribution of all diagnostic evidence is normalized so that the sum of the confidence contributions of all evidence equals 1.

[0176] S504: Identification of conflict diagnostic evidence and dynamic credibility correction.

[0177] Constructed using all valid evidence in the diagnostic evidence set as rows and columns, at any position within the matrix. The element value is the evidence calculated in step S502. With evidence Overall similarity between The position of the matrix diagonal At that time, the overall similarity of the evidence itself is fixed at 1. After completing the pairwise cross-calculation of all the evidence, a complete square matrix, namely the overall similarity matrix, is generated.

[0178] Based on the comprehensive similarity matrix, conflict diagnosis evidence is identified, and a conflict coefficient matrix is ​​constructed. , of which elements Indicates diagnostic evidence and The degree of conflict between them. For each piece of diagnostic evidence. Calculate its average conflict coefficient ,like If the evidence is deemed conflicting, it is marked as conflicting evidence. A threshold of 0.7 distinguishes highly conflicting evidence, and an average conflict coefficient exceeding this value indicates that the evidence contradicts the majority of diagnostic conclusions. If the average conflict coefficient of all evidence in the diagnostic evidence set is greater than 0.7, it is determined to be a fully conflicting evidence scenario, triggering a full-conflict fallback mechanism: abandoning the confidence contribution calculation result based on evidence similarity, switching to a historical credibility priority weighting strategy, using the historical diagnostic accuracy rate of each diagnostic service as the basic weight, and combining the historical fault clue matching degree in the diagnostic context package with the current operating condition adaptation degree for secondary weighting. This secondary weighting is divided into two layers of calculation logic: the first layer uses the historical diagnostic accuracy rate of each diagnostic service with long-term stable output as the basic benchmark weight, representing the inherent diagnostic reliability of the corresponding diagnostic service itself; the second layer introduces two correction coefficients, historical fault clue matching degree and current operating condition adaptation degree, to multiplicatively correct the benchmark weight, completing the secondary weighting operation, thereby correcting the defect that the single historical accuracy rate cannot take into account the historical fault patterns of the equipment and the real-time operating conditions, recalculating the corrected confidence contribution of each piece of evidence, and generating an evidence priority sequence after normalization to ensure the continuous progress of the evidence fusion process.

[0179] It is important to emphasize that the purpose of the above-mentioned dual-layer weighted design is as follows: In the context of full conflict, multimodal evidence is contradictory, and the credibility cannot be effectively distinguished by the similarity between the evidence. Therefore, the weight is jointly modified by three objective dimensions: long-term service calibration performance, historical failure recurrence pattern of equipment, and current failure susceptibility characteristics. This weakens the judgment distortion caused by conflict and improves the stability of diagnostic results in extreme conflict scenarios.

[0180] The specific weighting calculation method is as follows: initial weight of a single piece of evidence = historical diagnostic accuracy of diagnostic service × historical fault clue matching degree × working condition adaptability; after all evidence has completed the initial weighting calculation, all weights are subjected to global normalization processing, and the normalized value is the final corrected confidence contribution of the evidence.

[0181] The historical fault clue matching degree is set as follows: if the anomaly category of the evidence is completely consistent with the fault location and historical fault clue, the matching degree is 1.2; if it belongs to the same component group, the matching degree is 1.0; if there is no correlation, the matching degree is 0.8. The operating condition adaptability degree is set as follows: if the probability of occurrence of the fault corresponding to the evidence under the current dominant operating condition is higher than the average level, the adaptability degree is 1.1; otherwise, it is 1.0. The above gradient coefficients are calibrated based on the statistical laws of industrial faults and on-site operation and maintenance experience: a matching degree of 1.0 is set as a neutral benchmark, 1.2 and 1.1 slightly increase the weight of effective evidence that matches historical faults and adapts to the current operating condition, and 0.8 moderately reduces the weight of evidence that is not related to the equipment's historical faults, thereby widening the difference in the credibility of evidence within a reasonable range, while avoiding excessive polarization of weights that could lead to fusion distortion.

[0182] Dynamic confidence correction is performed on diagnostic evidence marked as conflicting, using the following formula:

[0183] ,

[0184] For non-conflicting evidence, its confidence contribution remains unchanged. After correction, the confidence contributions of all diagnostic evidence are normalized again. All diagnostic evidence is then rearranged from high to low according to the corrected confidence contributions to generate an evidence priority sequence.

[0185] S505: Calculation of comprehensive fault confidence based on weighted evidence fusion.

[0186] Based on the evidence priority sequence and the adjusted confidence contribution, the overall confidence level of each candidate fault hypothesis is calculated. The set of candidate fault hypotheses consists of all distinct "anomaly category-fault location" combinations from the diagnostic evidence set. For each candidate fault hypothesis... Its overall confidence level The calculation formula is:

[0187] ,

[0188] in, For all candidates, the fault hypothesis is supported. The diagnostic evidence set is used to rank all candidate fault hypotheses from highest to lowest overall confidence level, generating a sequence of candidate fault hypotheses.

[0189] S506: Validation and structured output of comprehensive diagnostic results.

[0190] The candidate fault hypothesis with the highest overall confidence is selected as the preliminary comprehensive diagnosis result, and final verification is performed by combining historical fault clues in the diagnostic context package with the current operating condition. If the preliminary comprehensive diagnosis result is consistent with the fault category and fault location in the historical fault clues and conforms to the fault occurrence pattern under the current operating condition, it is taken as the final comprehensive diagnosis result; otherwise, the candidate fault hypothesis with the second highest overall confidence is taken as the final comprehensive diagnosis result.

[0191] The final comprehensive diagnostic results are output in a structured format, including the following: comprehensive anomaly category, comprehensive fault location, comprehensive confidence level, list of supporting evidence, explanation of conflicting evidence, fault severity assessment, and recommended handling suggestions. The fault severity assessment is based on the comprehensive confidence level: a comprehensive confidence level ≥ 0.8 indicates a severe fault, 0.6 ≤ comprehensive confidence level < 0.8 indicates a moderate fault, and a comprehensive confidence level < 0.6 indicates a minor fault. Recommended handling suggestions are selected from the maintenance measures library of the industrial fault structured knowledge base based on the comprehensive anomaly category and comprehensive fault location, matching the highest priority standard handling suggestions. The final comprehensive diagnostic results are stored in the equipment management platform, updating the equipment's historical diagnostic records to provide data support for subsequent diagnoses.

[0192] refer to Figure 3 , Figure 3 A flowchart for generating comprehensive diagnostic results through evidence fusion.

[0193] This embodiment forms a diagnostic context package that characterizes the current state of the equipment, available data modes, and historical fault relationships by performing time alignment, equipment identifier matching, operating condition correlation, and fault clue extraction on the equipment operation data, multimodal detection data, and historical diagnostic records of industrial equipment during the diagnostic cycle. Furthermore, the vibration diagnostic model, image recognition model, and text analysis model are encapsulated into standardized diagnostic services according to the model context protocol. A service capability set is established, including input fields, applicable modes, fault ranges, output structures, and historical call states. A collaborative invocation strategy is generated based on the diagnostic context package and the service capability set, enabling dynamic filtering, priority ranking, and parallel invocation of different diagnostic services. Simultaneously, the anomaly categories, fault locations, confidence levels, evidence fragments, and treatment suggestions output by each diagnostic service are mapped to a unified diagnostic evidence set. Conflict identification, evidence rearrangement, and credibility correction are performed on multi-source evidence by combining location consistency, fault correlation, and confidence contribution. Finally, a comprehensive diagnostic result is generated, including a comprehensive anomaly category, comprehensive fault location, comprehensive confidence level, a list of supporting evidence, and recommended treatment suggestions. This improves the standardized access capability, on-site collaboration capability, and credibility of the diagnostic conclusions of the multimodal diagnostic model.

[0194] Example 2

[0195] This invention discloses a multimodal diagnostic collaborative system based on a model context protocol, comprising a data acquisition module, a service registration module, a policy generation module, an evidence collection module, and a fusion scheduling module.

[0196] The data acquisition module obtains equipment operation data, multimodal detection data and historical diagnostic records of industrial equipment during the diagnostic cycle. It performs time alignment, equipment identifier matching and operating status association on the equipment operation data and multimodal detection data, and extracts fault tags and maintenance clues corresponding to the current equipment from the historical diagnostic records to form a diagnostic context package.

[0197] The service registration module encapsulates the vibration diagnosis model, image recognition model, and text analysis model into diagnostic services according to the model context protocol, parses the input fields, applicable modes, fault range, output structure, and historical call status of each diagnostic service, and establishes a service capability set.

[0198] The strategy generation module selects matching diagnostic services from the service capability set based on the data modality, operating status, fault clues and diagnostic objectives in the diagnostic context package, and generates collaborative invocation strategies according to data integrity, service response status and fault correlation.

[0199] The evidence collection module sends diagnostic context packets to the corresponding diagnostic services according to the collaborative invocation strategy, receives the anomaly categories, fault locations, confidence levels, evidence fragments and handling suggestions output by each diagnostic service, and maps them into a unified format to form a diagnostic evidence set.

[0200] The fusion scheduling module calculates the location consistency, fault correlation, and confidence contribution among various diagnostic evidences based on the diagnostic evidence set, performs evidence rearrangement and confidence correction on conflicting diagnostic evidences, and generates a comprehensive diagnostic result.

[0201] refer to Figure 4 , Figure 4 This is a structural diagram of a multimodal diagnostic collaborative system based on a model context protocol.

[0202] The specific functional implementation of each module is described in the relevant content of the multimodal diagnostic collaboration method based on model context protocol in Example 1, and will not be repeated here.

[0203] The above description is merely a preferred embodiment of the present invention. The scope of protection of the present invention is not limited to the above embodiments. All technical solutions falling within the scope of the present invention's concept are within the scope of protection of the present invention. It should be noted that for those skilled in the art, any improvements and modifications made without departing from the principles of the present invention should also be considered within the scope of protection of the present invention.

Claims

1. A multimodal diagnostic collaborative method based on model context protocol, characterized in that, Includes the following steps: Acquire equipment operation data, multimodal detection data, and historical diagnostic records, and form a diagnostic context package by time alignment, equipment identifier matching, operating condition association, and fault clue extraction; The vibration diagnosis model, image recognition model, and text analysis model are encapsulated into a diagnostic service according to the model context protocol, and a service capability set is established. Based on the diagnostic context package and service capability set, select matching diagnostic services and generate collaborative invocation strategies according to data integrity, service response status and fault correlation. The diagnostic service is invoked according to the collaborative invocation strategy. The anomaly category, fault location, confidence level, evidence fragments and handling suggestions output by each diagnostic service are mapped into a unified format to form a diagnostic evidence set. Based on the diagnostic evidence set, the location consistency, fault correlation and confidence contribution of each diagnostic piece of evidence are calculated. Conflicting diagnostic evidence is rearranged and confidence is corrected to generate a comprehensive diagnostic result. The process of generating a collaborative invocation strategy includes: parsing the available modal set, dominant operating condition status, historical fault label set, and diagnostic target priority from the diagnostic context package to construct a structured diagnostic requirement feature vector; performing modal matching verification, operating condition adaptability verification, and service availability verification on the service capability set based on the structured diagnostic requirement feature vector to form a candidate service set; calculating a comprehensive matching score for the diagnostic services in the candidate service set to filter and obtain the final invocation service set; calculating a priority score based on the final invocation service set to generate a service invocation priority sequence, and generating an adaptive fully parallel collaborative invocation strategy from the service invocation priority sequence. Establish a service capability set, including: defining a service identifier layer, input / output interface layer, capability description layer, and operational status layer according to the model context protocol; encapsulating the vibration diagnosis model, image recognition model, and text analysis model into standardized diagnostic services, forming vibration diagnosis services, image recognition services, and text analysis services; performing automated parsing of diagnostic service metadata on the standardized diagnostic services to obtain an input field description table, an applicable mode set, a set of diagnosable fault labels, an output structure description table, and historical call status; and performing quantitative calibration and verification of the diagnostic service capabilities on the standardized diagnostic services, integrating metadata information and quantitative capability indicators into a service capability set. The process of generating a comprehensive diagnostic result includes: performing standardized preprocessing on valid diagnostic evidence in the diagnostic evidence set to extract a set of basic features; calculating the consistency of location, fault correlation, and comprehensive similarity among different diagnostic evidence based on the set of basic features to obtain a comprehensive similarity matrix; calculating the confidence contribution of diagnostic evidence based on the comprehensive similarity matrix, identifying conflicting diagnostic evidence and performing dynamic confidence correction to generate an evidence priority sequence; constructing candidate fault hypotheses from different combinations of anomaly categories and fault locations in the diagnostic evidence set, calculating the comprehensive confidence based on the evidence priority sequence, generating a candidate fault hypothesis sequence, and outputting the comprehensive diagnostic result after verification by historical fault clues and current operating conditions. Dynamic credibility correction is performed on conflicting diagnostic evidence to generate an evidence priority sequence. This includes: calculating the consistency of diagnostic evidence based on the inclusion relationship of fault locations and component group division in the industrial fault structured knowledge base; calculating the fault correlation between diagnostic evidence based on the set of associated fault labels corresponding to anomaly categories in the industrial fault structured knowledge base; weighting the consistency of locations and the fault correlation to obtain the comprehensive similarity between diagnostic evidence; constructing an evidence conflict coefficient matrix based on the comprehensive similarity, calculating the average conflict coefficient and identifying conflicting diagnostic evidence; performing dynamic credibility correction on conflicting diagnostic evidence; and generating an evidence priority sequence.

2. The multimodal diagnostic collaborative method based on model context protocol as described in claim 1, characterized in that, The diagnostic context package includes: Simultaneous acquisition and standardization conversion of equipment operation data, multimodal detection data, and historical diagnostic records are performed to obtain standardized data; Based on a unified time benchmark, time alignment is performed on standardized data to generate time-synchronized multimodal data sequences. Based on a global device coding system, device identifier matching is performed on the multimodal data sequences to remove data with abnormal identifiers. The operating condition labels are determined by clustering based on operating parameters. The operating condition labels are then associated with the multimodal data sequence at each time point to form a multimodal dataset with operating condition labels. Fault labels, fault occurrence time, fault location, fault severity, fault cause, and maintenance measures are extracted from historical diagnostic records to form a structured fault history database. This database is then integrated with a multimodal dataset labeled with operating conditions to form a diagnostic context package.

3. The multimodal diagnostic collaborative method based on model context protocol as described in claim 1, characterized in that, Diagnostic services include: Vibration diagnostic services preprocess vibration acceleration signals, extract time-domain and frequency-domain features, construct feature vectors and input them into a random forest classifier, and output anomaly category, fault location, confidence level, evidence fragments and treatment suggestions; The image recognition service performs preprocessing on infrared thermal imaging images and visible light images, uses feature extraction and template matching to identify suspected fault areas, and performs secondary confirmation through a lightweight convolutional neural network, outputting anomaly category, fault location, confidence level, evidence fragments, and handling suggestions; The text analysis service preprocesses structured text data and uses a combination of exact and fuzzy matching to match it with an industrial fault structured knowledge base, outputting anomaly category, fault location, confidence level, evidence fragments, and handling suggestions.

4. The multimodal diagnostic collaborative method based on model context protocol as described in claim 1, characterized in that, Forming a diagnostic evidence set includes: Receive the raw output data returned by the diagnostic service, perform field integrity checks, field validity checks, and logical consistency checks on the raw output data, and retain the valid raw output; Establish a structured diagnostic evidence data structure compatible with the industrial fault structured knowledge base, and map the effective raw output into diagnostic evidence containing a unique evidence identifier, source service identifier, diagnostic timestamp, anomaly category, fault location, confidence level, evidence fragments, and handling suggestions; Multimodal differentiated evidence fragment structure extraction is performed on diagnostic evidence. The extraction results are written into the evidence fragment field, and empty set verification, correlation verification and redundancy verification are performed to form a diagnostic evidence set.

5. The multimodal diagnostic collaborative method based on model context protocol as described in claim 4, characterized in that, Perform multimodal differential evidence fragment extraction, including: Extract time-domain feature values, frequency-domain feature values, characteristic frequency peaks and their corresponding amplitudes from the evidence fragments output by the vibration diagnostic service to construct a vibration feature evidence vector; Extract feature point coordinates, descriptors, template matching similarity, bounding box coordinates of suspected fault areas, and network feature vectors from the evidence fragments output by the image recognition service to construct image feature evidence vectors; Extract the text content, start position, end position, and matching similarity value of matching keywords from the evidence fragments output by the text analysis service to construct a text feature evidence vector; The feature evidence vector is associated and bound with the corresponding anomaly category and fault location to establish a ternary association relationship between the feature evidence vector, the fault location, and the anomaly category.

6. A multimodal diagnostic collaborative system based on a model context protocol, used to implement the multimodal diagnostic collaborative method based on a model context protocol as described in any one of claims 1-5, characterized in that, It includes a data acquisition module, a service registration module, a policy generation module, an evidence collection module, and a fusion scheduling module. The data acquisition module obtains equipment operation data, multimodal detection data, and historical diagnostic records. After time alignment, equipment identifier matching, operating condition association, and fault clue extraction, a diagnostic context package is formed. The service registration module encapsulates the vibration diagnosis model, image recognition model, and text analysis model into a diagnostic service according to the model context protocol, and establishes a service capability set. The strategy generation module filters matching diagnostic services based on the diagnostic context package and service capability set, and generates collaborative invocation strategies according to data integrity, service response status and fault correlation. The evidence collection module invokes the diagnostic service according to the collaborative invocation strategy, and receives the anomaly type, fault location, confidence level, evidence fragments and handling suggestions output by each diagnostic service and maps them into a unified format to form a diagnostic evidence set; The fusion scheduling module calculates the location consistency, fault correlation, and confidence contribution among various diagnostic evidences based on the diagnostic evidence set, performs evidence rearrangement and confidence correction on conflicting diagnostic evidences, and generates a comprehensive diagnostic result.

Images