An equipment intelligent support system based on an offline large model

Abstract

The present invention discloses an equipment intelligent support system based on an offline large model, which relates to the field of equipment management and support, and includes: a collection module, which collects surface defect images and time-series signals of equipment through drones, collects text logs, processes the defect images, time-series signals and text logs respectively, and generates a multi-modal data set; a feature fusion module, which is used to extract features from the multi-modal data set, fuse the three types of features, and generate multi-modal features; an update module, which is used to extract fault triples from semantic vectors, construct a Bayesian causal graph according to the fault triples, dynamically update the node confidence, and generate a knowledge graph; a maintenance strategy module, which is used to match the multi-modal features with the knowledge graph to locate the root cause of the fault; a lightweight model deployment module, which is used to pre-train a lightweight model on an edge server, accelerate parameter aggregation using a quantum annealing algorithm, optimize the global large model according to the aggregated parameters, and generate a strategy in combination with reinforcement learning.

Description

Technical Field

[0001] The present invention relates to the field of equipment management and support, and particularly to an equipment intelligent support system based on an offline large model. Background Art

[0002] In recent years, artificial intelligence technology has made remarkable progress in the field of equipment intelligent support. The intelligent support system based on large models realizes equipment status monitoring and fault diagnosis by integrating multi-modal data, and has gradually become a research hotspot. For example, industrial large models have shown potential in the fields of equipment health management and fault prediction by combining large-scale parameters with multi-modal data. In the prior art, multi-modal data fusion methods can improve the feature representation ability, and the combination of Bayesian networks and spiking neural networks (SNNs) provides a new idea for causal reasoning and dynamic knowledge update. In addition, federated learning and quantum annealing algorithms are used for edge server model optimization to solve the problems of data privacy and limited computing power. The multi-source data acquisition technology of drones combined with infrared cameras and vibration sensors, as well as the SLAM-based 3D modeling method, further improve the accuracy and spatial correlation of equipment surface defect detection.

[0003] However, the prior art still faces a series of challenges. First, traditional methods rely on static rules in cross-modal alignment and are difficult to handle the dynamic association problem of heterogeneous data in complex industrial scenarios. For example, the lack of an adaptive mechanism for high-frequency vibration noise filtering and wavelet decomposition layer selection results in incomplete retention of fault feature frequency bands. Summary of the Invention

[0004] In view of the above existing problems, the present invention is proposed.

[0005] Therefore, the present invention provides an equipment intelligent support system based on an offline large model to solve the problem that traditional methods rely on static rules in cross-modal alignment and are difficult to handle the dynamic association problem of heterogeneous data in complex industrial scenarios.

[0006] To solve the above technical problems, the present invention provides the following technical solutions:

[0007] The present invention provides an equipment intelligent support system based on an offline large model, which includes:

[0008] An acquisition module, a feature fusion module, an update module, a maintenance strategy module, and a lightweight model deployment module:

[0009] The acquisition module collects surface defect images and time series signals of the equipment through a drone, and simultaneously collects text logs, and processes the defect images, time series signals, and text logs respectively to generate a multi-modal data set;

[0010] The feature fusion module is used to extract features from the multi-modal dataset to obtain image features, temporal features, and text features, and fuse the three types of features to generate multi-modal features;

[0011] The update module is used to extract fault triples from the semantic vectors, construct a Bayesian causal graph based on the fault triples, input the multi-modal features into a spiking neural network, dynamically update the node confidence, and generate a knowledge graph;

[0012] The maintenance strategy module is used to match the multi-modal features with the knowledge graph to locate the root cause of the fault;

[0013] The lightweight model deployment module is used to pre-train a lightweight model on an edge server, use a quantum annealing algorithm to accelerate the aggregation of lightweight model parameters, generate a global large model based on the aggregated lightweight model parameters, generate a strategy by combining reinforcement learning, and select the optimal maintenance strategy by comprehensively considering the cost, downtime loss, and spare part inventory.

[0014] As a preferred solution of the equipment intelligent guarantee system based on the offline large model of the present invention, wherein: the surface defect image refers to a crack image and a corrosion image;

[0015] An infrared camera and a three-axis vibration sensor are mounted on the unmanned aerial vehicle to collect defect images and temporal signals, and text logs are obtained in real time through a PLC interface.

[0016] As a preferred solution of the equipment intelligent guarantee system based on the offline large model of the present invention, wherein: the processing of the defect image, temporal signal, and text log respectively to generate a multi-modal dataset is specifically as follows:

[0017] Separate the illumination component and the reflection component of the defect image, combine the illumination mean template of the historical fault image to generate an optimized illumination component, and enhance the defect image according to the optimized illumination component and the reflection component;

[0018] Adopt wavelet basis decomposition to extract low-frequency fundamental frequency and high-frequency abnormal features;

[0019] Use BERT to identify the component name and fault type, and combine the regular expression matching logic chain to generate a structured triple;

[0020] Map the pixel coordinates of the enhanced defect image to the device 3D model through SLAM technology, and associate the vibration signal with the specific part;

[0021] Package the enhanced defect image and coordinates, the filtered mean value and the associated part label, and the semantic vector of the fault triple into the HDF5 format to generate a multi-modal dataset.

[0022] As a preferred solution of the equipment intelligent guarantee system based on the offline large model described in the present invention, wherein: the multi-modal data set is subjected to feature extraction to obtain image features, temporal features and text features, and the specific steps are as follows:

[0023] Scale the enhanced defect image, and extract the high-dimensional features of the penultimate fully connected layer through the ResNet model to obtain image features;

[0024] Obtain temporal features by encoding the temporal dependence relationship of temporal signals through LSTM;

[0025] Map the text log to a semantic vector through BERT embedding and extract the fault triple.

[0026] As a preferred solution of the equipment intelligent guarantee system based on the offline large model described in the present invention, wherein: the fusion of the three features to generate multi-modal features means mapping the image features, temporal features and text features to the same dimension through dynamic weight attention.

[0027] As a preferred solution of the equipment intelligent guarantee system based on the offline large model described in the present invention, wherein: the extraction of the fault triple from the semantic vector and the construction of the Bayesian causal graph according to the fault triple, the specific steps are as follows:

[0028] Use the pre-trained Transformer decoder to parse the BERT semantic vector and extract the structured triple;

[0029] Take the triple elements as nodes, construct directed edges according to the causal chain to form the Bayesian network topology, and fill the prior distribution of the nodes based on the probability of the cause leading to the phenomenon and the success rate of the solution to repair the cause statistically from the historical maintenance data, to obtain the Bayesian causal graph.

[0030] As a preferred solution of the equipment intelligent guarantee system based on the offline large model described in the present invention, wherein: inputting the multi-modal features into the spiking neural network to dynamically update the node confidence and generate a knowledge graph, specifically:

[0031] Convert the multi-modal features into a pulse sequence, set a global threshold to trigger a pulse signal, and map the feature intensity to the pulse interval;

[0032] The Bayesian network nodes correspond to SNN neuron clusters, and the spike-time-dependent plasticity rule STDP is adopted;

[0033] The node confidence is updated driven by the pulse frequency, and finally a knowledge graph with weighted labels is generated.

[0034] As a preferred solution of the equipment intelligent support system based on the offline large model of the present invention, wherein: matching the multi-modal features with the knowledge graph to locate the root cause of the fault, specifically:

[0035] Map the multi-modal fusion features to the knowledge graph node space through a fully connected layer, generate a node feature contribution vector, calculate the cosine similarity between the feature contribution vector and the knowledge graph nodes, and screen the candidate node set whose similarity meets the actual requirements;

[0036] Map the multi-modal features to evidence vectors, and update the Bayesian network conditional probability through a non-linear evidence fusion function;

[0037] Starting from the symptom nodes, traverse the Bayesian network causal path in reverse, calculate the comprehensive confidence of the path, fuse the node confidence and the updated conditional probability, sort the candidate root causes in descending order of the comprehensive confidence, and select the node with the highest confidence as the root cause of the fault.

[0038] As a preferred solution of the equipment intelligent support system based on the offline large model of the present invention, wherein: pre-training a lightweight model on the edge server, using the quantum annealing algorithm to accelerate parameter aggregation, generating a global large model according to the aggregated parameters, and generating a strategy in combination with reinforcement learning, specifically:

[0039] Adopt the MobileNetV3-Small architecture, and compress the ResNet parameter quantity proportionally through depthwise separable convolution and channel compression technology;

[0040] Based on the quantum annealing algorithm, transform the problem of aggregating the local model parameters of the edge device into a QUBO model, and model the node selection strategy through binary variables.

[0041] As a preferred solution of the equipment intelligent support system based on the offline large model of the present invention, wherein: selecting the optimal maintenance strategy by comprehensively considering the cost, downtime loss and spare part inventory, specifically:

[0042] Obtain the maintenance cost through the historical equipment maintenance records, obtain the downtime through the text logs, obtain the spare part inventory through the purchase orders, and obtain the severity of the fault according to the historical fault records;

[0043] Construct a state space based on the maintenance cost, downtime, spare part inventory and fault severity, set an action space based on immediate repair, delayed repair and replacement of spare parts, and train a policy network in combination with the DDPG algorithm to output a Pareto optimal maintenance decision;

[0044] Take BERT-wwm as the teacher model and MobileNetV3-Tiny as the student model, align the soft label distribution through the KL divergence loss, and compress the parameter quantity proportionally;

[0045] Convert the lightweight model into a PLC-executable function block, receive sensor data in real time, upload the real-time data to the edge server via the IP protocol, and trigger incremental model updates.

[0046] The edge server periodically fine-tunes the global model with new data, re-distills and generates a lightweight version, and encrypts and distributes it to the PLC. If the accuracy of the validation set of the updated model decreases, it will automatically roll back to the old version and issue an alarm.

[0047] The beneficial effects of the present invention are as follows: The present invention collects surface defect images, vibration time series signals, and operation logs of equipment through multi-modal collaboration of unmanned aerial vehicles, realizes spatio-temporal alignment by combining dynamic time warping and SLAM technology, and enhances the distinguishability of defect features by using the illumination component optimization algorithm, solving the problems of isolation of traditional single-modal data and noise interference; fuses image, time series, and text features through a dynamic weight attention mechanism, constructs a unified representation space, combines the dynamic collaboration mechanism of Bayesian causal graphs and spiking neural networks, and uses spike trigger rules to update node confidence and causal strength in real time, significantly improving the real-time performance and accuracy of complex fault reasoning; optimizes the federated learning parameter aggregation path with the quantum annealing algorithm, screens high-value edge nodes to reduce communication costs, combines reinforcement learning strategies to balance maintenance costs, downtime, and spare part inventory, and generates multi-objective optimal decisions; realizes efficient reasoning on the edge server through knowledge distillation and lightweight model deployment, and combines incremental updates and exception rollback mechanisms to ensure model stability. Description of the Drawings

[0048] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for description in the embodiments will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0049] Figure 1 It is the flowchart of the equipment intelligent guarantee system based on the offline large model in this embodiment.

[0050] Figure 2 It is the schematic diagram of dynamic feature fusion and Bayesian causal graph in this embodiment.

[0051] Figure 3 It is the architecture diagram of quantum annealing-driven federated learning parameter aggregation in this embodiment.

[0052] Figure 4 It is the architecture diagram of the equipment intelligent guarantee system based on the offline large model in this embodiment. Detailed Embodiments

[0053] To make the above objects, features, and advantages of the present invention more apparent and understandable, the following provides a detailed description of the specific embodiments of the present invention in conjunction with the accompanying drawings of the specification.

[0054] In the following description, many specific details are set forth to facilitate a full understanding of the present invention. However, the present invention may also be implemented in other ways different from those described herein. Those skilled in the art may make similar extensions without departing from the connotation of the present invention. Therefore, the present invention is not limited by the specific embodiments disclosed below.

[0055] Secondly, the so-called "one embodiment" or "embodiment" herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation manner of the present invention. The appearances of "in one embodiment" in different places in this specification do not all refer to the same embodiment, nor are they separate or selectively exclusive embodiments from other embodiments.

[0056] This embodiment, referring to Figures 1 to 4 , is the first embodiment of the present invention. This embodiment provides an equipment intelligent guarantee system based on an offline large model, including the following steps:

[0057] A collection module, a feature fusion module, an update module, a maintenance strategy module, and a lightweight model deployment module:

[0058] The collection module collects surface defect images and time series signals of the equipment through a drone, and simultaneously collects text logs. The defect images, time series signals, and text logs are processed respectively to generate a multi-modal data set;

[0059] The surface defect images refer to crack images and corrosion images;

[0060] An HD infrared camera and a three-axis vibration sensor are mounted on the drone. The time stamps of the camera and the vibration sensor are synchronized through the GPS signal. The vibration sensor is calibrated for sensitivity using the ISO 16063-21 standard to ensure that the measurement range covers 0-10 kHz. The camera corrects distortion through the checkerboard calibration method and calibrates the white balance using a gray card to eliminate light interference;

[0061] The drone flies according to a preset route, at a distance of 2-5 m from the equipment surface, takes images of the surface defect area of the equipment, dynamically adjusts the height through a laser rangefinder to ensure that the defect images cover key parts (gearbox, bearing housing), continuously collects the acceleration signals of the key parts of the equipment (such as gearbox, bearing housing) through the vibration sensor, and records the time stamps;

[0062] The text logs (fault codes and maintenance records) are obtained in real time through the PLC interface of Modbus TCP / IP, which is suitable for Ethernet-based communication and supports real-time data transmission;

[0063] Separate the illumination component and the reflection component of the defect image, and combine the illumination mean template (illumination intensity of the corroded area) of the historical fault image to generate an optimized illumination component. The expression is:

[0064] ;

[0065] where represents the new illumination intensity at the pixel position , represents the average value of the historical illumination template, which is based on the illumination statistical value of the same area in the historical fault image, and the unit is lux (lux), represents the original illumination component at the pixel position , represents that the contribution ratio of the historical illumination template is 70%, represents that the contribution ratio of the current illumination component is 30%;

[0066] Enhance the defect image according to the optimized illumination component and reflection component. The expression is:

[0067] ;

[0068] where represents the reflection component, which represents the crack texture and corrosion texture on the object surface and is obtained by decomposing the original image. The value range is [0,1]. The closer the value of is to 1, the stronger the reflection is, and the closer it is to 0, the stronger the absorption is.

[0069] Select the sym8 wavelet basis for 5-layer decomposition, and each layer of decomposition generates an approximation coefficient and a detail coefficient;

[0070] The approximation coefficient A represents the low-frequency component of the signal (the fundamental frequency vibration when the device is running normally);

[0071] The detail coefficient D represents the high-frequency component of the signal (abnormal vibration, noise);

[0072] Use the SUREShrink threshold method to filter out the noise with a frequency greater than 15 kHz for the high-frequency coefficients and retain the fault feature frequency band;

[0073] Use the BERT model to identify the component names and fault types in the text log. The fault types include cracks, deformations, and corrosion, and the component names include;

[0074] In the analysis of the fault log, regular expressions are used to extract phenomena, causes, and measures from unstructured text. By matching the key event patterns with regular expressions, phenomena → causes → measures;

[0075] Set the phenomenon - cause rule: If the text contains "abnormal vibration" and the context has "insufficient lubrication", then generate a causal chain: 〈abnormal vibration → insufficient lubrication〉;

[0076] If the text contains "too high temperature" and the context has "bearing wear", then generate a causal chain: 〈too high temperature → bearing wear〉;

[0077] Set the cause - solution rule: If the cause = "insufficient lubrication", then the solution = "supplement lubricant";

[0078] If the cause = "bearing wear", then the solution = "replace bearing";

[0079] Input an example and then output triples;

[0080] Adopt the dynamic time warping (DTW) algorithm to align the vibration signal with the image frame according to the timestamp, ensuring the data association at the same moment (such as abnormal vibration corresponding to the crack image);

[0081] Map the pixel coordinates of the enhanced defect image to the 3D model of the device through SLAM technology, associating the vibration signal with the specific part;

[0082] Bind the vibration signal to the corresponding part in the 3D model according to the installation position of the vibration sensor;

[0083] Package the enhanced defect image and its 3D coordinates, the filtered mean value, the associated part label, and the semantic vector of the fault triple into the HDF5 format to generate a multi - modal dataset.

[0084] The feature fusion module is used to extract features from the multi - modal dataset, obtain image features, temporal features, and text features, and fuse the three features to generate multi - modal features;

[0085] Scale the enhanced defect image and normalize it to the range of [0, 1], and extract the high - dimensional features of the penultimate fully - connected layer through the ResNet model to obtain image features;

[0086] Obtain temporal features by encoding the temporal dependence relationship of the temporal signal through LSTM. Specifically: Segment the filtered vibration signal according to the time window (1 second / segment), each segment contains several sampling points, extract the temporal features (mean, variance, peak value) and frequency - domain features (FFT main frequency, harmonic energy ratio) of each segment, input the feature sequence into a bidirectional LSTM network to capture the forward and backward dependence relationships, and obtain temporal features;

[0087] Map text logs to semantic vectors through BERT embedding and extract fault triples, specifically: tokenize the structured logs, remove stop words, retain entities (component names, fault types), use BERT-wwm (whole word masking pre-trained student model) to encode the fault triples, optimize the semantic space through contrastive learning loss to make the vector distances of similar fault triples closer, and output text features;

[0088] Map the image features, temporal features, and text features to the same dimension (256 dimensions) through dynamic weight attention, and the expression is:

[0089] ;

[0090] Among them, represents the fused multi-modal feature vector, represents the index variable of the image features, temporal features, and text features, takes values of 1, 2, and 3, corresponding to the image features, temporal features, and text features respectively, represents the th attention weight of the feature, represents the th feature vector mapped to 256 dimensions.

[0091] Update module, used to extract fault triples from the semantic vectors, construct a Bayesian causal graph according to the fault triples, input the multi-modal features into a spiking neural network, dynamically update the node confidence, and generate a knowledge graph;

[0092] Through a pre-trained Transformer architecture triple decoder, parse structured triples (phenomenon → cause → solution) from the semantic vectors generated by BERT, such as 〈abnormal vibration → insufficient lubrication → replenish lubricant〉, input the semantic vectors to obtain fault triples;

[0093] Take the phenomenon, cause, and solution of the fault triples as nodes respectively, and establish directed edges according to the causal chain (phenomenon → cause → solution). The Bayesian network structure corresponding to the triple 〈abnormal vibration, insufficient lubrication, replenish lubricant〉 is: insufficient lubrication → abnormal vibration → replenish lubricant;

[0094] Statistically calculate the conditional probability of the cause leading to the phenomenon and the repair success rate of the solution for the cause from historical maintenance data, that is, the prior probability;

[0095] For each cause node, fill in the probability distribution of the phenomenon it causes according to historical data, and for each solution node, fill in the probability distribution of repairing the cause to generate a Bayesian causal graph;

[0096] Convert each dimensional value of the multi-modal feature into a pulse sequence, uniformly set a global threshold, trigger a pulse signal when the feature value exceeds the global threshold, and map the feature intensity to the pulse time interval through pulse time encoding. The larger the original feature intensity value of the multi-modal feature, the shorter the pulse interval;

[0097] Each node in the Bayesian causal graph corresponds to a cluster of SNN neurons;

[0098] Adopt the spike-timing-dependent plasticity rule STDP to dynamically adjust the synaptic weight according to the time difference between the spikes of the front and back neurons;

[0099] If the spike of the parent node is earlier than that of the child node, then long-term potentiation is triggered to enhance the synaptic weight. The expression is:

[0100] ;

[0101] Among them, represents the change amount of the synaptic weight, represents the learning rate, represents the base of the natural logarithm, represents the time difference between the spikes of the front and back neurons, with the unit of millisecond (ms), represents time, is the time constant, can be set to 10 ms (millisecond), which balances the sensitivity and stability of weight adjustment in the real-time inference scenario of industrial equipment and controls the weight decay speed, The smaller it is, the more sensitive the weight adjustment is to the time difference and the faster the decay;

[0102] If the spike of the parent node is later than that of the child node, then long-term depression is triggered to weaken the synaptic weight. The expression is:

[0103] ;

[0104] Dynamically adjust the causal strength of the edge based on the conditional probability of the Bayesian network and the number of pulse triggers, and generate a knowledge graph with weighted labels. The size of the node represents the confidence level, and the thickness of the edge represents the causal strength;

[0105] The node confidence level is determined by the pulse frequency. The higher the pulse frequency, the greater the increase in the confidence level. Dynamically adjust the node confidence level according to the pulse frequency. The expression is:

[0106] ;

[0107] Among them, represents the node at time after update of the confidence level, represents the node at time The updated confidence level represents the gain coefficient, with a value range of [0, 1]. The larger the gain coefficient value, the more significant the impact of the pulse frequency on the confidence level represents the node The pulse frequency of represents the target maximum pulse frequency

[0108] The maintenance strategy module is used to match the multimodal features with the knowledge graph to locate the root cause of the fault

[0109] Map the multimodal fusion features to the knowledge graph node space through a fully connected layer to generate the feature contribution vector of each node. The expression is

[0110] ;

[0111] Among them represents the feature contribution vector of node represents the index variable of the node is the activation function is the weight matrix of the node mapping layer, which is optimized in the training stage through the backpropagation algorithm and is used to map the multimodal features to the knowledge graph node space represents the multimodal features represents the bias term of the node mapping layer

[0112] Calculate the cosine similarity between the node feature contribution vector and all nodes in the knowledge graph, and filter out the candidate node set with similarity > 0.7

[0113] Map the multimodal features to the evidence vector , and the expression is

[0114] ;

[0115] Among them represents the evidence vector represents the proportion of the crack area, indicating the support strength of the image feature for the current causal relationship. The value range is [0, 1], and the closer to 1, the stronger the image evidence represents the proportion of abnormal vibration energy, indicating the support strength of the vibration signal feature for the causal relationship. The value range is [0, 1], and the closer to 1, the stronger the time series evidence represents the triple semantic matching degree, with a value range of [0, 1]. The closer to 1, the higher the matching degree between the text description and the current fault

[0116] Use the non-linear evidence fusion function to update the conditional probability of the Bayesian network. The expression is ​

[0117] ;

[0118] Among them, represents the updated conditional probability, which is the new probability value of "cause X leads to phenomenon Y" supported by the evidence vector E. represents the historically statistical conditional probability, which is the prior probability of "cause X leads to phenomenon Y" obtained based on historical maintenance data. , and respectively represent the influence weights of the image modality, temporal modality, and text modality on the causal relationship, which are obtained through training with historical data. represents the base of the natural logarithm;

[0119] Starting from the phenomenon node (such as "abnormal vibration"), traverse all causal paths backward along the Bayesian network, and calculate the comprehensive confidence of each causal path. The expression is:

[0120] ;

[0121] Among them, represents the comprehensive confidence, represents the number of nodes, represents the index variable of the nodes in the path, represents the th node's confidence, represents the updated conditional probability, represents the th node in the path, representing a fault phenomenon, cause, or solution, represents the th node in the path;

[0122] Arrange the candidate root causes in descending order of the comprehensive confidence, and select the first-ranked comprehensive confidence as the fault root cause.

[0123] The lightweight model deployment module is used to pre-train a lightweight model on the edge server, accelerate the aggregation of lightweight model parameters using the quantum annealing algorithm, generate a global large model based on the aggregated lightweight model parameters, generate a strategy by combining reinforcement learning, and select the optimal maintenance strategy by comprehensively considering the cost, downtime loss, and spare parts inventory;

[0124] Adopt MobileNetV3-Small as the basic architecture, compress the number of parameters of ResNet to 1 / 10 through depthwise separable convolution and channel compression technology, and enhance the feature extraction ability through adaptive attention;

[0125] Obtain the maintenance cost through the historical equipment maintenance records, obtain the downtime through the text logs, obtain the spare parts inventory through the purchase orders, and obtain the fault severity according to the historical fault records;

[0126] Use the output probability distribution of the student model as soft labels, guide the lightweight model training through the divergence loss, and retain more than 95% semantic understanding accuracy;

[0127] On the edge device carried by the drone, perform local training on the multi-modal features. The optimization objective is to minimize the cross-entropy loss. By comparing the true distribution with the predicted distribution, measure the prediction error of the student model for the multi-classification task. The expression is:

[0128] ;

[0129] Among them, represents the edge supervision loss function, represents the total number of categories, represents the index of the category, represents the one-hot encoding of the true label. When the category is the true category = 1, otherwise = 0, represents the predicted probability, represents taking the natural logarithm of the predicted probability, represents the regularization coefficient, represents the set of trainable parameters of the student model;

[0130] Transform the parameter aggregation problem of federated learning into a QUBO model that can be solved by quantum annealing. Specifically:

[0131] Set binary variables (0 or 1) for each edge device (node). For example, if a node participates in this parameter aggregation, it is marked as 1, otherwise it is 0. Then calculate the Euclidean distance between the model parameters of different nodes. The greater the difference between the node pairs, the higher the penalty weight is given in the optimization objective;

[0132] Assign weights according to the amount of local data of the node. Nodes with high data quality are given positive weights in the objective function, and vice versa;

[0133] Integrate the parameter difference and data quality into the Hamiltonian. If a node has high data quality and small parameter differences from other nodes, the binary variable corresponding to the edge server will be preferentially selected; otherwise, it will be suppressed;

[0134] Set the communication cost limit data of federated learning, which can be achieved by adding a penalty term to the Hamiltonian. If the total number of selected nodes exceeds the limit number, the value of the energy function will increase significantly, forcing the quantum annealer to avoid such solutions;

[0135] Map the binary variables of each node to the qubits of the quantum annealing machine to form a physical qubit chain;

[0136] Through the physical annealing process of the quantum annealing machine, the Hamiltonian automatically converges to the lowest energy state, corresponding to the optimal node selection combination. For example, the annealing machine outputs a set of binary variables within milliseconds, indicating which nodes should participate in the current aggregation;

[0137] Use the D-Wave 2000Q quantum annealing machine, set the annealing time, solve for the optimal parameter combination, and the output is the global model parameter, with the expression:

[0138] ;

[0139] where, represents the global model parameter generated after the server in federated learning aggregates all the local model parameters of the participating clients, represents traversing the clients in all client indices , represents the local model parameter of the th client, updated through local data training, and represents the set of clients participating in the current round of federated learning, represents the number of clients participating in the aggregation;

[0140] Collect maintenance costs, downtime duration, spare part inventory, and failure severity (mapped by the confidence of the Bayesian causal graph), and the set generates the state space;

[0141] Set the action space, and the action space includes immediate repair, delayed repair, and replacement of spare parts;

[0142] Construct a reward function based on the state space, with the expression:

[0143] ;

[0144] where, represents the reward value at time , , , and respectively represent the priorities of maintenance costs, downtime duration, spare part inventory, and failure severity, represents the normalized maintenance cost, obtained by dividing the actual maintenance cost by the preset maximum value, represents the downtime duration, represents the spare part inventory percentage, represents the failure severity;

[0145] Using Deep Deterministic Policy Gradient (DDPG), the Actor network outputs continuous actions, and the Critic network evaluates the Q-value. After 10,000 rounds of training, the policy converges to the Pareto front. The generated policies are sorted in descending order according to the Q-value, and the policy with the largest Q-value is selected as the optimal maintenance policy;

[0146] Using the pre-trained global large model BERT-wwm as the "teacher", inputting multi-modal data (images, vibration signals, text logs), and outputting the probability distribution of fault classification (soft labels);

[0147] Selecting a lightweight architecture (MobileNetV3-Tiny), inputting the same data, and adjusting the parameters to imitate the decision-making logic of the teacher model by comparing the soft labels of the teacher model and its own output;

[0148] Removing redundant neuron connections in the student model, retaining the core feature extraction layer, compressing the number of parameters to 1 / 50 of the original model, and verifying the classification accuracy of the student model on the historical dataset to ensure that the accuracy loss ≤ 3% (for example, if the accuracy of the teacher model is 95%, the student model ≥ 92%);

[0149] Converting the student model into a binary format recognizable by the PLC and encapsulating it into an independent function block to support calling through the PLC programming language;

[0150] Receiving defect images, time series signals, and text logs connected to the PLC in real time through the IP protocol, and normalizing the input data and cropping the image to the key area at the PLC end;

[0151] The PLC regularly packs and uploads real-time data to the edge server;

[0152] The edge server fine-tunes the global large model with new data, updates the fault classification rules (bearing wear judgment threshold), and regenerates a new version of the student model through knowledge distillation;

[0153] Encrypting and pushing the updated student model to the PLC to overwrite the old version;

[0154] If the accuracy of the updated student model on the validation set drops > 5%, automatically trigger a rollback, restore to the previous stable version, and issue an alarm;

[0155] Recording the time from data input to output instruction of each student model, and triggering an alarm when it exceeds 50 ms;

[0156] Monitoring the PLC memory usage rate, and terminating low-priority tasks (such as log storage) when it exceeds 80%.

[0157] In summary, the present invention collects surface defect images, vibration time series signals, and operation logs through multi-modal collaboration of unmanned aerial vehicles, realizes spatio-temporal alignment by combining dynamic time warping and SLAM technology, and enhances the recognizability of defect features using the illumination component optimization algorithm, solving the problems of isolated traditional single-modal data and noise interference; fuses image, time series, and text features through a dynamic weight attention mechanism to construct a unified representation space, combines the dynamic collaboration mechanism of Bayesian causal graphs and spiking neural networks, and uses spike triggering rules to update node confidence and causal strength in real time, significantly improving the real-time performance and accuracy of complex fault reasoning; optimizes the federated learning parameter aggregation path using the quantum annealing algorithm, screens high-value edge nodes to reduce communication costs, combines reinforcement learning strategies to balance maintenance costs, downtime, and spare part inventory, and generates multi-objective optimal decisions; realizes efficient inference on edge servers through knowledge distillation and lightweight model deployment, and combines incremental update and exception rollback mechanisms to ensure model stability.

[0158] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit them. Although the present invention has been described in detail with reference to the preferred embodiments, those of ordinary skill in the art should understand that the technical solutions of the present invention can be modified or equivalently replaced without departing from the spirit and scope of the technical solutions of the present invention, and they should all be covered by the scope of the claims of the present invention.

Claims

1. An equipment intelligent support system based on an offline large model, characterized in that: Including: a data collection module, a feature fusion module, an update module, a maintenance strategy module, and a lightweight model deployment module: The data collection module collects surface defect images and time series signals of the device through a drone, and simultaneously collects text logs, and processes the defect images, time series signals, and text logs respectively to generate a multi-modal data set; The feature fusion module is used to extract features from the multi-modal data set to obtain image features, time series features, and text features, and fuse the three features to generate multi-modal features; The update module is used to extract fault triples from the semantic vectors, construct a Bayesian causal graph according to the fault triples, input the multi-modal features into a spiking neural network, dynamically update the node confidence, and generate a knowledge graph; The maintenance strategy module is used to match the multi-modal features with the knowledge graph to locate the root cause of the fault; The lightweight model deployment module is used to pre-train a lightweight model on an edge server, use a quantum annealing algorithm to accelerate the aggregation of lightweight model parameters, optimize the global large model according to the aggregated lightweight model parameters, generate a strategy by combining reinforcement learning, and select the optimal maintenance strategy by synthesizing costs, downtime losses, and spare part inventories; Processing the defect images, time series signals, and text logs respectively to generate a multi-modal data set, specifically: Separating the illumination component and the reflection component of the defect image, combining the illumination mean template of historical fault images to generate an optimized illumination component, and enhancing the defect image according to the optimized illumination component and the reflection component; Adopting wavelet basis decomposition to extract low-frequency fundamental frequency and high-frequency abnormal features; Using BERT to identify component names and fault types, and combining regular expression matching logic chains to generate structured triples; Mapping the pixel coordinates of the enhanced defect image to the device 3D model through SLAM technology, and associating vibration signals with specific parts; Encapsulating the enhanced defect image and coordinates, the filtered mean value, the associated part labels, and the semantic vectors of the fault triples into the HDF5 format to generate a multi-modal data set; The steps for extracting image features, time series features, and text features from the multi-modal data set are as follows: Scaling the enhanced defect image, and extracting high-dimensional features of the penultimate fully connected layer through a ResNet model to obtain image features; Obtaining time series features by encoding the time series dependence relationship of the time series signal through LSTM; Mapping the text log to a semantic vector through BERT embedding and extracting fault triples.

2. The equipment intelligent guarantee system based on an offline large model according to claim 1, characterized in that: The surface defect image refers to a crack image and a corrosion image; An infrared camera and a three-axis vibration sensor are mounted on the drone to collect defect images and time series signals, and text logs are obtained in real time through a PLC interface.

3. The equipment intelligent guarantee system based on an offline large model according to claim 1, characterized in that: The fusing of the three features to generate multi-modal features means mapping the image features, time series features, and text features to the same dimension through dynamic weight attention.

4. The equipment intelligent support system based on an offline large model according to claim 3, characterized in that: The steps for extracting fault triples from the semantic vectors and constructing a Bayesian causal graph according to the fault triples are as follows: Using a pre-trained Transformer decoder to parse the BERT semantic vectors and extract structured triples; Taking triple elements as nodes, constructing directed edges according to the causal chain to form the topology of the Bayesian network, and filling the prior distribution of the nodes based on the probability of the cause leading to the phenomenon and the success rate of the solution to repair the cause statistically from historical maintenance data, a Bayesian causal graph is obtained.

5. The equipment intelligent guarantee system based on the offline large model according to claim 4, characterized in that: Inputting the multi-modal features into the spiking neural network to dynamically update the node confidence and generate a knowledge graph, specifically: Converting the multi-modal features into spike trains, setting a global threshold to trigger the spike signal, and mapping the feature intensity to the spike interval; The nodes of the Bayesian network correspond to the SNN neuron clusters, and the spike-timing-dependent plasticity rule STDP is adopted; The node confidence is updated driven by the spike frequency, and finally a knowledge graph with weighted labels is generated.

6. The equipment intelligent support system based on the offline large model according to claim 5, characterized in that: Matching the multi-modal features with the knowledge graph to locate the root cause of the fault, specifically: Mapping the multi-modal fusion features to the node space of the knowledge graph through a fully connected layer to generate a node feature contribution vector, calculating the cosine similarity between the feature contribution vector and the nodes of the knowledge graph, and screening the candidate node set whose cosine similarity meets the actual requirements; Mapping the multi-modal features to an evidence vector, and updating the conditional probability of the Bayesian network through a non-linear evidence fusion function; Starting from the phenomenon node, traversing the Bayesian network causal path in reverse, calculating the comprehensive confidence of the path, fusing the node confidence and the updated conditional probability, arranging the candidate root causes in descending order of the comprehensive confidence, and selecting the node with the highest confidence as the root cause of the fault.

7. The equipment intelligent support system based on the offline large model according to claim 6, characterized in that: Pre-training a lightweight model on the edge server, using the quantum annealing algorithm to accelerate parameter aggregation, tuning the global large model according to the aggregated parameters, and generating a strategy in combination with reinforcement learning, specifically: Adopting the MobileNetV3-Small architecture, and compressing the number of parameters of ResNet proportionally through depthwise separable convolution and channel compression techniques; Based on the quantum annealing algorithm, transforming the problem of aggregating the local model parameters of the edge device into a QUBO model, and modeling the node selection strategy through binary variables.

8. The equipment intelligent support system based on an offline large model according to claim 7, characterized in that: Selecting the optimal maintenance strategy by comprehensively considering the cost, downtime loss, and spare parts inventory, specifically: Obtaining the maintenance cost through the historical equipment maintenance records, obtaining the downtime through the text logs, obtaining the spare parts inventory through the purchase orders, and obtaining the severity of the fault according to the historical fault records; Constructing a state space based on the maintenance cost, downtime, spare parts inventory, and fault severity, setting an action space based on immediate repair, delayed repair, and replacement of spare parts, and training a policy network in combination with the DDPG algorithm to output the Pareto optimal maintenance decision; Taking BERT-wwm as the teacher model and MobileNetV3-Tiny as the student model, aligning the soft label distribution through the KL divergence loss, and compressing the number of parameters proportionally; Converting the lightweight model into a PLC executable function block, receiving sensor data in real time, uploading the real-time data to the edge server through the IP protocol, and triggering the incremental update of the model; The edge server periodically fine-tunes the global model with new data, re-distills to generate a lightweight version and encrypts it for distribution to the PLC. If the accuracy of the validation set of the updated model decreases, it automatically rolls back to the old version and alarms.

Images