Equipment intelligent guarantee system based on off-line large model

Abstract

The invention discloses an equipment intelligent safeguard system based on an offline large model, and relates to the field of equipment management and safeguard, and the system comprises a collection module which collects an equipment surface defect image and a time sequence signal through an unmanned plane, collects a text log, processes the defect image, the time sequence signal and the text log, and generates a multi-modal data set; the feature fusion module is used for performing feature extraction on the multi-modal data set and fusing the three features to generate a multi-modal feature; the updating module is used for extracting a fault triple from the semantic vector, constructing a Bayesian causal graph according to the fault triple, dynamically updating node confidence and generating a knowledge graph; the maintenance strategy module is used for matching the multi-modal features with a knowledge graph and positioning a fault root cause; and the lightweight model deployment module is used for pre-training a lightweight model in an edge server, accelerating parameter aggregation by using a quantum annealing algorithm, adjusting and optimizing a global large model according to the aggregated parameters, and generating a strategy in combination with reinforcement learning.

Description

Technical Field

[0001] The present invention relates to the field of equipment management and support, and in particular to an equipment intelligent support system based on an offline large model. Background Art

[0002] In recent years, artificial intelligence technology has made significant progress in the field of intelligent equipment support. Intelligent support systems based on large models have gradually become a research hotspot by integrating multimodal data to achieve equipment status monitoring and fault diagnosis. For example, industrial large models have demonstrated potential in equipment health management, fault prediction and other fields by combining large-scale parameters with multimodal data. In existing technologies, multimodal data fusion methods can improve feature representation capabilities, while the combination of Bayesian networks and pulse neural networks (SNNs) provides new ideas for causal reasoning and dynamic knowledge updating. In addition, federated learning and quantum annealing algorithms are used for edge server model optimization to solve data privacy and computing power constraints. The multi-source data acquisition technology of drones combined with infrared cameras and vibration sensors, as well as the 3D modeling method based on SLAM, further improve the accuracy and spatial correlation of equipment surface defect detection.

[0003] However, existing technologies still face a series of challenges. First, traditional methods rely on static rules for cross-modal alignment, which makes it difficult to handle the dynamic association of heterogeneous data in complex industrial scenarios. For example, high-frequency vibration noise filtering and wavelet decomposition layer selection lack an adaptive mechanism, resulting in incomplete retention of fault feature frequency bands. Summary of the invention

[0004] In view of the above existing problems, the present invention is proposed.

[0005] Therefore, the present invention provides an equipment intelligent support system based on an offline large model to solve the problem that traditional methods rely on static rules in cross-modal alignment and are difficult to handle the dynamic association of heterogeneous data in complex industrial scenarios.

[0006] In order to solve the above technical problems, the present invention provides the following technical solutions:

[0007] The present invention provides an equipment intelligent support system based on an offline large model, which comprises:

[0008] Acquisition module, feature fusion module, update module, maintenance strategy module, and lightweight model deployment module:

[0009] The acquisition module collects equipment surface defect images and time series signals through drones, collects text logs at the same time, processes the defect images, time series signals and text logs respectively, and generates a multimodal data set;

[0010] The feature fusion module is used to extract features from the multimodal data set to obtain image features, time series features and text features, and fuse the three features to generate multimodal features;

[0011] The updating module is used to extract fault triples from the semantic vector, construct a Bayesian causal graph based on the fault triples, input the multimodal features into the spiking neural network, dynamically update the node confidence, and generate a knowledge graph;

[0012] The maintenance strategy module is used to match multimodal features with the knowledge graph to locate the root cause of the fault;

[0013] The lightweight model deployment module is used to pre-train the lightweight model on the edge server, use the quantum annealing algorithm to accelerate the aggregation of lightweight model parameters, generate a global large model according to the aggregated lightweight model parameters, combine reinforcement learning to generate strategies, and select the optimal maintenance strategy based on comprehensive costs, downtime losses and spare parts inventory.

[0014] As a preferred solution of the equipment intelligent support system based on offline large model described in the present invention, wherein: the surface defect image refers to a crack image and a corrosion image;

[0015] An infrared camera and a three-axis vibration sensor are installed on the drone to collect defect images and timing signals, and text logs are obtained in real time through the PLC interface.

[0016] As a preferred solution of the equipment intelligent support system based on the offline large model described in the present invention, the defect image, time series signal and text log are processed separately to generate a multimodal data set, specifically:

[0017] Separate the illumination component and reflection component of the defect image, combine the illumination mean template of the historical fault image, generate the optimized illumination component, and enhance the defect image according to the optimized illumination component and reflection component;

[0018] Wavelet basis decomposition is used to extract low-frequency fundamental frequency and high-frequency abnormal features;

[0019] Use BERT to identify component names and fault types, and combine regular expression matching logic chains to generate structured triples;

[0020] The pixel coordinates of the enhanced defect image are mapped to the 3D model of the equipment through SLAM technology, and the vibration signal is associated with the specific part;

[0021] The enhanced defect images and coordinates, filtered mean values ​​and associated part labels, and semantic vectors of fault triples are encapsulated in HDF5 format to generate a multimodal dataset.

[0022] As a preferred solution of the equipment intelligent support system based on the offline large model described in the present invention, wherein: the feature extraction of the multimodal data set is performed to obtain image features, time series features and text features, and the specific steps are:

[0023] The enhanced defect image is scaled, and the high-dimensional features of the penultimate fully connected layer are extracted through the ResNet model to obtain the image features;

[0024] The time series features are obtained by encoding the time series dependency of the time series signal through LSTM;

[0025] The text logs are mapped into semantic vectors through BERT embedding and fault triples are extracted.

[0026] As a preferred solution of the equipment intelligent support system based on offline large model described in the present invention, the fusion of three features to generate multimodal features refers to mapping image features, time series features and text features to the same dimension through dynamic weighted attention.

[0027] As a preferred solution of the equipment intelligent support system based on the offline large model described in the present invention, wherein: the fault triples are extracted from the semantic vector, and the Bayesian causal graph is constructed according to the fault triples, the specific steps are:

[0028] Use the pre-trained Transformer decoder to parse the BERT semantic vector and extract structured triples;

[0029] The triple elements are taken as nodes, and directed edges are constructed according to the causal chain to form a Bayesian network topology. Based on the statistical probability of the cause leading to the phenomenon and the success rate of the solution repairing the cause based on the historical maintenance data, the node prior distribution is filled to obtain the Bayesian causal graph.

[0030] As a preferred solution of the equipment intelligent support system based on the offline large model described in the present invention, wherein: the multimodal features are input into the pulse neural network, the node confidence is dynamically updated, and the knowledge graph is generated, specifically:

[0031] Convert multimodal features into pulse sequences, set a global threshold to trigger pulse signals, and map feature intensity to pulse intervals;

[0032] The Bayesian network nodes correspond to SNN neuron clusters, and the spike time-dependent plasticity rule STDP is adopted;

[0033] The node confidence is updated by the pulse frequency, and finally a knowledge graph with weighted labels is generated.

[0034] As a preferred solution of the equipment intelligent support system based on offline large model described in the present invention, the multimodal features are matched with the knowledge graph to locate the root cause of the fault, specifically:

[0035] Map the multimodal fusion features to the knowledge graph node space through the fully connected layer, generate the node feature contribution vector, calculate the cosine similarity between the feature contribution vector and the knowledge graph node, and select the candidate node set whose similarity meets the actual requirements;

[0036] Map multimodal features into evidence vectors and update the Bayesian network conditional probability through a nonlinear evidence fusion function;

[0037] Starting from the phenomenon node, the causal path of the Bayesian network is traversed in reverse, the comprehensive confidence of the path is calculated, the node confidence and the updated conditional probability are integrated, the candidate root causes are arranged in descending order according to the comprehensive confidence, and the node with the highest confidence is selected as the root cause of the fault.

[0038] As a preferred solution of the equipment intelligent support system based on offline large model described in the present invention, the lightweight model is pre-trained on the edge server, the quantum annealing algorithm is used to accelerate parameter aggregation, and the global large model is generated according to the aggregated parameters, combined with the reinforcement learning generation strategy, specifically:

[0039] The MobileNetV3-Small architecture is used to proportionally compress the ResNet parameters through deep separable convolution and channel compression technology.

[0040] Based on the quantum annealing algorithm, the local model parameter aggregation problem of edge devices is transformed into the QUBO model, and the node selection strategy is modeled through binary variables.

[0041] As a preferred solution of the equipment intelligent support system based on the offline large model of the present invention, the optimal maintenance strategy is selected based on the comprehensive cost, downtime loss and spare parts inventory, which is specifically:

[0042] Obtain maintenance costs through historical equipment maintenance records, obtain downtime through text logs, obtain spare parts inventory through purchase orders, and obtain fault severity based on historical fault records;

[0043] The state space is constructed based on maintenance cost, downtime, spare parts inventory and fault severity. The action space is set based on immediate maintenance, delayed maintenance and replacement of spare parts. The policy network is trained with the DDPG algorithm to output the Pareto optimal maintenance decision.

[0044] Using BERT-wwm as the teacher model and MobileNetV3-Tiny as the student model, the soft label distribution is aligned through KL divergence loss, and the number of parameters is compressed proportionally;

[0045] Convert the lightweight model into a PLC executable function block, receive sensor data in real time, upload the real-time data to the edge server via IP protocol, and trigger the incremental update of the model;

[0046] The edge server regularly fine-tunes the global model with new data, re-distills it to generate a lightweight version, and encrypts it and sends it to the PLC. If the accuracy of the model verification set decreases after the update, it automatically rolls back to the old version and issues an alarm.

[0047] The beneficial effects of the present invention are as follows: the present invention uses drones to collaboratively collect equipment surface defect images, vibration timing signals and operation logs, combines dynamic time warping with SLAM technology to achieve spatiotemporal alignment, and uses an illumination component optimization algorithm to enhance the recognizability of defect features, thereby solving the problems of traditional single-modal data isolation and noise interference; uses a dynamic weighted attention mechanism to fuse image, timing and text features, constructs a unified representation space, combines the dynamic collaborative mechanism of Bayesian causal graphs and pulse neural networks, and uses pulse triggering rules to update node confidence and causal strength in real time, significantly improving the real-time and accuracy of reasoning for complex faults; uses a quantum annealing algorithm to optimize the aggregation path of federated learning parameters, screens high-value edge nodes to reduce communication costs, combines reinforcement learning strategies to balance maintenance costs, downtime and spare parts inventory, and generates multi-objective optimal decisions; realizes efficient reasoning of edge servers through knowledge distillation and lightweight model deployment, and combines incremental updates with abnormal rollback mechanisms to ensure model stability. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative work.

[0049] Figure 1 This is a flow chart of the equipment intelligent support system based on the offline large model in this embodiment.

[0050] Figure 2 Schematic diagram of dynamic feature fusion and Bayesian causal graph in this embodiment.

[0051] Figure 3 This is an architectural diagram of the quantum annealing-driven federated learning parameter aggregation in this embodiment.

[0052] Figure 4 This is an architecture diagram of the equipment intelligent support system based on the offline large model in this embodiment. DETAILED DESCRIPTION

[0053] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are described in detail below in conjunction with the accompanying drawings.

[0054] In the following description, many specific details are set forth to facilitate a full understanding of the present invention, but the present invention may also be implemented in other ways different from those described herein, and those skilled in the art may make similar generalizations without violating the connotation of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0055] Secondly, the term "one embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The term "in one embodiment" that appears in different places in this specification does not necessarily refer to the same embodiment, nor does it refer to a separate or selective embodiment that is mutually exclusive with other embodiments.

[0056] In this embodiment, refer to Figures 1 to 4 , which is the first embodiment of the present invention, and provides an equipment intelligent support system based on an offline large model, comprising the following steps:

[0057] Acquisition module, feature fusion module, update module, maintenance strategy module, and lightweight model deployment module:

[0058] The acquisition module uses drones to collect equipment surface defect images and time series signals, and collects text logs at the same time. The defect images, time series signals and text logs are processed separately to generate a multimodal data set;

[0059] Surface defect images refer to crack images and corrosion images;

[0060] The drone is equipped with a high-definition infrared camera and a three-axis vibration sensor. The camera and vibration sensor timestamp synchronization is achieved through GPS signals. The vibration sensor is calibrated for sensitivity using the ISO 16063-21 standard to ensure that the range covers 0-10kHz. The camera is calibrated for distortion using the checkerboard calibration method, and the white balance is calibrated using a grayscale card to eliminate light interference.

[0061] The drone flies along the preset route, 2-5m away from the equipment surface, captures images of defective areas on the equipment surface, dynamically adjusts the height through the laser rangefinder to ensure that the defect image covers key parts (gearbox, bearing seat), and continuously collects acceleration signals of key parts of the equipment (such as gearbox, bearing seat) through vibration sensors, and records timestamps;

[0062] Real-time access to text logs (fault codes and maintenance records) via Modbus TCP / IP PLC interface, suitable for Ethernet-based communications, supporting real-time data transmission;

[0063] Separate the illumination component and reflection component of the defect image, combine the illumination mean template of the historical fault image (illumination intensity of the corrosion area), and generate the optimized illumination component. The expression is: ;

[0064] in, Indicates the pixel position The new light intensity at Represents the average value of the historical illumination template, based on the illumination statistics of the same area in the historical fault image, in lux. Indicates the pixel position The original illumination component at It indicates that the contribution of historical lighting template accounts for 70%. Indicates that the current lighting component contributes 30%;

[0065] The defect image is enhanced according to the optimized illumination component and reflection component. The expression is: ;

[0066] in, Represents the reflection component, indicating the crack texture and corrosion texture on the surface of the object. It is obtained by decomposing the original image and its value range is [0,1]. The closer the value is to 1, the stronger the reflection is, and the closer it is to 0, the stronger the absorption is. represents the enhanced defect image;

[0067] Select sym8 wavelet basis to perform 5-layer decomposition, and generate approximate coefficients and detail coefficients at each layer;

[0068] The approximation coefficient A represents the low-frequency component of the signal (the fundamental frequency vibration when the equipment is operating normally);

[0069] The detail coefficient D represents the high-frequency components of the signal (abnormal vibration, noise);

[0070] The SUREShrink threshold method is used to filter out the noise > 15kHz for the high-frequency coefficients, and the fault characteristic frequency band is retained;

[0071] Use the BERT model to identify component names and fault types in text logs. Fault types include cracks, deformation, and corrosion. Component names include:

[0072] In fault log analysis, regular expressions are used to extract phenomena, causes, and measures from unstructured texts. Regular expressions are used to match key event patterns: phenomena → causes → measures.

[0073] Set phenomenon-cause rules: if the text contains "abnormal vibration" and the context has "insufficient lubrication", then generate a causal chain: 〈abnormal vibration → insufficient lubrication〉;

[0074] If the text contains "temperature is too high" and the context contains "bearing wear", a causal chain is generated: 〈temperature is too high → bearing wear〉;

[0075] Set the cause-solution rule: if the cause = "lack of lubrication", then the solution = "add lubricant";

[0076] If the cause = "bearing wear", then the solution = "replace the bearing";

[0077] Input examples and output triples;

[0078] The dynamic time warping (DTW) algorithm is used to align the vibration signal with the image frame by timestamp to ensure data association at the same time (e.g. abnormal vibration corresponds to crack image).

[0079] The pixel coordinates of the enhanced defect image are mapped to the 3D model of the equipment through SLAM technology, and the vibration signal is associated with the specific part;

[0080] Bind the vibration signal to the corresponding part in the 3D model according to the installation position of the vibration sensor;

[0081] The enhanced defect image and 3D coordinates, the filtered mean and associated part labels, and the semantic vector of the fault triplet are encapsulated in HDF5 format to generate a multimodal dataset.

[0082] The feature fusion module is used to extract features from the multimodal data set to obtain image features, time series features, and text features, and then fuse the three features to generate multimodal features.

[0083] The enhanced defect image is scaled and normalized to the range of [0,1], and the high-dimensional features of the penultimate fully connected layer are extracted through the ResNet model to obtain the image features;

[0084] The time series features are obtained by encoding the time series dependency of the time series signal through LSTM. Specifically, the filtered vibration signal is divided into time windows (1 second / segment), each segment contains several sampling points, and the time domain features (mean, variance, peak value) and frequency domain features (FFT main frequency, harmonic energy ratio) of each segment are extracted. The feature sequence is input into the bidirectional LSTM network to capture the previous and next dependencies and obtain the time series features.

[0085] Map text logs into semantic vectors and extract fault triplets through BERT embedding. Specifically, perform word segmentation on structured logs, remove stop words, retain entities (component name, fault type), use BERT-wwm (full-word masking pre-trained student model) to encode fault triplets, optimize the semantic space through contrastive learning loss, make the triplets of similar faults closer, and output text features.

[0086] The image features, temporal features and text features are mapped to the same dimension (256 dimensions) through dynamic weight attention, and the expression is: ;

[0087] in, represents the fused multimodal feature vector, Index variables representing image features, temporal features, and text features, The values ​​of are 1, 2, and 3, corresponding to image features, temporal features, and text features, respectively. Indicates The attention weight of each feature, Indicates The features are mapped to a 256-dimensional feature vector.

[0088] The update module is used to extract fault triples from the semantic vector, construct a Bayesian causal graph based on the fault triples, input multimodal features into the spiking neural network, dynamically update the node confidence, and generate a knowledge graph;

[0089] Through the pre-trained Transformer architecture triple decoder, the structured triples (phenomenon → cause → solution) are parsed from the semantic vector generated by BERT, such as "abnormal vibration → insufficient lubrication → replenishing lubricant", and the semantic vector is input to obtain the fault triple;

[0090] The fault triplet phenomenon, cause and solution are taken as nodes respectively, and directed edges are established according to the causal chain (phenomenon → cause → solution). The Bayesian network structure corresponding to the triplet "abnormal vibration, insufficient lubrication, replenishing lubricant" is: insufficient lubrication → abnormal vibration → replenishing lubricant;

[0091] The conditional probability of the cause leading to the phenomenon and the success rate of the solution to the cause are calculated from the historical maintenance data, that is, the prior probability;

[0092] For each cause node, fill in the probability distribution of the phenomenon caused by it according to historical data, and for each solution node, fill in the probability distribution of its repair cause to generate a Bayesian causal graph;

[0093] Each dimension value of the multimodal feature is converted into a pulse sequence, and a global threshold is uniformly set. When the feature value exceeds the global threshold, a pulse signal is triggered. Through pulse time coding, the feature intensity is mapped to the pulse time interval. The larger the original feature intensity value of the multimodal feature, the shorter the pulse interval.

[0094] Each node in the Bayesian causal graph corresponds to a SNN neuron cluster;

[0095] The spike time-dependent plasticity rule STDP is used to dynamically adjust the synaptic weight according to the spike time difference between the previous and next neurons.

[0096] If the parent node pulse is earlier than the child node, long-term potentiation is triggered and the synaptic weight is enhanced. The expression is: ;

[0097] in, represents the change in synaptic weight, represents the learning rate, represents the base of natural logarithms, Represents the time difference between the previous and next neuron pulses, in milliseconds (ms), Indicates time, is the time constant, It can be set to 10ms (milliseconds), balancing the sensitivity and stability of weight adjustment in real-time inference scenarios of industrial equipment, and controlling the weight decay speed. The smaller it is, the more sensitive the weight adjustment is to the time difference and the faster it decays;

[0098] If the parent node pulse is later than the child node pulse, long-term inhibition is triggered and the synaptic weight is weakened. The expression is: ;

[0099] The causal strength of the edge is dynamically adjusted based on the conditional probability and pulse triggering times of the Bayesian network to generate a knowledge graph with weight labels. The size of the node indicates the confidence level, and the thickness of the edge indicates the causal strength.

[0100] The node confidence is determined by the pulse frequency. The higher the pulse frequency, the greater the confidence improvement. The node confidence is dynamically adjusted according to the pulse frequency. The expression is: ;

[0101] in, Representation Node In time The updated confidence of Representation Node In time The updated confidence of Represents the gain coefficient, and its value range is [0,1]. The larger the gain coefficient value, the more significant the impact of pulse frequency on confidence. Representation Node The pulse frequency, Indicates the target maximum pulse frequency.

[0102] Maintenance strategy module, used to match multimodal features with knowledge graphs to locate the root cause of faults;

[0103] The multimodal fusion features are mapped to the knowledge graph node space through the fully connected layer to generate the feature contribution vector of each node, which is expressed as: ;

[0104] in, Representation Node The feature contribution vector of The index variable representing the node, is the activation function, is the weight matrix of the node mapping layer, which is optimized by the back propagation algorithm during the training phase and is used to map multimodal features to the knowledge graph node space. Represents multimodal features, Represents the bias term of the node mapping layer;

[0105] Calculate the cosine similarity between the node feature contribution vector and all nodes in the knowledge graph, and select the candidate node set with similarity greater than 0.7;

[0106] Mapping multimodal features into evidence vectors , the expression is: ;

[0107] in, represents the evidence vector, Indicates the proportion of crack area, indicating the support strength of image features for the current causal relationship. Its value range is [0,1]. The closer it is to 1, the stronger the image evidence. It represents the proportion of abnormal vibration energy and the support strength of vibration signal characteristics for causal relationship. Its value range is [0,1]. The closer it is to 1, the stronger the temporal evidence. Indicates the semantic matching degree of the triple, with a value range of [0,1]. The closer it is to 1, the higher the matching degree between the text description and the current fault;

[0108] The conditional probability of the Bayesian network is updated using the nonlinear evidence fusion function, which is expressed as: ;

[0109] in, represents the updated conditional probability, which means the new probability value of "cause X leads to phenomenon Y" under the support of evidence vector E. Represents the conditional probability of historical statistics, which represents the prior probability of "cause X leading to phenomenon Y" based on historical maintenance data. , and Respectively represent the influence weights of image modality, time series modality, and text modality on causal relationships, which are obtained through historical data training. represents the base of natural logarithms;

[0110] Starting from the phenomenon node (such as "abnormal vibration"), all causal paths are traversed in reverse along the Bayesian network, and the comprehensive confidence of each causal path is calculated. The expression is: ;

[0111] in, represents the comprehensive confidence, represents the number of nodes, The index variable representing the node in the path, Indicates The confidence of the node, represents the updated conditional probability, Indicates the first Nodes represent the fault phenomenon, cause or solution. Indicates the first nodes;

[0112] Arrange the candidate root causes in descending order of comprehensive confidence, and select the one with the highest comprehensive confidence as the root cause of the fault.

[0113] The lightweight model deployment module is used to pre-train lightweight models on edge servers, use the quantum annealing algorithm to accelerate the aggregation of lightweight model parameters, generate a global large model based on the aggregated lightweight model parameters, combine reinforcement learning to generate strategies, and select the optimal maintenance strategy based on comprehensive costs, downtime losses, and spare parts inventory;

[0114] MobileNetV3-Small is used as the basic architecture. Through deep separable convolution and channel compression technology, the number of ResNet parameters is compressed to 1 / 10, and the feature extraction capability is enhanced through adaptive attention.

[0115] Obtain maintenance costs through historical equipment maintenance records, obtain downtime through text logs, obtain spare parts inventory through purchase orders, and obtain fault severity based on historical fault records;

[0116] The output probability distribution of the student model is used as a soft label, and the divergence loss is used to guide the lightweight model training, retaining more than 95% of the semantic understanding accuracy;

[0117] On the edge device carried by the drone, the multimodal features are trained locally, and the optimization goal is to minimize the cross entropy loss. By comparing the true distribution with the predicted distribution, the prediction error of the student model for the multi-classification task is measured. The expression is: ;

[0118] in, represents the edge supervision loss function, represents the total number of categories, represents the index of the category, Represents the one-hot encoding of the true label, when the category is the true category =1, otherwise =0, represents the predicted probability, represents the natural logarithm of the predicted probability, represents the regularization coefficient, Represents the set of trainable parameters of the student model;

[0119] The parameter aggregation problem of federated learning is transformed into a QUBO model that can be solved by quantum annealing. Specifically:

[0120] Set a binary variable (0 or 1) for each edge device (node). For example, if the node participates in this parameter aggregation, it is marked as 1, otherwise it is 0. Then calculate the Euclidean distance between the model parameters of different nodes. The node pairs with greater differences are given higher penalty weights in the optimization objective.

[0121] Weights are assigned based on the amount of local data of the node. Nodes with high data quality are assigned positive weights in the objective function, and nodes with low data quality are assigned negative weights.

[0122] Integrate parameter differences and data quality into Hamiltonian. If the data quality of a node is high and the parameter difference with other nodes is small, the binary variable corresponding to the edge server will be preferred; otherwise, it will be suppressed.

[0123] Setting the communication cost limit data of federated learning can be achieved by adding a penalty term in the Hamiltonian. If the total number of selected nodes exceeds the limit, the energy function value will increase significantly, forcing the quantum annealer to avoid such solutions.

[0124] Map the binary variables of each node to the quantum bits of the quantum annealer to form a physical quantum bit chain;

[0125] Through the physical annealing process of the quantum annealer, the Hamiltonian automatically converges to the lowest energy state, corresponding to the optimal node selection combination. For example, the annealer outputs a set of binary variables in milliseconds, indicating which nodes should participate in the current aggregation;

[0126] Use the D-Wave 2000Q quantum annealer, set the annealing time, solve the optimal parameter combination, and output the global model parameters. The expression is: ;

[0127] in, It represents the global model parameters generated by the server in federated learning after aggregating the local model parameters of all participating clients. Indicates traversal of the client All client indexes in , Indicates The local model parameters of each client are updated through local data training, indicating The set of clients participating in the current round of federated learning, Indicates the number of clients participating in the aggregation;

[0128] Collect maintenance cost, downtime, spare parts inventory and fault severity (mapped by Bayesian causal graph confidence) and generate state space;

[0129] Set the action space, which includes immediate repair, delayed repair and replacement of spare parts;

[0130] Construct the reward function based on the state space, and the expression is: ;

[0131] in, Indicates at time The reward value, , , and They represent the priorities of maintenance cost, downtime, spare parts inventory, and fault severity, respectively. Represents the normalized maintenance cost, which is obtained by dividing the actual maintenance cost by the preset maximum value. Indicates the downtime duration. Indicates the spare parts inventory percentage, Indicates the severity of the fault;

[0132] Using deep deterministic policy gradient DDPG, the Actor network outputs continuous actions, and the Critic network evaluates the Q value. After 10,000 rounds of training, the strategy converges to the Pareto frontier. The generated strategies are sorted in descending order according to the Q value, and the strategy with the largest Q value is selected as the optimal maintenance strategy.

[0133] Use the trained global large model BERT-wwm as the "teacher", input multimodal data (images, vibration signals, text logs), and output the probability distribution of fault classification (soft labels);

[0134] Select a lightweight architecture (MobileNetV3-Tiny), input the same data, and adjust the parameters to imitate the decision logic of the teacher model by comparing the soft labels of the teacher model with its own output;

[0135] Remove redundant neuron connections in the student model, retain the core feature extraction layer, compress the number of parameters to 1 / 50 of the original model, and verify the classification accuracy of the student model on the historical data set to ensure that the accuracy loss is ≤3% (for example, the accuracy of the teacher model is 95%, and the student model is ≥92%).

[0136] Convert the student model into a binary format recognizable by the PLC, encapsulate it into an independent function block, and support calling through the PLC programming language;

[0137] Through the IP protocol, the defect images, timing signals and text logs connected to the PLC are received in real time, and the input data is normalized and the images are cropped to the key areas on the PLC side;

[0138] The PLC periodically packages and uploads real-time data to the edge server;

[0139] The edge server uses the new data to fine-tune the global large model, update the fault classification rules (bearing wear judgment threshold), and re-perform knowledge distillation to generate a new version of the student model;

[0140] Encrypt the updated student model and push it to the PLC, overwriting the old version;

[0141] If the accuracy of the student model in the validation set drops by more than 5% after the update, a rollback is automatically triggered, restoring to the previous stable version and an alarm is issued;

[0142] Record the time from data input to output instruction of each student model, and trigger an alarm when it exceeds 50ms;

[0143] Monitor PLC memory usage and terminate low-priority tasks (such as log storage) when it exceeds 80%.

[0144] In summary, the present invention uses drones to collaboratively collect equipment surface defect images, vibration timing signals and operation logs, combines dynamic time warping with SLAM technology to achieve spatiotemporal alignment, and uses an illumination component optimization algorithm to enhance the recognizability of defect features, thereby solving the problems of traditional single-modal data isolation and noise interference. The dynamic weighted attention mechanism is used to fuse image, timing and text features to construct a unified representation space, and the dynamic collaborative mechanism of the Bayesian causal graph and the pulse neural network is combined to use pulse triggering rules to update node confidence and causal strength in real time, significantly improving the real-time and accuracy of reasoning for complex faults. The quantum annealing algorithm is used to optimize the aggregation path of federated learning parameters, and high-value edge nodes are screened to reduce communication costs. The reinforcement learning strategy is combined to balance maintenance costs, downtime and spare parts inventory, and to generate multi-objective optimal decisions. Knowledge distillation and lightweight model deployment are used to achieve efficient reasoning of edge servers, and incremental updates and abnormal rollback mechanisms are combined to ensure model stability.

[0145] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention, which should all be included in the scope of the claims of the present invention.

Claims

1. An equipment intelligent support system based on an offline large model, characterized by: include: Acquisition module, feature fusion module, update module, maintenance strategy module, and lightweight model deployment module; The acquisition module collects equipment surface defect images and time series signals through drones, collects text logs at the same time, processes the defect images, time series signals and text logs respectively, and generates a multimodal data set; The feature fusion module is used to extract features from the multimodal data set to obtain image features, time series features and text features, and fuse the three features to generate multimodal features; The updating module is used to extract fault triples from the semantic vector, construct a Bayesian causal graph based on the fault triples, input the multimodal features into the spiking neural network, dynamically update the node confidence, and generate a knowledge graph; The maintenance strategy module is used to match multimodal features with the knowledge graph to locate the root cause of the fault; The lightweight model deployment module is used to pre-train the lightweight model on the edge server, use the quantum annealing algorithm to accelerate the aggregation of lightweight model parameters, tune the global large model according to the aggregated lightweight model parameters, combine the reinforcement learning generation strategy, and select the optimal maintenance strategy based on the comprehensive cost, downtime loss and spare parts inventory.

2. The equipment intelligent support system based on offline large model as claimed in claim 1, characterized in that: The surface defect image refers to a crack image and a corrosion image; An infrared camera and a three-axis vibration sensor are installed on the drone to collect defect images and timing signals, and text logs are obtained in real time through the PLC interface.

3. The equipment intelligent support system based on offline large model as claimed in claim 2, characterized in that: The defect image, time series signal and text log are processed respectively to generate a multimodal data set, specifically: Separate the illumination component and reflection component of the defect image, combine the illumination mean template of the historical fault image, generate the optimized illumination component, and enhance the defect image according to the optimized illumination component and reflection component; Wavelet basis decomposition is used to extract low-frequency fundamental frequency and high-frequency abnormal features; Use BERT to identify component names and fault types, and combine regular expression matching logic chains to generate structured triples; The pixel coordinates of the enhanced defect image are mapped to the 3D model of the equipment through SLAM technology, and the vibration signal is associated with the specific part; The enhanced defect images and coordinates, filtered mean values ​​and associated part labels, and semantic vectors of fault triples are encapsulated in HDF5 format to generate a multimodal dataset.

4. The equipment intelligent support system based on offline large model as claimed in claim 3, characterized in that: The feature extraction of the multimodal data set is performed to obtain image features, time series features and text features, and the specific steps are as follows: The enhanced defect image is scaled, and the high-dimensional features of the penultimate fully connected layer are extracted through the ResNet model to obtain the image features; The time series features are obtained by encoding the time series dependency of the time series signal through LSTM; The text logs are mapped into semantic vectors through BERT embedding and fault triples are extracted.

5. The equipment intelligent support system based on offline large model as claimed in claim 4, characterized in that: The fusion of the three features to generate multimodal features refers to mapping image features, time series features and text features to the same dimension through dynamic weighted attention.

6. The equipment intelligent support system based on offline large model as claimed in claim 5, characterized in that: The specific steps of extracting fault triples from the semantic vector and constructing a Bayesian causal graph according to the fault triples are as follows: Use the pre-trained Transformer decoder to parse the BERT semantic vector and extract structured triples; The triple elements are taken as nodes, and directed edges are constructed according to the causal chain to form a Bayesian network topology. Based on the statistical probability of the cause leading to the phenomenon and the success rate of the solution repairing the cause based on the historical maintenance data, the node prior distribution is filled to obtain the Bayesian causal graph.

7. The equipment intelligent support system based on offline large model as claimed in claim 6, characterized in that: The multimodal features are input into the spiking neural network, the node confidence is dynamically updated, and the knowledge graph is generated, specifically: Convert multimodal features into pulse sequences, set a global threshold to trigger pulse signals, and map feature intensity to pulse intervals; The Bayesian network nodes correspond to SNN neuron clusters, and the spike time-dependent plasticity rule STDP is adopted; The node confidence is updated by the pulse frequency, and finally a knowledge graph with weighted labels is generated.

8. The equipment intelligent support system based on offline large model as claimed in claim 7, characterized in that: The multimodal features are matched with the knowledge graph to locate the root cause of the fault, specifically: Map the multimodal fusion features to the knowledge graph node space through the fully connected layer, generate the node feature contribution vector, calculate the cosine similarity between the feature contribution vector and the knowledge graph node, and select the candidate node set whose similarity meets the actual requirements; Map multimodal features into evidence vectors and update the Bayesian network conditional probability through a nonlinear evidence fusion function; Starting from the phenomenon node, the causal path of the Bayesian network is traversed in reverse, the comprehensive confidence of the path is calculated, the node confidence and the updated conditional probability are integrated, the candidate root causes are arranged in descending order according to the comprehensive confidence, and the node with the highest confidence is selected as the root cause of the fault.

9. The equipment intelligent support system based on offline large model as claimed in claim 8, characterized in that: The lightweight model is pre-trained on the edge server, the quantum annealing algorithm is used to accelerate parameter aggregation, the global large model is tuned according to the aggregated parameters, and the strategy is generated by combining reinforcement learning, specifically: The MobileNetV3-Small architecture is used to proportionally compress the ResNet parameters through deep separable convolution and channel compression technology. Based on the quantum annealing algorithm, the local model parameter aggregation problem of edge devices is transformed into the QUBO model, and the node selection strategy is modeled through binary variables.

10. The equipment intelligent support system based on offline large model as claimed in claim 9, characterized in that: The optimal maintenance strategy is selected based on the comprehensive cost, downtime loss and spare parts inventory, which is: Obtain maintenance costs through historical equipment maintenance records, obtain downtime through text logs, obtain spare parts inventory through purchase orders, and obtain fault severity based on historical fault records; The state space is constructed based on maintenance cost, downtime, spare parts inventory and fault severity. The action space is set based on immediate maintenance, delayed maintenance and replacement of spare parts. The policy network is trained with the DDPG algorithm to output the Pareto optimal maintenance decision. Using BERT-wwm as the teacher model and MobileNetV3-Tiny as the student model, the soft label distribution is aligned through KL divergence loss, and the number of parameters is compressed proportionally; Convert the lightweight model into a PLC executable function block, receive sensor data in real time, upload the real-time data to the edge server via IP protocol, and trigger the incremental update of the model; The edge server regularly fine-tunes the global model with new data, re-distills it to generate a lightweight version, and encrypts it and sends it to the PLC. If the accuracy of the model verification set decreases after the update, it automatically rolls back to the old version and issues an alarm.

Images