Equipment fault prediction method and device, equipment and medium

Abstract

The invention discloses an equipment fault prediction method and device, equipment and a medium, and relates to the technical field of Internet of Things. The method comprises the following steps: acquiring real-time operation data of monitored external equipment; processing the real-time operation data by using a lightweight neural network model so as to extract feature information representing the state of the equipment; the lightweight neural network model is obtained based on teacher model training through a knowledge distillation technology; based on the feature information, performing fault tendency prediction through a lightweight neural network model to obtain a prediction result used for indicating the probability that the external device has a target fault in the future; inputting the feature information into a preset probability clustering model for clustering analysis so as to identify the state mode category of the external equipment; and based on the prediction result and the state mode category, generating maintenance decision information for the external equipment. Through the technical scheme of the invention, the prior art can be optimized, and the accuracy and benefit of intelligent fault prediction of the telemetering terminal can be improved.

Description

Technical Field

[0001] This invention relates to the field of Internet of Things (IoT) technology, and in particular to a method, apparatus, device, and medium for predicting device failures. Background Technology

[0002] Traditional maintenance relies on periodic inspections or post-failure repairs, which are inefficient, costly, and prone to economic losses due to unplanned downtime. The industry urgently needs to shift from "preventive maintenance" to "predictive maintenance," which involves analyzing equipment operating data to anticipate faults. The maturity of the Internet of Things (IoT), edge computing, artificial intelligence algorithms, and dedicated AI chips has made it possible to collect equipment data on a large scale at low cost. Intelligent telemetry terminals, as edge nodes, can perform local real-time data analysis and AI inference, providing technical support for predictive maintenance. Current technologies use a pipeline of "data preprocessing → feature extraction and pattern recognition → time-series prediction and decision-making," combined with CNN (Convolutional Neural Network), LSTM (Long Short-Term Memory), and K-Means algorithms to achieve fault prediction. However, this technology has significant shortcomings: CNN and LSTM models have high computational overhead, making them difficult to adapt to resource-constrained IoT terminals; the K-Means algorithm is susceptible to noise and performs poorly in clustering faults with non-convex shapes and complex distributions. Therefore, providing a solution to these technical problems is a problem that those skilled in the art need to address. Summary of the Invention

[0003] In view of this, the purpose of this invention is to provide a method, apparatus, device, and medium for predicting equipment failures, which can optimize existing technologies and improve the accuracy and efficiency of intelligent failure prediction by telemetry terminals; it can accurately identify early signs before equipment failures occur, avoiding huge losses caused by unplanned downtime. The specific solution is as follows: In a first aspect, this application discloses a method for predicting equipment failures, including: Acquire real-time operating data of the monitored external devices; The real-time operating data is processed using a lightweight neural network model to extract feature information representing the device status; the lightweight neural network model is trained based on a teacher model using knowledge distillation technology. Based on the feature information, a fault tendency prediction is performed using the lightweight neural network model to obtain a prediction result indicating the probability of the external device experiencing a target fault in the future. The feature information is input into a preset probability clustering model for cluster analysis to identify the state pattern category of the external device; Based on the prediction results and the state pattern category, maintenance decision information for the external device is generated.

[0004] Optionally, constructing the lightweight neural network model includes: Construct a teacher model that incorporates convolutional neural networks and long short-term memory networks; Construct a student model with fewer parameters than the teacher model, wherein the student model includes the convolutional neural network and the long short-term memory network; The teacher model's output is used to guide the training of the student model through knowledge distillation, and the trained student model is then lightweighted to obtain the lightweight neural network model.

[0005] Optionally, the step of using knowledge distillation technology to guide the training of the student model using the output of the teacher model includes: The training dataset is input into the teacher model to obtain the teacher feature map, teacher soft label, and teacher hidden state output by the model. Input the training dataset into the student model to obtain the student feature map, student soft label, and student hidden state output by the model; The losses between the teacher feature map and the student feature map, the teacher soft label and the student soft label, and the teacher hidden state and the student hidden state are determined respectively to obtain the corresponding feature map loss, soft label loss and hidden state loss; The feature map loss, the soft label loss, and the hidden state loss are weighted and combined to determine the total loss function, and the parameters of the student model are updated through backpropagation.

[0006] Optionally, the trained student model can be lightweighted, including: The structural part of the convolutional neural network in the student model is subjected to a first lightweighting process, and the structural part of the long short-term memory network in the student model is subjected to a second lightweighting process. The first lightweighting process includes at least one of the following methods: replacing the backbone network of the convolutional neural network with a depth-separable convolutional structure, performing channel pruning on the convolutional layers of the convolutional neural network, and quantizing the weights and activation values ​​in the convolutional neural network. The second lightweighting process includes at least one of the following methods: pruning the hidden units of the Long Short-Term Memory (LSTM) network, performing low-rank decomposition on the weight matrix of the LSTM network, and quantizing the weights and activation values ​​of the LSTM network.

[0007] Optionally, the preset probability clustering model is a Gaussian mixture model, and the step of inputting the feature information into the preset probability clustering model for cluster analysis to identify the state pattern category of the external device includes: The Gaussian mixture model is trained based on the feature information, and the number of Gaussian components in the Gaussian mixture model is determined. The parameters of multiple Gaussian components are learned using the expectation-maximization algorithm, and the posterior probability of each feature belonging to each Gaussian component is determined; the parameters include the mean vector, covariance matrix, and mixture weights. The state mode category of the external device is identified based on the posterior probability.

[0008] Optionally, before performing fault tendency prediction using the lightweight neural network model based on the feature information, the method further includes: The lightweight neural network model is converted into an edge inference format using a model conversion tool, and the lightweight neural network model in the edge inference format is deployed to the telemetry terminal. An OTA update mechanism is established, and an updated lightweight neural network model is received from the cloud server based on the OTA update mechanism to update the original lightweight neural network model on the telemetry terminal.

[0009] Optionally, generating maintenance decision information for the external device based on the prediction result and the state pattern category includes: High-frequency fault patterns are identified from the state pattern categories, and the high-frequency fault patterns are determined based on the mixing weight of each Gaussian component in the preset probability clustering model; An abnormal state that does not conform to any known fault mode is detected from the state mode category. The abnormal state is determined based on the fact that the posterior probability of the data point belonging to any Gaussian component is lower than a preset threshold. Based on the prediction results, the high-frequency fault modes, and the abnormal states, maintenance decision information for the external device is generated, and different priorities are assigned to the maintenance decision information.

[0010] Secondly, this application discloses a device for predicting equipment failures, comprising: The data acquisition module is used to acquire real-time operating data of the monitored external devices; The feature extraction module is used to process the real-time operating data using a lightweight neural network model to extract feature information representing the device status; the lightweight neural network model is obtained by knowledge distillation technology and trained based on a teacher model. The prediction result output module is used to perform fault tendency prediction based on the feature information and through the lightweight neural network model to obtain a prediction result indicating the probability of the external device experiencing a target fault in the future. The state pattern category recognition module is used to input the feature information into a preset probability clustering model for cluster analysis in order to identify the state pattern category of the external device. The maintenance decision information generation module is used to generate maintenance decision information for the external device based on the prediction results and the state mode category.

[0011] Thirdly, this application discloses an electronic device including a processor and a memory; wherein the memory is used to store a computer program, which is loaded and executed by the processor to implement the device fault prediction method as described above.

[0012] Fourthly, this application discloses a computer-readable storage medium for storing a computer program; wherein the computer program, when executed by a processor, implements the device failure prediction method as described above.

[0013] The beneficial technical effects of this application are as follows: By adopting a lightweight neural network model obtained through knowledge distillation, the ability to identify key fault features learned from complex teacher models is retained while significantly reducing the computational complexity and resource consumption of the model. This makes it possible to achieve real-time and efficient localized AI inference on resource-constrained telemetry terminals, effectively solving the bottleneck of the current difficulty in deploying the large model on edge nodes. Furthermore, by introducing a probabilistic clustering model that can output probability distributions to perform soft partitioning of feature information, compared with traditional hard clustering algorithms such as K-Means, it can capture the overlapping and gradual relationships between equipment states more precisely, thereby more accurately identifying "high-frequency fault areas" and "high-incidence fault types", improving the accuracy and robustness of fault state pattern detection. Finally, by combining the prediction results of the lightweight model with the deep pattern recognition results of probabilistic clustering analysis to generate maintenance decisions, an upgrade from "preventive maintenance" to more accurate "predictive maintenance" is achieved. While ensuring equipment reliability, the allocation of operation and maintenance resources is optimized, and the risk of unplanned downtime is reduced.

[0014] In addition, the equipment failure prediction device, equipment and storage medium provided in this application correspond to the above-mentioned equipment failure prediction method and have the same effect. Attached Figure Description

[0015] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0016] Figure 1 This is a typical flowchart of an intelligent telemetry terminal system architecture; Figure 2 A flowchart for intelligent fault prediction in an existing technology; Figure 3 This is a flowchart of a device failure prediction method disclosed in this application; Figure 4 This application discloses a knowledge distillation flowchart; Figure 5 This application discloses a flowchart of model lightweighting and optimization. Figure 6 This is a flowchart illustrating the internal workflow of an improved clustering analysis module disclosed in this application. Figure 7 This is a schematic diagram of the structure of an equipment failure prediction device disclosed in this application; Figure 8 This is a structural diagram of an electronic device disclosed in this application. Detailed Implementation

[0017] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0018] With the rise and maturation of the Internet of Things (IoT) and data acquisition technologies, edge computing, artificial intelligence (AI) algorithms, and dedicated AI chips, it has become possible to collect device operation data on a large scale and at low cost. As an edge node in the "cloud-edge-device" architecture, the intelligent telemetry terminal is responsible for aggregating this sensor data. Through its built-in processor with a certain computing power, it can perform real-time data analysis and AI inference locally, uploading only key results or summary data to the cloud, achieving rapid response and saving bandwidth. Artificial intelligence algorithms provide the technical means to extract valuable information from massive amounts of device data, effectively capturing trends of device performance degradation and early fault characteristics. Simultaneously, to efficiently run AI algorithms on resource-constrained edge devices, ultra-low-power dedicated AI chips have been developed, providing the hardware foundation for the intelligence of intelligent telemetry terminals. Figure 1 The diagram shown is a typical flowchart of an intelligent telemetry terminal system.

[0019] Existing technologies have proposed intelligent fault prediction schemes for operation and maintenance terminals in distributed Internet of Things (IoT) systems. For example... Figure 2The diagram shows the overall workflow of the system. Its core is to build an intelligent pipeline of "data preprocessing → feature extraction and pattern recognition → time series prediction and decision-making." Its technology stack is a typical and advanced combination in current AI industrial applications. In the data preprocessing stage, it uses data cleaning (duplicate removal, outlier handling), missing value imputation (mean / interpolation), and data normalization (Min-Max Scaling to the [0,1] interval). This is the cornerstone of all machine learning projects. Industrial field data often suffers from a large amount of noise, missing values, and inconsistent dimensions. Preprocessing directly determines the quality of the subsequent AI model's "food" and is a key prerequisite for ensuring the model's accuracy and stability. Normalization can accelerate model convergence and prevent certain large-value features from dominating model training. The solution uses a classic CNN structure, treating the equipment's multivariate time series data (such as temperature, voltage, and vibration) as a "feature image" in the time dimension. CNN can automatically learn complex, non-linear correlation patterns between different parameters, thereby automatically identifying potential fault features, eliminating the tedious and expert-experience-dependent manual feature engineering of traditional methods. Long Short-Term Memory (LSTM) networks are used to control the forgetting, updating, and output of information, recording the evolution of the device's operating state over a long period, thereby capturing subtle, gradual performance degradation trends before failures occur. Ultimately, the model outputs the probability of future failures, achieving true predictive maintenance rather than simple fault detection. The K-Means clustering algorithm is used to cluster the high-dimensional fault feature vectors extracted by the CNN. This is used to discover the inherent structure of the data. Fault data with similar characteristics can be automatically grouped together, thereby identifying "high-frequency fault regions" and "high-incidence fault types."

[0020] However, the above-mentioned solutions have the following technical drawbacks: ① The computational overhead of training and inference using CNN and LSTM models is very high, posing a significant challenge for resource-constrained IoT terminal devices; ② The inherent limitations of the K-Means algorithm. Noise in the preprocessed data can significantly impact cluster center calculation. The algorithm tends to discover spherical clusters; for non-convex, complexly distributed fault clusters, K-Means may perform poorly.

[0021] To address this, this application provides a device failure prediction solution that can transform a massive cloud-based AI model into an "elite model" suitable for real-time operation on IoT terminals, solving the problems of computing resources and deployment costs; it can also optimize clustering algorithms to extract deeper and more valuable operation and maintenance data from complex data.

[0022] This invention discloses a method for predicting equipment failures. See [link to relevant documentation]. Figure 3 As shown, the method includes: Step S11: Obtain real-time operating data of the monitored external device.

[0023] The application scenarios of this invention are water affairs / water conservancy / industrial IoT: providing technical support for project operation and maintenance that can more economically and efficiently predict the failures of key equipment such as pumps, valves, motors, conveyors, and robotic arms. In one feasible implementation, this solution can be deployed on a telemetry terminal with data acquisition and edge computing capabilities. The telemetry terminal deploys a lightweight AI model on the terminal side to analyze the operating data of connected external devices (such as pumps, valves, sensors, motors, etc.) in real time, predict their failure tendencies in advance, and generate early warnings before failures occur. When predicting equipment failures, the real-time operating data of the monitored external devices is first acquired, such as vibration, temperature, current, and voltage time-series data. Through the analysis and processing of the real-time operating data of the monitored external devices, equipment failure prediction is achieved.

[0024] Step S12: The real-time operating data is processed using a lightweight neural network model to extract feature information representing the device status; the lightweight neural network model is obtained by knowledge distillation technology and trained based on a teacher model.

[0025] The current model training and inference processes are complex, posing a significant challenge to resource-constrained telemetry terminals. Therefore, in this embodiment, a large, complex "teacher model" is trained to guide a smaller, lighter "student model" in learning. This compresses the knowledge of the large model into the smaller one, facilitating edge deployment, reducing the number of parameters, minimizing accuracy loss, and improving inference speed. Then, the resulting lightweight neural network model is used to process the acquired real-time operational data, extracting feature information characterizing the device's operating status.

[0026] Specifically, a teacher model is constructed that includes a convolutional neural network and a long short-term memory network; a student model with fewer parameters than the teacher model is constructed, the student model including the convolutional neural network and the long short-term memory network; through knowledge distillation, the output of the teacher model is used to guide the training of the student model, and the trained student model is lightweighted to obtain the lightweight neural network model.

[0027] In this embodiment, a large, complex convolutional neural network (CNN) plus a large, complex long short-term memory (LSTM) network is used as the teacher model, while a lightweight CNN + lightweight LSTM architecture is designed as the student model. The teacher model is trained based on the training dataset, and then knowledge distillation is used to allow the student model to learn the teacher model's output, ensuring that both the CNN's feature maps and the LSTM's final predictions closely approximate the teacher model's output. Specifically, historical operating data and fault records from external devices are collected to form the model training dataset, with the parameter types of the historical and real-time operating data being consistent. Finally, lightweight optimization is performed on the student model to reduce computational complexity and resource consumption, resulting in a lightweight neural network model.

[0028] Step S13: Based on the feature information, perform fault tendency prediction through the lightweight neural network model to obtain a prediction result indicating the probability of the external device experiencing a target fault in the future.

[0029] Since the model needs to be deployed on the telemetry terminal, the trained lightweight neural network model (which may be pruned or quantized) is converted into a specialized edge inference format using a model conversion tool. This edge inference format lightweight neural network model is then deployed to the telemetry terminal. Model conversion tools include TensorFlow Lite Converter or ONNX Runtime; edge inference formats include .tflite or .onnx files. Simultaneously, an OTA (Over-The-Air) update mechanism is established so that when a better lightweight model is trained in the cloud, it can be securely and efficiently pushed to the edge device for iterative updates. Specifically, based on the OTA update mechanism, the updated lightweight neural network model is received from the cloud server to update the existing lightweight neural network model on the telemetry terminal.

[0030] In this embodiment, based on feature information extracted from real-time operational data, a lightweight neural network model is used to predict fault tendency, resulting in a direct classification prediction. It answers the question, "Based on the current data, what is the probability that the equipment will experience a specific type of fault in the future?", that is, which known fault type is most likely to correspond to the current state. For example, there is a 95% probability that the equipment will experience a bearing overheating fault.

[0031] Step S14: Input the feature information into a preset probability clustering model for cluster analysis to identify the state pattern category of the external device.

[0032] In this embodiment, feature information is input into a preset probabilistic clustering model for cluster analysis. The preset probabilistic clustering model employs a Gaussian Mixture Model (GMM). Its core idea is to assume that the data is generated by a mixture of multiple Gaussian distributions (i.e., clusters). It is a soft clustering algorithm where each point belongs to a cluster with a certain probability, better reflecting the complex situation in the real world where equipment failure modes may overlap. It answers the question: "Which typical state of the current device is most similar to in the past?" It identifies not specific faults, but rather the type or cluster of states. For example, it might cluster device states into ["healthy operating mode", "minor wear mode", "severe aging mode", "unknown abnormal mode"].

[0033] Step S15: Based on the prediction results and the state mode category, generate maintenance decision information for the external device.

[0034] Finally, by combining the prediction results of the lightweight neural network model with the state pattern categories obtained after clustering analysis by the preset probability clustering model, the fault tendency of the external device is comprehensively judged, and maintenance decision information for the external device is generated and output by the telemetry terminal.

[0035] Understandably, generating maintenance decision information is a process of information fusion and decision-making, comprehensively considering the outputs of the two models mentioned above: On the one hand, the risk level is judged based on the prediction results output by the lightweight neural network model. If the lightweight model predicts that the probability of a certain fault exceeds a preset threshold (such as 80%), an alarm is triggered. On the other hand, the current state is classified into a category by the preset probability clustering model, and the problem type is determined based on the state pattern category. If it is classified into the "severe aging mode," this corroborates the high fault probability. In this way, based on the above information, the system can generate accurate maintenance decisions, such as: "Alarm: High risk of motor bearing failure (85%), and the equipment is in 'severe aging mode.' Recommendation: Immediately arrange for bearing replacement and comprehensive overhaul." If only the predicted risk is high, but the state pattern is "healthy," the system may recommend "strengthening monitoring"; conversely, if the state pattern is "unknown anomaly," but the predicted risk is not high, the system may recommend "manual inspection." This cross-validation makes the decision-making more intelligent and reliable.

[0036] The beneficial technical effects of this application are as follows: By adopting a lightweight neural network model obtained through knowledge distillation, the ability to identify key fault features learned from complex teacher models is retained while significantly reducing the computational complexity and resource consumption of the model. This makes it possible to achieve real-time and efficient localized AI inference on resource-constrained telemetry terminals, effectively solving the bottleneck of the current difficulty in deploying the large model on edge nodes. Furthermore, by introducing a probabilistic clustering model that can output probability distributions to perform soft partitioning of feature information, compared with traditional hard clustering algorithms such as K-Means, it can capture the overlapping and gradual relationships between equipment states more precisely, thereby more accurately identifying "high-frequency fault areas" and "high-incidence fault types", improving the accuracy and robustness of fault state pattern detection. Finally, by combining the prediction results of the lightweight model with the deep pattern recognition results of probabilistic clustering analysis to generate maintenance decisions, an upgrade from "preventive maintenance" to more accurate "predictive maintenance" is achieved. While ensuring equipment reliability, the allocation of operation and maintenance resources is optimized, and the risk of unplanned downtime is reduced.

[0037] Based on the foregoing embodiments, this application provides a detailed description of the process of using the output of the teacher model to guide the training of the student model through knowledge distillation technology. The core idea is to enable the compact student model to learn the "knowledge" of the large teacher model, including the probability distribution of the final output (soft labels), the feature representations of intermediate layers (feature maps), and the temporal dynamics of the hidden states (for LSTM). The complete knowledge distillation process for CNN-LSTM models is as follows: Figure 4 As shown. Specifically, it includes the following steps: Step 1: Input the training dataset into the teacher model to obtain the teacher feature map, teacher soft label, and teacher hidden state output by the model; Step 2: Input the training dataset into the student model to obtain the student feature map, student soft label, and student hidden state output by the model; Step 3: Determine the losses between the teacher feature map and the student feature map, the teacher soft label and the student soft label, and the teacher hidden state and the student hidden state, respectively, to obtain the corresponding feature map loss, soft label loss and hidden state loss; Step 4: Perform a weighted combination of the feature map loss, the soft label loss, and the hidden state loss to determine the total loss function, and update the parameters of the student model through backpropagation.

[0038] In this embodiment, historical operating data and historical fault records of external devices are collected to form a training dataset for the model. The parameter types of the historical operating data and the real-time operating data are consistent. The training dataset is input into the teacher model to obtain the soft labels (probability distribution scaled by a temperature coefficient T), feature maps of the CNN intermediate layers, and the hidden states of the LSTM at each time step. Simultaneously, the student model input is aligned by inputting the same training dataset to obtain the corresponding soft labels, intermediate layer feature maps, and hidden states.

[0039] Furthermore, a total loss function is constructed by weighted combination of feature map loss, soft label loss, and hidden state loss. Backpropagation is used to update the student model parameters, and training is iterated until the total loss converges, ensuring that the student model's accuracy approximates that of the teacher model. Specifically, the feature map loss uses mean squared error (MSE) to measure the difference between the intermediate layer feature maps of the student and teacher model CNNs; the soft label loss uses KL divergence to measure the difference between the soft labels of the student and teacher models; and the hidden state loss uses cosine similarity to measure the difference between the hidden states of the student and teacher model LSTMs. A complete example is shown below:

[0040]

[0041]

[0042] Through the above examples, student models can achieve 90-95% of the accuracy of teacher models, reduce the number of parameters by 60-80%, and increase inference speed by 2-5 times.

[0043] In this embodiment, knowledge distillation technology is used to train a more streamlined student model based on a teacher model consisting of a large convolutional neural network (CNN) and a large long short-term memory (LSTM) network. The student model is lightweighted through at least one of the following methods: replacing its CNN portion with a depthwise separable convolutional structure, pruning its CNN and / or LSTM portions (channel / unit pruning), and performing low-bit quantization on its weights and activation values. Specifically, the steps include: The structural part of the convolutional neural network in the student model is subjected to a first lightweighting process, and the structural part of the long short-term memory network in the student model is subjected to a second lightweighting process. The first lightweighting process includes at least one of the following methods: replacing the backbone network of the convolutional neural network with a depth-separable convolutional structure, performing channel pruning on the convolutional layers of the convolutional neural network, and quantizing the weights and activation values ​​in the convolutional neural network. The second lightweighting process includes at least one of the following methods: pruning the hidden units of the Long Short-Term Memory (LSTM) network, performing low-rank decomposition on the weight matrix of the LSTM network, and quantizing the weights and activation values ​​of the LSTM network.

[0044] In the first specific implementation, the CNN part undergoes a first lightweighting process: (1) Replace the standard CNN backbone with a lightweight architecture designed for mobile devices, such as MobileNetV3 or SqueezeNet. Use techniques such as depthwise separable convolution to reduce the amount of convolution computation and achieve a balance between accuracy and speed.

[0045] (2) Channel pruning of the convolutional layers of a CNN, removing filters with small output activation values ​​(the entire channel is removed), offers a more practical speed improvement than weight pruning. The importance of filters is typically evaluated using the weight-based L1 norm, as shown in the following example:

[0046]

[0047]

[0048] This channel pruning method can reduce the number of parameters by 30-60%, increase inference speed by about 1.5-3 times, and after good fine-tuning, the accuracy loss is usually <2%. This method is particularly suitable for CNN models that need to be deployed on resource-constrained IoT devices, and can significantly reduce computational requirements and energy consumption.

[0049] (3) Perform INT8 quantization on the CNN part. Since it mainly involves convolution calculations, the speedup effect after quantization is very significant. INT8 quantization is the process of converting the weights and activation values ​​in the neural network from 32-bit floating-point numbers (FP32) to 8-bit integers (INT8).

[0050] Quantification formula: ; Inverse quantization formula: ; Where: scale is the scaling factor. , Zero is used to map the 0 point of FP32 to a certain value of INT8.

[0051] Taking vibration signals as an example, let's illustrate: Step 1: Create and train the FP32 model;

[0052]

[0053]

[0054] Step 2: Perform INT8 quantization after training;

[0055]

[0056] Step 4: Quantization model inference example;

[0057]

[0058] Through the complete INT8 quantization process described above, the size of the CNN model can be reduced by about 75% (FP32→INT8), the inference speed can be increased by 3-5 times (thanks to integer operations), memory usage can be significantly reduced, and power consumption can be significantly reduced.

[0059] In the second specific implementation, a second lightweighting process is applied to the LSTM portion: (1) Unit pruning: Reduce the model complexity and computational cost by decreasing the number of hidden units in the LSTM. Each LSTM unit contains four gating mechanisms (input gate, forget gate, output gate, and candidate memory unit). Reducing the number of hidden units will simultaneously reduce: the hidden state dimension, the cell state dimension, and the size of all related weight matrices. A complete example is as follows:

[0060]

[0061] By pruning LSTM cells, the number of parameters can be reduced by 25-30%, the inference speed can be increased by about 1.3-2.5 times, and the accuracy loss is usually <3% after good fine-tuning.

[0062] (2) Low-rank decomposition: Decomposing a large LSTM weight matrix into the product of two or more smaller matrices reduces the number of parameters and computational complexity. The principle is as follows: For a large matrix We can obtain the following through singular value decomposition: ;in , , , where r is the rank of the matrix. We can choose to keep the top k largest singular values ​​( ), to obtain the approximate matrix: ;in , .

[0063] The complete example is as follows:

[0064]

[0065] By using LSTM low-rank decomposition, the number of parameters can be reduced by 40-70%, the inference speed can be increased by about 1.5-3 times, and the accuracy loss is usually <5% after good fine-tuning.

[0066] (3) Quantization: Matrix operations in LSTM can also be quantized from FP32 to INT8 or FP16. This requires converting weights, activation values, and intermediate calculations from 32-bit floating-point numbers to 8-bit integers while maintaining numerical precision and model performance. An example is shown below:

[0067]

[0068]

[0069] By using LSTM INT8 quantization, the model size can be reduced by 75%, the inference speed can be increased by about 1.5-3 times, and the accuracy loss after training with good quantization awareness is usually <2%.

[0070] The specific process is as follows: Figure 5 As shown, a high-performance teacher model is first trained based on the original data. Then, knowledge distillation is used to transfer the predicted knowledge of the teacher model to a lighter-weight student model (such as a lightweight CNN and LSTM combined architecture) to optimize its feature extraction and classification capabilities. At the same time, redundant connections are removed by parameter pruning and weight precision is compressed by low-bit quantization to simplify the model. Finally, a real-time inference model adapted to edge devices is formed, which significantly reduces the consumption of computing resources while maintaining fault prediction accuracy comparable to the original model.

[0071] In summary, by employing lightweight techniques such as knowledge distillation, channel pruning, and low-bit quantization, the complex "teacher model" (large-scale CNN+LSTM) is compressed into a lightweight "student model." Through a systematic lightweighting and optimization process, a massive cloud-based AI model can be successfully transformed into an "elite model" suitable for real-time operation on IoT terminals. The number of model parameters is reduced by 60-80%, and the computational accuracy is upgraded from FP32 to INT8. The model size is reduced by approximately 75%, and the inference speed is increased by 3-5 times. This allows the algorithm to run in real-time on resource-constrained telemetry terminals, significantly reducing the deployment cost of edge devices and avoiding bandwidth consumption during data transmission to the cloud.

[0072] Because the K-Means algorithm used has limitations, it cannot efficiently identify "high-frequency fault areas" and "frequent fault types." Therefore, this application describes the workflow of an improved clustering analysis module in its embodiments. Figure 6 As shown, its working principle is as follows: The parameters (mean, variance, mixture weights) of each Gaussian component are learned through the Expectation-Maximization algorithm. Ultimately, each data point is assigned to the cluster with the highest posterior probability. This method provides the "probability of belonging to a certain cluster," rather than just a hard partition. For example, it can identify "this fault has a 70% probability of being type A and a 30% probability of being type B," providing richer information. The preset probabilistic clustering model is a Gaussian Mixture Model (GMM). Because its output is probabilistic, it can be easily integrated into subsequent probabilistic graphical models or decision systems. Specifically, the clustering analysis process using the preset probabilistic clustering model includes the following steps: The Gaussian mixture model is trained based on the feature information, and the number of Gaussian components in the Gaussian mixture model is determined. The parameters of multiple Gaussian components are learned using the expectation-maximization algorithm, and the posterior probability of each feature belonging to each Gaussian component is determined; the parameters include the mean vector, covariance matrix, and mixture weights. The state mode category of the external device is identified based on the posterior probability.

[0073] In this embodiment, the feature information extracted from the lightweight model is first input into the Gaussian Mixture Model (GMM) and standardized to eliminate dimensional differences. Then, based on the preprocessed feature vectors, GMM models with different numbers of clustering components are constructed. The optimal number of clustering components K is automatically selected using the Bayesian information criterion or the Akaike information criterion, i.e., determining the optimal number of Gaussian components in the Gaussian Mixture Model, avoiding manual specification. The parameters of each Gaussian component are obtained by training the GMM model. Components with large mixture weights correspond to "high-incidence fault types"; the mean of each component represents the core characteristics of that type of fault; and through the covariance matrix, the GMM can describe elliptical clusters rather than just spherical ones.

[0074] The probability density function of a Gaussian mixture model: ; in: : The mixture weights of the k-th Gaussian component ( ); : The mean vector of the k-th Gaussian component; The covariance matrix of the k-th Gaussian component; : Multivariate Gaussian distribution.

[0075] EM algorithm steps: E-step (expectation step): Calculate the posterior probability of each feature vector belonging to each cluster component, using the formula: ,in The mixing weights for the k-th cluster component are... It is the mean vector. Let covariance matrix be the variance matrix. It follows a multivariate Gaussian distribution; M-step (maximization step): Updates GMM parameters based on posterior probabilities. (N is the total number of eigenvectors); ; .

[0076] Repeat the E-step and M-step until the parameters converge to obtain the trained GMM model. Input the feature information extracted from the real-time running data into the trained GMM model, and output the posterior probability of each feature vector belonging to each cluster, as the soft clustering result. A complete example is as follows:

[0077]

[0078]

[0079]

[0080]

[0081]

[0082]

[0083] It should be pointed out that, in Figure 6In this study, three branches of cluster analysis work in parallel. Branch 1: Fault Mode Identification, quantifying fault uncertainty. This is the most basic application of Gaussian Mixture Model (GMM). Based on the principle of "maximum posterior probability," it assigns a label to each data point representing the most likely known fault (e.g., "bearing wear," "poor lubrication"), indicating which known fault the current data point most likely belongs to. Branch 2: High-Frequency Fault Analysis. This branch no longer focuses on individual data points but performs statistical analysis based on the "mixed weights" parameter of the GMM model. Gaussian components with high mixed weights represent fault modes that have occurred frequently in historical data. High-frequency fault modes are identified to guide the allocation of preventative maintenance resources. Branch 3: Anomaly Detection, discovering anomalous fault modes. This branch utilizes the "soft clustering" characteristic of GMM. If a data point has a low probability of belonging to all known clusters (Gaussian components), it is likely an anomaly that does not conform to any known pattern. This answers the question, "Does a new, unknown fault mode or anomaly exist?"

[0084] Furthermore, the step of generating maintenance decision information for the external device based on the prediction result and the state pattern category includes the following steps: High-frequency fault patterns are identified from the state pattern categories, and the high-frequency fault patterns are determined based on the mixing weight of each Gaussian component in the preset probability clustering model; An abnormal state that does not conform to any known fault mode is detected from the state mode category. The abnormal state is determined based on the fact that the posterior probability of the data point belonging to any Gaussian component is lower than a preset threshold. Based on the prediction results, the high-frequency fault modes, and the abnormal states, maintenance decision information for the external device is generated, and different priorities are assigned to the maintenance decision information.

[0085] In this embodiment, structured maintenance recommendations are generated based on prediction results, high-frequency failure modes, and abnormal states. In one feasible implementation, to guide maintenance personnel on how to allocate limited resources, the system needs to inform users which recommendation should be processed first. Therefore, recommendations are prioritized, with all maintenance recommendations sorted from highest to lowest priority. Recommendations for abnormal states are assigned the highest priority.

[0086] For example, for high-frequency failure modes, recommendations are generated that include priority, failure mode identifier, number of affected devices, confidence level, and preventive maintenance actions; for abnormal states, emergency recommendations are generated with the highest priority, identified as unknown modes, and immediate inspection actions.

[0087] In summary, by replacing traditional K-Means clustering with Gaussian Mixture Model (GMM) and automatically determining the optimal number of clusters using the BIC / AIC criteria, GMM supports non-spherical cluster identification and probabilistic soft clustering, and can automatically detect abnormal patterns. Therefore, fault classification accuracy is improved, and complex fault morphologies (such as elliptical clusters) can be identified. Simultaneously, it provides probabilistic confidence levels for fault attribution (e.g., "70% is a Class A fault"), providing richer and more reliable data for maintenance decisions. Based on prediction results, high-frequency fault modes, and abnormal states, maintenance decision information for the external equipment is generated, ultimately achieving advantages such as accurate prediction, efficient operation, and flexible adaptation, thereby improving the overall stability of the system.

[0088] Accordingly, this application also discloses a device for predicting equipment failures, see [link to relevant documentation]. Figure 7 As shown, the device includes: Data acquisition module 11 is used to acquire real-time operating data of the monitored external device; The feature extraction module 12 is used to process the real-time operating data using a lightweight neural network model to extract feature information representing the device status; the lightweight neural network model is obtained by knowledge distillation technology and trained based on a teacher model. The prediction result output module 13 is used to perform fault tendency prediction based on the feature information and through the lightweight neural network model to obtain a prediction result indicating the probability of the external device experiencing a target fault in the future. The state pattern category identification module 14 is used to input the feature information into a preset probability clustering model for cluster analysis in order to identify the state pattern category of the external device. The maintenance decision information generation module 15 is used to generate maintenance decision information for the external device based on the prediction results and the state mode category.

[0089] For more detailed information on the working process of each of the above modules, please refer to the relevant content disclosed in the foregoing embodiments, which will not be repeated here.

[0090] Therefore, the above-described solution in this embodiment, by employing a lightweight neural network model obtained through knowledge distillation, significantly reduces the computational complexity and resource consumption of the model while retaining the ability to identify key fault features learned from complex teacher models. This makes it possible to achieve real-time and efficient localized AI inference on resource-constrained telemetry terminals, effectively solving the bottleneck of the current difficulty in deploying the large model on edge nodes. Furthermore, by introducing a probabilistic clustering model that can output probability distributions to perform soft partitioning of feature information, compared with traditional hard clustering algorithms such as K-Means, it can capture the overlapping and gradual relationships between equipment states more precisely, thereby more accurately identifying "high-frequency fault areas" and "high-incidence fault types," improving the accuracy and robustness of fault state pattern detection. Finally, by combining the prediction results of the lightweight model with the deep pattern recognition results of probabilistic clustering analysis to generate maintenance decisions, an upgrade from "preventive maintenance" to more accurate "predictive maintenance" is achieved. While ensuring equipment reliability, this optimizes the allocation of operation and maintenance resources and reduces the risk of unplanned downtime.

[0091] Furthermore, embodiments of this application also disclose an electronic device, Figure 8 This is a structural diagram of an electronic device 20 according to an exemplary embodiment. The content of the diagram should not be construed as limiting the scope of this application.

[0092] Figure 8 This is a schematic diagram of the structure of an electronic device 20 provided in an embodiment of this application. Specifically, the electronic device 20 may include: at least one processor 21, at least one memory 22, a power supply 23, a communication interface 24, an input / output interface 25, and a communication bus 26. The memory 22 stores a computer program, which is loaded and executed by the processor 21 to implement the relevant steps in the device fault prediction method disclosed in any of the foregoing embodiments. Furthermore, the electronic device 20 in this embodiment may specifically be a telemetry terminal.

[0093] In this embodiment, the power supply 23 is used to provide operating voltage for each hardware device on the electronic device 20; the communication interface 24 can create a data transmission channel between the electronic device 20 and external devices, and the communication protocol it follows can be any communication protocol applicable to the technical solution of this application, and is not specifically limited here; the input / output interface 25 is used to acquire external input data or output data to the outside world, and its specific interface type can be selected according to specific application needs, and is not specifically limited here.

[0094] In addition, the memory 22, as a carrier for resource storage, can be a read-only memory, random access memory, disk, or optical disk, etc. The resources stored on it can include an operating system 221, computer programs 222, and data 223, etc. The data 223 can include various types of data. The storage method can be temporary storage or permanent storage.

[0095] The operating system 221 is used to manage and control the various hardware devices on the electronic device 20 and the computer program 222, which may be Windows Server, Netware, Unix, Linux, etc. In addition to including a computer program capable of performing the device failure prediction method executed by the electronic device 20 as disclosed in any of the foregoing embodiments, the computer program 222 may further include a computer program capable of performing other specific tasks.

[0096] Furthermore, this application also discloses a computer-readable storage medium, which includes random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disks, magnetic disks, optical disks, or any other form of storage medium known in the art. The computer program, when executed by a processor, implements the aforementioned device fault prediction method. Specific steps of this method can be found in the corresponding content disclosed in the foregoing embodiments, and will not be repeated here.

[0097] Furthermore, embodiments of this application also provide a computer program product, including a computer program / instructions, which, when executed by a processor, implements any of the above-described methods for predicting device faults.

[0098] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since it corresponds to the method disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to in the method section.

[0099] The steps of the device failure prediction method or algorithm described in conjunction with the embodiments disclosed herein can be implemented directly by hardware, a software module executed by a processor, or a combination of both. The software module can be located in random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art.

[0100] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0101] The above provides a detailed description of the equipment fault prediction provided by the present invention. Specific examples have been used to illustrate the principles and implementation methods of the present invention. The descriptions of the above embodiments are only intended to help understand the method and core ideas of the present invention. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of the present invention. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A method for predicting equipment failure, characterized in that, include: Acquire real-time operating data of the monitored external devices; The real-time operating data is processed using a lightweight neural network model to extract feature information representing the device status; The lightweight neural network model is obtained by training a teacher model using knowledge distillation techniques. Based on the feature information, a fault tendency prediction is performed using the lightweight neural network model to obtain a prediction result indicating the probability of the external device experiencing a target fault in the future. The feature information is input into a preset probability clustering model for cluster analysis to identify the state pattern category of the external device; Based on the prediction results and the state pattern category, maintenance decision information for the external device is generated.

2. The equipment failure prediction method according to claim 1, characterized in that, Constructing the lightweight neural network model includes: Construct a teacher model that incorporates convolutional neural networks and long short-term memory networks; Construct a student model with fewer parameters than the teacher model, wherein the student model includes the convolutional neural network and the long short-term memory network; The teacher model's output is used to guide the training of the student model through knowledge distillation, and the trained student model is then lightweighted to obtain the lightweight neural network model.

3. The equipment failure prediction method according to claim 2, characterized in that, The method of using knowledge distillation to guide the training of the student model using the output of the teacher model includes: The training dataset is input into the teacher model to obtain the teacher feature map, teacher soft label, and teacher hidden state output by the model. Input the training dataset into the student model to obtain the student feature map, student soft label, and student hidden state output by the model; The losses between the teacher feature map and the student feature map, the teacher soft label and the student soft label, and the teacher hidden state and the student hidden state are determined respectively to obtain the corresponding feature map loss, soft label loss and hidden state loss; The feature map loss, the soft label loss, and the hidden state loss are weighted and combined to determine the total loss function, and the parameters of the student model are updated through backpropagation.

4. The equipment failure prediction method according to claim 2, characterized in that, The trained student model is lightweighted, including: The structural part of the convolutional neural network in the student model is subjected to a first lightweighting process, and the structural part of the long short-term memory network in the student model is subjected to a second lightweighting process. The first lightweighting process includes at least one of the following methods: replacing the backbone network of the convolutional neural network with a depth-separable convolutional structure, performing channel pruning on the convolutional layers of the convolutional neural network, and quantizing the weights and activation values ​​in the convolutional neural network. The second lightweighting process includes at least one of the following methods: pruning the hidden units of the Long Short-Term Memory (LSTM) network, performing low-rank decomposition on the weight matrix of the LSTM network, and quantizing the weights and activation values ​​of the LSTM network.

5. The equipment failure prediction method according to claim 1, characterized in that, The preset probability clustering model is a Gaussian mixture model. The step of inputting the feature information into the preset probability clustering model for cluster analysis to identify the state pattern category of the external device includes: The Gaussian mixture model is trained based on the feature information, and the number of Gaussian components in the Gaussian mixture model is determined. The parameters of multiple Gaussian components are learned using the expectation-maximization algorithm, and the posterior probability of each feature belonging to each Gaussian component is determined; the parameters include the mean vector, covariance matrix, and mixture weights. The state mode category of the external device is identified based on the posterior probability.

6. The equipment failure prediction method according to claim 1, characterized in that, Before performing fault tendency prediction using the lightweight neural network model based on the feature information, the method further includes: The lightweight neural network model is converted into an edge inference format using a model conversion tool, and the lightweight neural network model in the edge inference format is deployed to the telemetry terminal. An OTA update mechanism is established, and an updated lightweight neural network model is received from the cloud server based on the OTA update mechanism to update the original lightweight neural network model on the telemetry terminal.

7. The equipment failure prediction method according to any one of claims 1 to 6, characterized in that, The step of generating maintenance decision information for the external device based on the prediction result and the state pattern category includes: High-frequency fault patterns are identified from the state pattern categories, and the high-frequency fault patterns are determined based on the mixing weight of each Gaussian component in the preset probability clustering model. An abnormal state that does not conform to any known fault mode is detected from the state mode category. The abnormal state is determined based on the fact that the posterior probability of the data point belonging to any Gaussian component is lower than a preset threshold. Based on the prediction results, the high-frequency fault modes, and the abnormal states, maintenance decision information for the external device is generated, and different priorities are assigned to the maintenance decision information.

8. A device for predicting equipment failures, characterized in that, include: The data acquisition module is used to acquire real-time operating data of the monitored external devices; The feature extraction module is used to process the real-time operating data using a lightweight neural network model to extract feature information representing the device status; the lightweight neural network model is obtained by knowledge distillation technology and trained based on a teacher model. The prediction result output module is used to perform fault tendency prediction based on the feature information and through the lightweight neural network model to obtain a prediction result indicating the probability of the external device experiencing a target fault in the future. The state pattern category recognition module is used to input the feature information into a preset probability clustering model for cluster analysis in order to identify the state pattern category of the external device. The maintenance decision information generation module is used to generate maintenance decision information for the external device based on the prediction results and the state mode category.

9. An electronic device, characterized in that, The electronic device includes a processor and a memory; wherein the memory is used to store a computer program, which is loaded and executed by the processor to implement the device fault prediction method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, Used to store a computer program; wherein the computer program, when executed by a processor, implements the device failure prediction method as described in any one of claims 1 to 7.

Images