Multi-dimensional knowledge distillation compression method of cloud-side collaborative fault diagnosis model

Abstract

The invention relates to a multi-dimensional knowledge distillation compression method of a cloud-side collaborative fault diagnosis model, and belongs to the technical field of mechanical part monitoring and fault diagnosis. The method comprises the following steps: collecting multi-measurement-point vibration data through a sensor, and carrying out lightweight compression on the multi-measurement-point vibration data by adopting a measurement point optimization method; a cloud fault diagnosis model and an equipment side fault diagnosis model are established, the cloud fault diagnosis model compresses the network model by adopting a multi-dimensional knowledge distillation mode, lightweight network parameters are obtained, and the network parameters are transmitted to the equipment side fault diagnosis model; and a diagnosis result output by the equipment side fault diagnosis model is transmitted to the cloud platform and the local monitoring terminal, and the running state of the equipment is monitored in real time through interaction between the mobile terminal and the cloud platform. According to the lightweight method provided by the invention, the fault diagnosis model has better cross-platform compatibility and can be more easily deployed on different types of equipment, which is of great significance to the wide application and popularization of the fault diagnosis system.

Description

Technical Field

[0001] The present invention belongs to the technical field of mechanical component monitoring and fault diagnosis, and relates to a multi-dimensional knowledge distillation and compression method for a cloud-edge collaborative fault diagnosis model. Background Art

[0002] Cloud-edge collaborative fault diagnosis is an efficient and real-time fault diagnosis technology, which is crucial for ensuring the stable operation of mechanical equipment, preventing accidents in a timely manner, optimizing maintenance strategies, and promoting the intelligentization of the production process. For the traditional mechanical equipment monitoring system based on the cloud center, the massive sensor data uploaded places a huge burden on the cloud server, and directly processing and identifying these data on the cloud server poses high requirements on the data storage capacity, computing power, and energy consumption of the server [3]. Therefore, considering the requirements of real-time performance and economy of fault identification, the edge-cloud collaborative state monitoring system has emerged as an important method for realizing the intelligent monitoring and fault diagnosis of mechanical equipment. The main reasons for the cloud-edge collaborative fault diagnosis system being unable to timely feedback the real-time operation status of mechanical equipment are slow test data transmission and a large number of diagnostic model parameters. The edge-cloud collaborative state monitoring system needs to arrange corresponding edge devices near the equipment to analyze the data sent by the sensors at the edge end, and conduct real-time monitoring and fault diagnosis on the operation status of the equipment. The cloud-edge collaborative fault diagnosis system processes the original collected data at the edge end, uploads the fault features to the cloud, and receives the fault diagnosis network model sent from the cloud.

[0003] However, edge devices usually have small storage space and weak computing power, which limits the application of some large-scale networks with good fault identification capabilities. Network model compression means minimizing the number of network parameters and layers without significantly reducing the recognition accuracy, thereby improving the efficiency of fault diagnosis and reducing the dependence on hardware. The lightweight model has lower requirements for hardware performance, and enterprises can deploy and operate the cloud-edge collaborative fault diagnosis system without purchasing high-performance hardware devices. Summary of the Invention

[0004] In view of this, the purpose of the present invention is to provide a multi-dimensional knowledge distillation compression method for a cloud-edge collaborative fault diagnosis model, which mainly includes two parts: lightweight measurement point data and lightweight network parameters. First, vibration data of multiple measurement points are collected by multiple sensors, and then the measurement point optimization method is used to compress the collected data to remove the redundancy of the data. Then, the optimized data is uploaded to the cloud for network training of the teacher fault diagnosis model, and the network model is compressed using the multi-dimensional knowledge distillation method to achieve network lightweighting. The compressed network parameters are transmitted to the fault diagnosis model on the edge side through remote Over-the-Air Technology (OTA). Finally, the fault diagnosis results are uploaded to the cloud platform and the local monitoring end, where the network model on the cloud is only used to train and update the edge fault diagnosis model parameters, and is not used for equipment operation status monitoring.

[0005] In order to achieve the above object, the present invention provides the following technical solutions:

[0006] A multidimensional knowledge distillation and compression method for a cloud-edge collaborative fault diagnosis model, comprising the following steps:

[0007] S1. Collect vibration data from multiple measuring points through sensors and use the measuring point optimization method to perform lightweight compression on the vibration data from multiple measuring points;

[0008] S2. Establish a cloud-side fault diagnosis model and a device-side fault diagnosis model respectively. The cloud-side fault diagnosis model compresses the network model using multi-dimensional knowledge distillation to obtain lightweight network parameters, and then transmits the network parameters to the device-side fault diagnosis model.

[0009] S3. The diagnostic results output by the device-side fault diagnosis model are transmitted to the cloud platform and the local monitoring terminal, and the device operation status is monitored in real time through interaction between the mobile terminal and the cloud platform.

[0010] Furthermore, in step S1, the following steps are included:

[0011] S11, multi-point vibration data collection: Vibration data is collected by setting vibration sensors at corresponding measuring points to obtain the original signal data point set P = {P1, P2, ..., P n}, where P1, P2, ..., P n For different measuring points;

[0012] S12. Establish the measurement point redundancy evaluation factor MI of all measurement point data in the original signal data point set i ;

[0013] S13, quantitatively evaluate the fault recognition accuracy ACC of the measurement point through the multi-information source fault feature extraction unit i ;

[0014] S14, based on the redundancy evaluation factor MI i And fault identification accuracy ACC i Establish the measurement point importance evaluation factor λ i ;

[0015] S15, perform lightweight processing on the original data point set P through the backward search algorithm and the measurement point importance evaluation factor, and output the optimal test point set P * .

[0016] Furthermore, in step S12, the following steps are specifically included:

[0017] S121. The expected amount of information is used as the redundancy evaluation index of the measurement point to describe the measurement of the amount of information required to eliminate system uncertainty. It is defined as follows:

[0018]

[0019] Where p(x) is the prior probability of event x occurring, and X is the set of all values of x;

[0020] S122. Use conditional entropy to analyze the relationship between multiple variables. For random variables X and Y, the conditional entropy is defined as follows:

[0021]

[0022] Where p(x,y) is the joint probability density function of the two variables, and p(x|y) is the posterior probability of x occurring under the condition that y occurs;

[0023] S123. Mutual information is used to characterize the redundant information between different measurement points and quantitatively describe the correlation between two variables. The calculation formula is as follows:

[0024]

[0025] The larger the mutual information value, the stronger the correlation between the two variables x and y;

[0026] S124. Calculate the fault isolation redundancy factor of the measuring point and the shared information of the measuring point. The fault isolation redundancy factor of the measuring point refers to the mutual information between different fault states of the same measuring point. The calculation formula for the fault isolation redundancy factor of the measuring point i is:

[0027]

[0028] Where k and j represent the fault type number, n is the number of fault status types, and are the data collected by measuring point i under fault state k and fault state j respectively;

[0029] The common information volume of measuring points refers to the common information between the data collected by different measuring points under the same fault. The calculation formula of the common information volume of measuring point i is:

[0030]

[0031] Where k represents the fault type number, j is the measurement point number, n is the number of fault state types, and m is the number of measurement points. and The data collected at measuring point i and measuring point j respectively under fault state k;

[0032] S125. Measure the redundancy of the measuring points based on their isolation redundancy factor and the amount of public information, and calculate the redundancy evaluation factor for the measuring points:

[0033] MI i =I i / M i

[0034] Among them, MI i is the redundancy evaluation factor of the measuring point, M i is the fault isolation redundancy factor of the measuring point, I i is the public information volume of the measuring point.

[0035] Furthermore, in step S13, the following steps are specifically included:

[0036] S131. Determine the fault category based on the fault probabilities output by the fault diagnosis network. The expression for the fault category is defined as:

[0037] Z=[Max(O1),Max(O2),...,Max(O L )]

[0038]

[0039] Where O=[O1,O2,...,O L ] is the output of the network, L is the number of recognized categories, if Z is the same as the true type label, the recognition is correct;

[0040] S132. The calculation formula for fault identification accuracy is as follows:

[0041]

[0042] Where Y is the true label of the sample, k is the sample number, and K is the total number of samples.

[0043] Furthermore, in step S14, the importance evaluation factor λ of the measuring point is i The calculation formula is:

[0044] λi = MI i + ACC i

[0045] where i is the measurement point number, and MI i is the measurement point redundancy evaluation factor of measurement point i, and ACC i is the fault identification accuracy rate of measurement point i.

[0046] Furthermore, in step S15, it specifically includes the following steps:

[0047] S151. First, arrange all measurement points in ascending order according to the measurement point importance evaluation factor to obtain the arranged measurement point set P' = {P'1, P'2..., P' n};

[0048] S152. Sequentially delete the measurement points with low importance coefficients from P' to obtain a new measurement point subset P * = {P1, P2,... P m};

[0049] S153. Use the multi-information fusion fault identification network model to evaluate the effectiveness of the measurement point subset P * ;

[0050] S154. If the identification accuracy rate is higher than the preset threshold, then the current measurement point subset P * is the optimal test point set; if the identification accuracy rate is lower than the preset threshold, then bring P * into step S151 and repeat the execution until the optimal test point set is output.

[0051] Furthermore, in step S2, it specifically includes the following steps:

[0052] S21. The constructed cloud fault diagnosis model includes the parameters of each student network Sub-model-V1, Sub-model-V2,..., Sub-model-V, and each teacher network teacher-V1, teacher-V2,..., teacher-V N . Train the student network SV through knowledge distillation:

[0053] S22. Calculate the knowledge distillation error of each student network. If the error meets the termination condition, the model structure compression is completed; otherwise, perform error backpropagation to calculate the error of each layer, update the student network parameters, and then return to S21;

[0054] S23. Quantize the parameters of the trained lightweight network model and transfer the quantized content to the device-side fault diagnosis model.

[0055] Further, in step S21, first, the teacher model is trained using the dataset, then the parameters of the trained teacher model are fixed. Then, the dataset is input into both the teacher model and the student model simultaneously to obtain the output of the teacher and the output of the student. The outputs of both sides are divided by the distillation temperature T and processed using the Softmax function to obtain the soft labels. The formula for the soft labels is as follows:

[0056]

[0057] where y k is the output value of the k-th node in the output layer; L is the number of output nodes.

[0058] Further, in step S22, the loss of the student model in knowledge distillation includes the cross-entropy loss between the soft labels of the student model and the teacher model, and the cross-entropy loss function between the output of the student model and the true labels. Then, the total loss function of the student model is as follows:

[0059]

[0060] where w are the weight parameters of the student model, Z k is the true label, O k is the actual output value of the student model, L is the number of output classes, is the soft label of the teacher model, is the soft label of the student model; the student model performs backpropagation according to this total loss function during training to update the parameters of the student model.

[0061] Further, in step S23, the original floating-point type parameters are mapped into lower-byte type parameters for model parameter quantization. The mapping formula from high-precision parameters to low-precision parameters is as follows:

[0062] Q = R / S + Z

[0063] S = (R max - R min ) / (Q max - Q min )

[0064] Z = Q max - R max / S

[0065] where R represents the input high-precision parameter, Q represents the quantized low-precision result, S is the scale factor, representing the smallest scale that can be represented after fixed-point quantization, R max and R min are the maximum and minimum values of the high-precision parameter, Q max and Q minLet \(a\) and \(b\) be the maximum and minimum values of the quantized low-precision data type, and \(Z\) be the offset value, which represents the quantized fixed-point value corresponding to the value \(0\).

[0066] The KL divergence is used as the loss function for the quantization process, and the formula is as follows:

[0067] \(D\) KL =\(\sum_{i}P(i)\times\log\frac{P(i)}{M(i)}\)

[0068] where \(P\) is the probability distribution of \(Q\), and \(M\) is the probability distribution of \(R\).

[0069] The beneficial effects of the present invention are as follows:

[0070] First, the present invention formulates an evaluation method for the importance evaluation factors of measurement points based on the redundancy of measurement points and the fault diagnosability. This method can effectively evaluate the importance of each measurement point for fault diagnosis, thereby significantly reducing the scale of test data. Through this method, not only can the cost and complexity of data processing be reduced, but also the subsequent data analysis process can be greatly simplified, improving the overall efficiency. In addition, this method is particularly suitable for applications in edge computing environments because it can effectively solve the problem of difficult deployment of diagnostic models at the edge due to large amounts of data.

[0071] Second, the present invention adopts a multi-dimensional knowledge distillation method to reduce the number of network parameters while ensuring that the recognition accuracy is not affected. Knowledge distillation is a technique for transferring the knowledge of a large and complex model (teacher model) to a small and more efficient model (student model). In this process, the present invention not only focuses on compressing the network parameters, but also endeavors to optimize the model structure to adapt to the limited computing resources of edge devices. In this way, the required storage space and computing power can be significantly reduced without sacrificing the classification accuracy, improving the efficiency of edge diagnosis.

[0072] Finally, the present invention proposes a cloud-edge collaborative fault diagnosis mode, that is, training the model in the cloud and performing the fault diagnosis task at the edge. This mode utilizes the powerful computing power and rich data resources in the cloud for model training, and then deploys the trained lightweight model to edge devices. The advantage of this is that it reduces the requirements for the computing power of edge devices, enabling real-time and efficient fault diagnosis even in resource-constrained environments. This is particularly important for industrial application scenarios that require high reliability and real-time response, and also greatly expands the application scope of deep learning-based fault diagnosis methods.

[0073] Other advantages, objects, and features of the present invention will be set forth in part in the following description, and in part will be obvious to those skilled in the art upon examination of the following, or may be learned from the practice of the present invention. The objects and other advantages of the present invention may be realized and obtained by the following description. Brief Description of the Drawings

[0074] In order to make the objectives, technical solutions, and advantages of the present invention clearer, the present invention will be described in detail with reference to the accompanying drawings, where:

[0075] Figure 1 is the overall process framework of a multi-dimensional knowledge distillation and compression method for a cloud-edge collaborative fault diagnosis model;

[0076] Figure 2 is a schematic diagram of the process for lightweight measurement point data;

[0077] Figure 3 is a schematic diagram of the process for lightweight network model parameters;

[0078] Figure 4 is a schematic diagram of the knowledge distillation training process of the cloud fault diagnosis model;

[0079] Figure 5 is a schematic diagram of the quantization process of the optimized student model parameters. Detailed Embodiments

[0080] The following specific examples illustrate the embodiments of the present invention. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and various details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the drawings provided in the following examples only illustrate the basic concept of the present invention schematically, and the following examples and the features in the examples can be combined with each other without conflict.

[0081] Among them, the drawings are only for illustrative purposes, showing only schematic diagrams, not physical diagrams, and should not be construed as a limitation to the present invention; for better illustrating the embodiments of the present invention, some components in the drawings will be omitted, enlarged, or reduced, which does not represent the size of the actual product; for those skilled in the art, it is understandable that some well-known structures and their descriptions in the drawings may be omitted.

[0082] In the accompanying drawings of the embodiments of the present invention, the same or similar reference numerals correspond to the same or similar components; in the description of the present invention, it should be understood that if there are terms such as "upper", "lower", "left", "right", "front", "rear", etc. indicating the orientation or positional relationship, they are based on the orientation or positional relationship shown in the accompanying drawings. This is only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation. Therefore, the terms describing the positional relationship in the accompanying drawings are only for illustrative purposes and should not be construed as a limitation of the present invention. For those of ordinary skill in the art, the specific meanings of the above terms can be understood according to specific circumstances.

[0083] Please refer to Figures 1 to 5 , which is a multi-dimensional knowledge distillation and compression method for a cloud-edge collaborative fault diagnosis model.

[0084] Embodiment

[0085] This embodiment proposes a multi-dimensional knowledge distillation and compression method for a cloud-edge collaborative fault diagnosis model, as shown in Figure 1 and includes the following steps:

[0086] S1. Collect multi-point vibration data through sensors, and perform lightweight compression on the multi-point vibration data using a measurement point optimization method;

[0087] S2. Establish a cloud fault diagnosis model and a device-side fault diagnosis model respectively. Among them, the cloud fault diagnosis model compresses the network model in a multi-dimensional knowledge distillation manner, obtains lightweight network parameters, and transmits the network parameters to the device-side fault diagnosis model;

[0088] S3. Transmit the diagnostic results output by the device-side fault diagnosis model to the cloud platform and the local monitoring terminal, and interact with the cloud platform through the mobile terminal to monitor the operation status of the device in real time.

[0089] In this embodiment, as shown in Figure 2 , step S1 specifically includes:

[0090] S11. Multi-point vibration data collection: Collect vibration data through vibration sensors arranged at the corresponding measurement points to obtain the original signal data point set P = {P1, P2,..., P n}), where P1, P2,..., P n are different measurement points.

[0091] In the original signal data point set, there are measurement points that are irrelevant or redundant to the fault. The measurement points that are irrelevant or redundant to the fault not only have little gain on the fault recognition effect, but also reduce the recognition efficiency and increase the model complexity. Therefore, in the subsequent steps, it is necessary to remove the measurement points that are irrelevant or redundant to the fault to reduce the data volume and improve the recognition efficiency.

[0092] S12. Establish the measurement point redundancy evaluation factors for all measurement point data in the original signal data set, including:

[0093] S121. Based on mutual information, propose a redundancy evaluation index for measurement points. Information entropy is the expectation of the amount of information, which can be used to describe the measure of the amount of information required to eliminate the uncertainty of the system. Its definition is as follows:

[0094]

[0095] where p(x) is the probability of event x occurring (prior probability), and X is the set of all values of x.

[0096] S122. To analyze the relationship between multiple variables, conditional entropy is proposed. For random variables X and Y, its conditional entropy is defined as follows:

[0097]

[0098] where p(x,y) is the joint probability density function of the two variables, and p(x|y) is the probability of x occurring under the condition that y occurs (posterior probability).

[0099] S123. To quantitatively describe the correlation between two variables, considering that the traditional correlation coefficient can only describe linear relationships and is insufficient for non-linear expressions, mutual information (MI) is used to characterize the redundant information between different measurement points. Its calculation formula is as follows:

[0100]

[0101] When there is a deterministic relationship between variables x and y, the mutual information value reaches the maximum. When the two variables are completely uncorrelated, the mutual information is 0 at this time, which indicates that the greater the mutual information value, the stronger the correlation between the two variables.

[0102] S124. Calculate the measurement point fault isolation redundancy factor and the measurement point common information volume: For the same measurement point, the less the same information between the data collected in various fault states, the better. The response of this measurement point to different faults is more different and sensitive. Therefore, the better the isolation of this measurement point for faults, that is, the lower the misdiagnosis rate. In the present invention, the mutual information between different fault states of the same measurement point is named the measurement point fault isolation redundancy factor. The calculation formula for the measurement point fault isolation redundancy factor of measurement point i is written as follows:

[0103]

[0104] where k and j represent the serial numbers of fault types, and n is the number of fault state types. and Data collected for measuring point i under fault state k and fault state j respectively.

[0105] For the data collected for different measuring points under the same fault, the less the same information, the better, because this indicates that the redundancy of each measuring point is lower; if the mutual information value between a certain measuring point and other measuring points is larger, it indicates that the measuring point contains more common information, and the applicability of the measuring point is better. The calculation formula for the common information amount of measuring point i is defined as follows:

[0106]

[0107] where k represents the fault type serial number, j is the measuring point number, n is the number of fault state types, m is the number of measuring points, and are the data collected for measuring point i and measuring point j respectively under fault state k.

[0108] S125. The isolation redundancy factor and common information amount of the measuring point jointly measure the redundancy of the measuring point. Therefore, the calculation formula for the measuring point redundancy evaluation factor is defined as follows:

[0109] MI i = I i / M i

[0110] S13. In order to quantitatively evaluate the fault diagnosability of the measuring point and avoid the influence of artificially extracting features, a multi-information source fault feature extraction unit is used to quantitatively evaluate the fault diagnosability of the measuring point, which includes:

[0111] S131. Determine its fault category through the fault probabilities output by the fault diagnosis network. The expression of the fault category is defined as follows:

[0112] Z = [Max(O1), Max(O2),..., Max(O L )]

[0113]

[0114] where O = [O1, O2,..., O L is the output of the network, and L is the number of recognition categories. If Z is the same as the true type label, the recognition is correct.

[0115] S132. The calculation formula for the fault recognition accuracy rate is as follows:

[0116]

[0117]

[0118] Where Y is the true label of the sample, k is the sample serial number, and K is the total number of samples.

[0119] S14. Based on the above evaluations of the redundancy and diagnosability of the measurement points, they are now constructed into an evaluation factor for the importance of measurement points, and its formula is as follows:

[0120] λ i =MI i +ACC i

[0121] Where i is the measurement point number, MI i is the evaluation factor for the redundancy of measurement point i, and ACC i is the fault recognition accuracy rate of measurement point i.

[0122] S15. Through the backward search algorithm and the evaluation factor for the importance of measurement points, the time complexity is reduced from O(m×n 2 ) to O(m×n), which includes:

[0123] S151. First, all measurement points are arranged in ascending order according to the evaluation factor for the importance of measurement points to obtain the arranged measurement point set P′={P′1, P′2..., P′ n};

[0124] S152. Successively delete the measurement points with low importance coefficients from P′ to obtain a new measurement point subset P * ={P1, P2,... P m};

[0125] S153. Use the multi-information fusion fault recognition network model to evaluate the effectiveness of the measurement point subset P * ;

[0126] S154. If the recognition accuracy rate is higher than the preset threshold, then the current measurement point subset P * is the optimal test point set; if the recognition accuracy rate is lower than the preset threshold, then P * is brought into step S151 to be repeatedly executed until the optimal test point set is output.

[0127] In this embodiment, in step S2, a cloud fault diagnosis model and a device-side fault diagnosis model are first established. The lightweight processed optimal measurement point data set is respectively input into the cloud fault diagnosis model and the device-side fault diagnosis model. Among them, the cloud fault diagnosis model trains the model through knowledge distillation to obtain the compressed model network parameters, and transmits the compressed model network parameters to the device-side fault diagnosis model. The device-side fault diagnosis model uses the compressed network parameters to perform fault recognition on the optimal measurement point data set and outputs the recognition result.

[0128] Specifically, as Figure 3 As shown in the figure, step S2 includes the following steps:

[0129] S21. In the constructed cloud fault diagnosis model, each student network Sub-model-V1, Sub-model-V2, …, Sub-model-V, and each teacher network teacher-V1, teacher-V2, …, teacher-V are included N For the parameters of, the student network SV is trained by means of knowledge distillation:

[0130] Knowledge distillation is a machine learning technique aimed at training a smaller model (student model) to imitate or replicate the knowledge of a larger model (teacher model). This process involves learning information from the teacher model and transferring it to the student model while maintaining or improving the performance of the student model. The core of knowledge distillation lies in how to effectively transfer the knowledge of the teacher model to the student model without significant performance degradation. Knowledge distillation can be divided into response-based, feature-based, and relation-based knowledge forms. Response-based knowledge usually refers to the neural responses of the last output layer of the teacher model, while feature-based knowledge involves the feature representations output by the teacher model. Relation-based knowledge involves knowledge transfer between models. The general framework of knowledge distillation includes three key parts: knowledge, distillation algorithm, and teacher-student structure. During the distillation process, the teacher model usually "distills" knowledge by controlling the "temperature", and this temperature factor can control the weight of each knowledge point, enabling the student model to learn the "dark knowledge" of the teacher model. After the structure of the network model is compressed, if the compressed model is directly used for training, since its parameters and number of layers are less than those of the original model, it is difficult to train to a performance similar to that of the original network. To prevent the accuracy of the compressed model from dropping too much, the knowledge distillation method can be used to transfer the knowledge learned by the original model to the compressed model. Generally, the original model is called the teacher model, and the compressed model is called the student model.

[0131] Specifically, the process of knowledge distillation is as Figure 4 shown.

[0132] First, the teacher model is trained using the dataset, then the parameters of the trained teacher model are fixed. Then, the dataset is input into both the teacher model and the student model simultaneously to obtain the output of the teacher and the output of the student. The outputs of both sides are divided by the distillation temperature T and processed using the Softmax function to obtain the soft labels Soft targets. Compared with the one-hot encoding corresponding to the true label, Soft targets carry more information. Therefore, the student network can obtain more information. The calculation formula for Soft targets is as follows:

[0133]

[0134] where y k is the output value of the k-th node in the output layer; L is the number of output nodes.

[0135] S22. Calculate the knowledge distillation error of each student network. If the error meets the termination condition, the model structure compression is completed; otherwise, perform error backpropagation to calculate the error of each layer, update the parameters of the student network, and then return to S21. Among them,

[0136] The loss of the student model in knowledge distillation is divided into two parts. One part is the cross-entropy loss between the student model and the soft targets of the teacher model, and the other part is the cross-entropy loss function between the output of the student model and the true labels. Then the total loss function of the student model is as follows:

[0137]

[0138] where w is the weight parameter of the student model, Z k is the true label, O k is the actual output value of the student model, L is the number of output classes, is the soft targets of the teacher model, is the soft targets of the student model. During the training process, the student model performs backpropagation according to this total loss function to update the parameters of the student model, so that the student model can train a generalization ability close to that of the teacher network.

[0139] S23. Quantize the model parameters;

[0140] During the continuous training process of the neural network, the parameter distribution is mainly limited to a relatively small range. The parameter distribution of the trained neural network is relatively concentrated, approximately a normal distribution. Most of the parameters in the CNN are of floating-point type, with a large storage range and high precision. Storing the parameters of the normal distribution in the network is often redundant. Therefore, the parameters in the model can be quantized, which can reduce the memory occupied by the model and improve the operation speed. To perform quantization of the model parameters, the original floating-point type parameters need to be mapped to parameters of a lower byte type. The quantization process is as Figure 5 shown. When performing quantization, the mapping formula from high-precision parameters to low-precision parameters is as follows:

[0141] Q = R / S + Z

[0142] S = (R max - R min ) / (Q max - Q min )

[0143] Z = Q max - R max / S

[0144] Where R represents the input high - precision parameter, Q represents the quantized low - precision result, S is the scale factor, representing the smallest scale that can be represented after fixed - point quantization, R max and R min are the maximum and minimum values of the high - precision parameter, Q max and Q min are the maximum and minimum values of the quantized low - precision data type, and Z is the offset value, representing the fixed - point value corresponding to the 0 value after quantization. Generally, it is considered that the more similar the quantized parameter is to the original data distribution, the less information loss of the original data after quantization. Therefore, the Kullback - Leibler divergence (KL - divergence) is used as the loss function, and the formula is as follows:

[0145] D KL = ∑P(i) * log(P(i) / M(i))

[0146] Where P is the probability distribution of Q, and M is the probability distribution of R.

[0147] The structure compression of each student network model is completed, and the parameters of each student network are quantized from 32 - bit to 8 - bit; the multi - dimensional compression of the feature fusion unit is completed, and the structure and parameter results of the compressed network are saved. The compressed network parameters are transmitted to the fault diagnosis model on the edge side through remote Over - the - Air Technology (OTA).

[0148] In this embodiment, in step S3, the result of the fault diagnosis is uploaded to the cloud platform and the local monitoring end. The network model on the cloud is only used for training and updating the parameters of the edge - side fault diagnosis model, and is not used for monitoring the operating state of the device. The operating state of the device on the cloud can be synchronized to the mobile monitoring end through communication, such as mobile phones, tablets, and computers, etc.

[0149] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit them. Although the present invention has been described in detail with reference to the preferred embodiments, those of ordinary skill in the art should understand that the technical solutions of the present invention can be modified or equivalently replaced without departing from the purpose and scope of the present technical solution, and they should all be covered within the scope of the claims of the present invention.

Claims

1. A multi-dimensional knowledge distillation and compression method for a cloud-edge collaborative fault diagnosis model, characterized in that: The method comprises the following steps: S1. Collect vibration data from multiple measuring points through sensors and use the measuring point optimization method to perform lightweight compression on the vibration data from multiple measuring points; S2. Establish a cloud-side fault diagnosis model and a device-side fault diagnosis model respectively. The cloud-side fault diagnosis model compresses the network model using multi-dimensional knowledge distillation to obtain lightweight network parameters, and then transmits the network parameters to the device-side fault diagnosis model. S3. The diagnostic results output by the device-side fault diagnosis model are transmitted to the cloud platform and the local monitoring terminal, and the device operation status is monitored in real time through interaction between the mobile terminal and the cloud platform.

2. The multi-dimensional knowledge distillation compression method of a cloud-edge collaborative fault diagnosis model according to claim 1, characterized in that: In step S1, the following steps are included: S11. Vibration data acquisition at multiple measurement points: Vibration data is collected through vibration sensors set at corresponding measurement points to obtain the original signal data point set P = {P1, P2,..., P n}, where P1, P2,..., P n are different measurement points; S12. Establish the measurement point redundancy evaluation factor MI for all measurement point data in the original signal data point set i ; S13. Quantitatively evaluate the fault recognition accuracy ACC of the measurement point through the multi-information source fault feature extraction unit i ; S14. Based on the measurement point redundancy evaluation factor MI i and the fault identification accuracy ACC i establish the measurement point importance evaluation factor λ i ; S15. Lightweight processing is performed on the original data point set P through the backward search algorithm and the measurement point importance evaluation factor, and the optimal test point set P is output * .

3. The multi-dimensional knowledge distillation compression method of a cloud-edge collaborative fault diagnosis model according to claim 2, characterized in that: In step S12, the following steps are specifically included: S121. The expected amount of information is used as the redundancy evaluation index of the measurement point to describe the measurement of the amount of information required to eliminate system uncertainty. It is defined as follows: Where p(x) is the prior probability of event x occurring, and X is the set of all values of x; S122. Use conditional entropy to analyze the relationship between multiple variables. For random variables X and Y, the conditional entropy is defined as follows: Where p(x,y) is the joint probability density function of the two variables, and p(x|y) is the posterior probability of x occurring under the condition that y occurs; S123. Mutual information is used to characterize the redundant information between different measurement points and quantitatively describe the correlation between two variables. The calculation formula is as follows: The larger the mutual information value, the stronger the correlation between the two variables x and y; S124. Calculate the fault isolation redundancy factor of the measuring point and the shared information of the measuring point. The fault isolation redundancy factor of the measuring point refers to the mutual information between different fault states of the same measuring point. The calculation formula for the fault isolation redundancy factor of the measuring point i is: where k and j represent the serial numbers of fault types, and n is the number of fault status types, and are the data collected at measurement point i under fault status k and fault status j, respectively; The common information volume of measuring points refers to the common information between the data collected by different measuring points under the same fault. The calculation formula of the common information volume of measuring point i is: where k represents the serial number of the fault type, j is the measuring point number, n is the number of fault state types, and m is the number of measuring points, and are the data collected at measuring point i and measuring point j respectively under the fault state k; S125. Measure the redundancy of the measuring points based on their isolation redundancy factor and the amount of public information, and calculate the redundancy evaluation factor for the measuring points: MI i = I i / M i Among them, MI i is the measurement point redundancy evaluation factor, M i is the measurement point fault isolation redundancy factor, and I i is the common information volume of the measurement point.

4. A multi-dimensional knowledge distillation compression method for a cloud-edge collaborative fault diagnosis model according to claim 2, characterized in that: In step S13, the following steps are specifically included: S131. Determine the fault category based on the fault probabilities output by the fault diagnosis network. The expression for the fault category is defined as: Z = [Max(O1), Max(O2),..., Max(O L )] where O = [O1, O2,..., O L is the output of the network, L is the number of recognition categories. If Z is the same as the true type label, the recognition is correct; S132. The calculation formula for fault identification accuracy is as follows: Where Y is the true label of the sample, k is the sample number, and K is the total number of samples.

5. The multi-dimensional knowledge distillation and compression method for a cloud-edge collaborative fault diagnosis model according to claim 2, characterized in that: In step S14, the measurement point importance evaluation factor λ i is calculated by the following formula: λ i = MI i + ACC i where i is the measuring point number, and MI i is the measuring point redundancy evaluation factor of measuring point i, and ACC i is the fault identification accuracy rate of measuring point i.

6. A multi-dimensional knowledge distillation and compression method for a cloud-edge collaborative fault diagnosis model according to claim 2, characterized in that: In step S15, the following steps are specifically included: S151. First, all measurement points are arranged in ascending order according to the measurement point importance evaluation factor to obtain the arranged measurement point set P′={P1′, P2′…, P n ′}; S152. Sequentially delete the measurement points with low importance coefficients from P′ to obtain a new subset of measurement points P * ={P1, P2, … P m}; S153. Evaluate the effectiveness of the measurement point subset P using the multi-information fusion fault identification network model * ; S154. If the recognition accuracy rate is higher than the preset threshold, the measurement point subset P at this time * is the optimal test point set; if the recognition accuracy rate is lower than the preset threshold, then P * is brought into step S151 to be repeatedly executed until the optimal test point set is output.

7. A multi-dimensional knowledge distillation and compression method for a cloud-edge collaborative fault diagnosis model according to claim 2, characterized in that: In step S2, the following steps are specifically included: S21. In the constructed cloud fault diagnosis model, it includes each student network Sub-model-V1, Sub-model-V2, …, Sub-model-V, and the parameters of each teacher network teacher-V1, teacher-V2, …, teacher-V. N Train the student network SV by means of knowledge distillation: S22. Calculate the knowledge distillation error of each student network. If the error meets the termination condition, the model structure compression is completed; otherwise, perform error backpropagation to calculate the error of each layer, update the student network parameters, and then return to S21. S23. Quantize the parameters of the lightweight network model after training, and pass the quantized content to the device-side fault diagnosis model.

8. A multi-dimensional knowledge distillation and compression method for a cloud-edge collaborative fault diagnosis model according to claim 7, characterized in that: In step S21, first, the teacher model is trained using the dataset. Then, the parameters of the trained teacher model are fixed. Next, the dataset is input into both the teacher model and the student model simultaneously to obtain the output of the teacher and the output of the student. The outputs of both parties are divided by the distillation temperature T and processed using the Softmax function to obtain the soft labels. The formula for the soft labels is as follows: where y k is the output value of the k-th node in the output layer; L is the number of output nodes.

9. A multi-dimensional knowledge distillation compression method for a cloud-edge collaborative fault diagnosis model according to claim 8, characterized in that: In step S22, the loss of the student model in knowledge distillation includes the cross-entropy loss between the soft labels of the student model and the teacher model, and the cross-entropy loss function between the output of the student model and the true labels. Then, the total loss function of the student model is as follows: where w are the weight parameters of the student model, Z k is the true label, O k is the actual output value of the student model, L is the number of output classes, is the soft label of the teacher model, is the soft label of the student model; the student model performs backpropagation according to this total loss function during training to update the parameters of the student model.

10. A multi-dimensional knowledge distillation and compression method for a cloud-edge collaborative fault diagnosis model according to claim 9, characterized in that: In step S23, the original floating-point type parameters are mapped into lower-byte type parameters for model parameter quantization. The mapping formula from high-precision parameters to low-precision parameters is as follows: Q = R / S + Z S = (R max - R min ) / (Q max - Q min ) Z = Q max -R max / S Where R represents the input high-precision parameter, Q represents the low-precision result after quantization, S is the scale factor, representing the smallest scale that can be represented after fixed-point quantization, R max and R min are the maximum and minimum values of the high-precision parameter, Q max and Q min are the maximum and minimum values of the quantized low-precision data type, Z is the offset value, which represents the fixed-point value corresponding to the 0 value after quantization; The KL divergence is used as the loss function for the quantization process, and the formula is as follows: D KL = ∑P(i) * log(P(i) / M(i)) where P is the probability distribution of Q, and M is the probability distribution of R.

Images