New equipment rapid fault diagnosis method based on transfer learning

Abstract

The invention discloses a new equipment rapid fault diagnosis method based on transfer learning. The method comprises the following steps: firstly, carrying out self-supervised feature representation learning on normal operation data of new and old equipment by using a variational auto-encoder, and aligning features to the same space; then pre-training a multi-source migration model based on meta-learning on an existing equipment fault data set to quickly obtain an initial diagnosis model of new equipment; fine tuning is carried out in combination with a small amount of annotated data, and the model is continuously optimized by adopting an interactive active learning strategy. And meanwhile, a centralized fault knowledge base is constructed through the cloud, end-cloud collaborative distributed online learning is realized, and the model can continuously adapt to a new environment. According to the method, the research and development efficiency of a new equipment fault diagnosis model is greatly improved, the problem of data shortage is effectively solved, fault knowledge sharing between equipment is realized, and the method has remarkable application value.

Description

Technical Field

[0001] The present invention relates to the technical field of machine learning and fault diagnosis, and in particular to a new equipment rapid fault diagnosis method based on transfer learning. Background Art

[0002] Equipment fault diagnosis is a key technology in the manufacturing industry, which aims to accurately identify the fault type based on the equipment operation data and provide support for equipment repair and maintenance. With the continuous development of industrial automation and intelligent manufacturing, the traditional manual experience-driven fault diagnosis method is increasingly unable to meet actual needs, and there is an urgent need for automated fault diagnosis technology based on data and intelligent algorithms.

[0003] At present, data-driven fault diagnosis technology is mainly divided into two categories: rule-based and machine learning-based. The rule-based method requires a large amount of domain knowledge and a set of rules summarized from manual experience. Its modeling process is costly and difficult to update in time. Although machine learning-based methods (such as support vector machines, neural networks, etc.) have good generalization capabilities, they require a large amount of labeled fault data to train the model. For newly developed equipment, the cost of obtaining this data is extremely high, which has become a bottleneck for application.

[0004] In addition, existing transfer learning methods usually directly transfer model parameters or feature knowledge of previous tasks to new tasks, with limited generalization ability, making it difficult to fully utilize the fault diagnosis knowledge of existing equipment. At the same time, there is also a lack of efficient online learning and knowledge sharing mechanisms, which makes it impossible for the model to continuously track and adapt to changes in the equipment's working environment. Therefore, how to quickly build and continuously optimize the fault diagnosis model of new equipment based on a small amount of data has become a key technical problem that needs to be solved urgently. Summary of the invention

[0005] In order to solve the problems of the prior art, an embodiment of the present invention provides a new device rapid fault diagnosis method based on transfer learning. The technical solution is as follows:

[0006] On the one hand, a new device rapid fault diagnosis method based on transfer learning is provided, comprising the following steps:

[0007] (1) Using an unsupervised variational autoencoder network or its variants, the normal operation data of new and old equipment is used for self-supervised pre-training to learn the implicit feature representation of equipment data;

[0008] (2) Use a domain adversarial neural network to align the learned features of the new and old devices into the same feature space;

[0009] (3) Pre-train a meta-learner network based on a meta-learning framework on multiple existing equipment fault datasets, so that it can learn efficient migration strategies to adapt to new equipment fault diagnosis tasks;

[0010] (4) Use the above meta-learner to transfer the training of the fault diagnosis model on the new equipment, and fine-tune it with a small amount of labeled fault data of the new equipment to obtain a fault diagnosis model with good performance;

[0011] (5) Setting online performance evaluation indicators for the diagnostic model, and using an interactive active learning strategy based on the indicators to iteratively optimize the diagnostic model;

[0012] (6) Build a centralized fault mode knowledge base in the cloud. Each device uploads newly collected fault data to the cloud. The cloud continuously updates the fault model based on the data and periodically sends the updated model to the device to achieve distributed knowledge sharing and continuous online model learning.

[0013] (7) The trained fault diagnosis model is deployed to edge devices or the cloud in the production environment to quickly diagnose the type of equipment fault based on the real-time operation data of the equipment.

[0014] Furthermore, in step (1), for the device operation data in the form of time series data, a convolutional variational autoencoder or a time series variational autoencoder is used as the encoder network.

[0015] Furthermore, in step (3), the meta-learner network is trained based on a model-independent gradient descent meta-learning algorithm or a model-independent semi-supervised meta-learning algorithm.

[0016] Furthermore, in step (5), the interactive active learning strategy adopts a maximum entropy strategy, a minimum redundancy strategy, a maximum prediction entropy strategy or a maximum model change strategy, etc.

[0017] Furthermore, in step (6), the centralized fault mode knowledge base in the cloud is updated based on joint training or knowledge distillation.

[0018] Furthermore, after step (4), the following steps are also included: pruning, quantizing or knowledge distilling the diagnostic model to reduce its computational and storage overhead, thereby achieving edge deployment.

[0019] Furthermore, after step (7), the following steps are also included: feeding back the diagnosis results to maintenance personnel to guide fault repair and equipment operation and maintenance work.

[0020] On the other hand, a new equipment rapid fault diagnosis system is provided, comprising: a memory for storing a computer program; a processor for executing the computer program to implement the new equipment rapid fault diagnosis method.

[0021] On the other hand, a computer-readable storage medium is provided, on which a computer program is stored, and when the program is executed by a processor, the new device rapid fault diagnosis method is implemented.

[0022] On the other hand, a computer program product is provided, comprising the computer-readable storage medium.

[0023] The technical solution provided by the embodiment of the present invention has the following beneficial effects:

[0024] 1. Rapidly build a new equipment fault diagnosis model

[0025] By combining self-supervised learning, transfer learning, meta-learning and other technologies, the present invention can use the fault diagnosis knowledge of existing equipment to quickly build an initial diagnosis model for new equipment, greatly shortening the model development cycle and reducing the demand for labeled data.

[0026] 2. Continuously improve the performance of diagnostic models

[0027] By adopting an interactive active learning strategy and a distributed online learning mechanism, the present invention can continuously optimize the diagnostic model during actual application, so that it can continuously adapt to new working environments and failure modes, ensuring long-term high-precision diagnostic performance.

[0028] 3. Share equipment failure knowledge

[0029] The present invention constructs a centralized fault mode knowledge base through the cloud, and each device end can upload new fault data and regularly obtain updated models, forming an efficient equipment fault knowledge sharing channel for mutual benefit and win-win results.

[0030] 4. Improve fault diagnosis efficiency

[0031] Compared with the traditional manual experience diagnosis method, the present invention can quickly and automatically diagnose the fault type according to the real-time operation data of the equipment, provide efficient support for maintenance work, and greatly improve the efficiency of fault handling.

[0032] 5. Can be expanded to other fields

[0033] The technical solution proposed in the present invention is not only applicable to equipment fault diagnosis, but its innovative transfer incremental learning paradigm can also be extended to heterogeneous data classification, knowledge transfer and continuous learning tasks in other fields.

[0034] 6. Low cost and high return

[0035] Since the need for a large amount of labeled fault data is avoided, the R&D and application costs of the present invention are relatively low. By improving the accuracy and efficiency of fault diagnosis, economic value can be created for enterprises, with good social and economic benefits.

[0036] In summary, the present invention proposes an innovative multi-technology fusion solution to meet the actual needs of rapid fault diagnosis of new equipment, which has significant technological advancement and application value. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0038] Figure 1 It is a flow chart of a new device rapid fault diagnosis method based on transfer learning according to an embodiment of the present invention. DETAILED DESCRIPTION

[0039] In order to make the objectives, technical solutions and advantages of the present invention more clear, the embodiments of the present invention will be further described in detail below with reference to the accompanying drawings.

[0040] This paper proposes a new device rapid fault diagnosis method based on transfer learning, see Figure 1 , the specific implementation steps are as follows:

[0041] 1. Self-supervised feature representation learning

[0042] The variational autoencoder (VAE) network or its variants such as convolutional VAE (ConvVAE) and time series VAE (TimeVAE) are used to perform self-supervised pre-training using the normal operating data of new and old devices (such as time series sensor data) to learn the implicit feature representation of device data.

[0043] This step extracts low-dimensional continuous potential feature representations from the original high-dimensional time series data by minimizing the reconstruction error of the input data and maximizing the KL divergence between the encoding vector and a prior distribution (such as Gaussian distribution), laying the foundation for subsequent supervised transfer learning.

[0044] 2. Feature space alignment based on adversarial

[0045] The feature representations of the above different domains (new and old devices) are aligned into a unified feature space using a domain adversarial neural network (DANN). DANN consists of three parts: feature extractor, label predictor, and domain discriminator.

[0046] Through adversarial training, the feature extractor learns to generate domain-invariant feature representations that can deceive the domain discriminator, thereby minimizing the distribution difference of new and old device features in the unified space, making the space more conducive to training a universal fault classifier network.

[0047] Specifically, this step uses a domain adversarial neural network (DANN) to align the feature spaces of new and old devices. The network structure of DANN is as follows:

[0048] 1) Feature Extractor Network G:

[0049] Input is new and old equipment sample x

[0050] The output is the corresponding feature representation G(x)

[0051] 2) Label predictor network C:

[0052] The input is G(x)

[0053] The output is the sample label (fault type) predicted probability C(G(x))

[0054] 3) Domain discriminator network D:

[0055] The input is G(x)

[0056] The output is the probability prediction D(G(x)) that the feature comes from the new device domain or the old device domain.

[0057] During adversarial training, G and C are first fixed, and D is made to minimize the domain discrimination loss of G(x), that is, to maximize the ability to distinguish between new and old domains. Then D is fixed, and G and C are made to minimize two losses: label prediction loss and domain discrimination loss, forcing G to learn domain-invariant features G(x) that can deceive D.

[0058] Through alternating adversarial training of G and D, the final G(x) will be a feature representation that is sufficient to enable the label predictor C(G(x)) to achieve good performance while maximizing the confusion domain D. At this point, G(x) is the unified feature space of the aligned new and old devices, which is conducive to the subsequent training of the general fault classifier.

[0059] 3. Multi-source transfer based on meta-learning

[0060] Use model-agnostic meta-learning algorithms such as MAML (Model-Agnostic Meta-Learning) or MASSML (Model-Agnostic Meta-Learning for Semi-Supervised Tasks) to pre-train the meta-learner network on multiple existing equipment failure data sets.

[0061] Taking MAML as an example, its inner loop is used to start from the current parameters of the meta-learner and adapt to the task training set through a small number of gradient update steps; the outer loop updates the meta-learner parameters through the performance of the task verification set, so that it can learn an efficient migration strategy that can quickly adapt to new tasks.

[0062] 4. New equipment fault diagnosis model migration training

[0063] Using the meta-learner network from step 3 and the new device feature representation from step 1, the diagnostic model is initially trained for transfer using optimization methods such as gradient descent.

[0064] Then, a small amount of newly collected new equipment annotated fault data is divided into a training set and a validation set. Adam and other optimization algorithms are used to perform a small number of iterations on the training set until satisfactory performance is obtained on the validation set. The initial migration model is fine-tuned and optimized to obtain the final high-performance fault diagnosis model. The model can be persisted in the form of pre-trained parameters + new training parameters.

[0065] Specifically, in this step, the new equipment annotated fault data needs to be divided first to construct a training set and a validation set. A commonly used method is stratified sampling:

[0066] 1) Count the number of samples of each type according to the fault type in the new equipment fault data

[0067] 2) In each fault category, randomly select a portion of samples in a certain ratio (such as 7:3) as the validation set

[0068] 3) The remaining samples are used as training sets

[0069] The validation set constructed in this way has good distribution estimation ability. The model parameters are optimized on the training set, and the model performance is evaluated on the validation set.

[0070] In the case of poor performance on the validation set, the following strategies can be adopted:

[0071] 1) Determine whether it is overfitting based on the learning curve. If so, adjust hyperparameters such as dropout and regularization

[0072] 2) If there is no overfitting, you can try a larger learning rate and more training epochs.

[0073] 3) When performance is not improving steadily, more new device data can be combined through transfer / integrated learning

[0074] Through the above fine-tuning and optimization, the final new equipment fault diagnosis model with good performance can be obtained.

[0075] 5. Interactive Active Learning Strategy

[0076] Set online performance evaluation indicators for the diagnostic model, such as fault detection accuracy, recall rate, F1 score, etc. Based on these indicators, adopt active learning strategies such as maximum entropy strategy, minimum redundancy strategy, maximum prediction entropy strategy, or maximum model change strategy to iteratively optimize the diagnostic model with minimal manual annotation overhead.

[0077] The specific approach is to select unlabeled samples with the highest value and that can cause the greatest change to the model, have them manually labeled and include them in the training set, and repeat this process until the model performance meets the requirements.

[0078] 6. Distributed online learning with end-cloud collaboration

[0079] A centralized fault mode knowledge base is built in the cloud, and the knowledge base model is initialized by pre-training through supervised or semi-supervised methods (such as semi-supervised VAE, GAN, etc.) based on the fault datasets of all existing devices.

[0080] Each device uploads the newly collected unlabeled fault data to the cloud. The cloud uses this data as an increment to continuously update and optimize the knowledge base model through semi-supervised learning (such as semi-supervised VAE) or knowledge distillation.

[0081] The updated knowledge base model is regularly sent to each device. The device initializes the local diagnosis model with the model, and uses the new data of the device itself, adopts online incremental learning or kontinuous learning and other methods to achieve continuous optimization and self-evolution of the local model.

[0082] In this way, through two-way collaborative training between the cloud and the device, we can continuously track and adapt to changes in the equipment working environment, and fully realize the distributed online updating and sharing of fault knowledge.

[0083] 7. Deployment and Application

[0084] Deploy the trained diagnostic model to edge devices or cloud servers. When the device is running, its operating data is obtained in real time and input into the diagnostic model, and the model will output the diagnostic results of the corresponding fault type.

[0085] The diagnostic results can be fed back to on-site maintenance personnel or remote experts in a timely manner to provide reference and guidance for fault cause analysis and treatment, thereby improving fault repair efficiency.

[0086] After step 4, the diagnostic model can be compressed by pruning, quantization, or knowledge distillation to reduce its computational and storage overhead, which is beneficial for edge deployment.

[0087] The above specific implementation methods fully explain the technical details of the present invention, covering multiple innovative links such as self-supervised representation learning, feature space alignment, meta-learning transfer, active learning optimization, end-cloud distributed online learning, etc., forming an efficient and complete new equipment fault diagnosis solution.

[0088] This solution innovatively integrates multiple advanced technologies, can quickly build diagnostic models in the absence of labeled data, and continuously optimize tracking model performance, while achieving efficient sharing of fault knowledge, with outstanding innovation and application value.

[0089] The above are only preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A new equipment rapid fault diagnosis method based on transfer learning, characterized in that: The following steps are involved: (1) Using an unsupervised variational autoencoder network or its variants, the normal operation data of new and old equipment is used for self-supervised pre-training to learn the implicit feature representation of equipment data; (2) Use a domain adversarial neural network to align the learned features of the new and old devices into the same feature space; (3) Pre-train a meta-learner network based on a meta-learning framework on multiple existing equipment fault datasets, so that it can learn efficient migration strategies to adapt to new equipment fault diagnosis tasks; (4) Use the above meta-learner to transfer the training of the fault diagnosis model on the new equipment, and fine-tune it with a small amount of labeled fault data of the new equipment to obtain a fault diagnosis model with good performance; (5) Setting online performance evaluation indicators for the diagnostic model, and using an interactive active learning strategy based on the indicators to iteratively optimize the diagnostic model; (6) Build a centralized fault mode knowledge base in the cloud. Each device uploads newly collected fault data to the cloud. The cloud continuously updates the fault model based on the data and periodically sends the updated model to the device to achieve distributed knowledge sharing and continuous online model learning. (7) The trained fault diagnosis model is deployed to edge devices or the cloud in the production environment to quickly diagnose the type of equipment fault based on the real-time operation data of the equipment.

2. The method according to claim 1, characterized in that In step (1), for the device operation data in the form of time series data, a convolutional variational autoencoder or a time series variational autoencoder is used as the encoder network.

3. The method according to claim 1, characterized in that In step (3), the meta-learner network is trained based on a model-independent gradient descent meta-learning algorithm or a model-independent semi-supervised meta-learning algorithm.

4. The method according to claim 1, characterized in that: In step (5), the interactive active learning strategy adopts a maximum entropy strategy, a minimum redundancy strategy, a maximum prediction entropy strategy or a maximum model change strategy.

5. The method according to claim 1, characterized in that In step (6), the centralized fault mode knowledge base in the cloud is updated based on joint training or knowledge distillation.

6. The method according to claim 1, characterized in that After step (4), the following steps are also included: pruning, quantizing or knowledge distilling the diagnostic model to reduce its computational and storage overhead, thereby achieving edge deployment.

7. The method according to claim 1, characterized in that After step (7), the following steps are also included: feeding back the diagnosis results to maintenance personnel to guide fault repair and equipment operation and maintenance work.

8. A new equipment rapid fault diagnosis system, characterized in that: include: Memory for storing computer programs; A processor, used to execute a computer program to implement the new device rapid fault diagnosis method described in any one of claims 1-7.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the new device rapid fault diagnosis method described in any one of claims 1-7 is implemented.

10. A computer program product, characterized in that Includes the computer-readable storage medium of claim 9.

Images