A fault prediction method based on distributed training of a hierarchical neural network

Abstract

The present invention discloses a fault prediction method based on distributed training of a hierarchical neural network, specifically: dividing a convolutional neural network into different network layers to obtain a model training network in which each layer can output model parameters and model training accuracy; using the hierarchical convolutional neural network and the maximum number of network layers that the device can train to extract features from the input data. The input data is trained until convergence in each layer of the network, and the obtained model parameters are transmitted to the next layer of the network for training. After completing the maximum number of network layers of the specified training, a corresponding fault detection model is obtained; the convolutional neural network is divided into three sub-layer convolutional neural networks, which are respectively deployed at the device end, the edge end, and the cloud end. Each layer outputs a prediction value on whether the input data contains a fault, obtaining a fault prediction network to perform fault prediction on the data without fault labels and non-fault labels. The present invention can reasonably utilize the computing resources and storage space of the device.

Description

Technical Field

[0001] The present invention relates to the field of machine learning, and more particularly to a fault prediction method based on distributed training of a hierarchical neural network. Background Art

[0002] With the development of science and technology and the increasing living needs of the people, the production efficiency and working intensity of modern mechanical equipment are gradually increasing, and the complexity and automation degree of the equipment are also continuously enhanced with the development of technology. The correlation between each module of the equipment is becoming closer and closer, and a minor fault in a certain module may cause the entire equipment to break down, thus affecting the normal operation and economic benefits of the factory. In recent years, the prediction of equipment faults has become a hot and difficult research topic. Traditional fault detection systems can only diagnose known faults and do not have the ability of self-organization and self-learning. When new types of faults occur, they lose the ability to detect and diagnose. In recent years, artificial neural networks have been widely applied to fault detection and shown a broader application prospect.

[0003] An artificial neural network, also known as a connection model, is an algorithmic mathematical model that mimics the behavioral characteristics of animal neural networks and performs distributed parallel information processing. This network relies on the complexity of the system and does not require diagnostic and reasoning rules. It approximates a nonlinear system through the topological structure and weight distribution between neurons, thus having a certain generalization ability. However, in solving the fault diagnosis of complex systems, the diagnostic system based on neural networks is very large. Whether training a model that contains all data features based on known faults and normal data or using a known model to perform fault prediction and diagnosis on existing data, it poses a great challenge to the computing power and storage capacity of the device. At the same time, if it is necessary to monitor in real time whether a certain part of the device fails during operation, the requirements for the performance of the device itself will be higher. However, the existing methods for using artificial neural networks to detect device faults cannot solve the problems of large input data, complex models, insufficient computing power and storage capacity of the device, resulting in low efficiency in the process of data feature extraction and fault detection, and being unable to adapt to the current high working intensity of the device. Summary of the Invention

[0004] The purpose of the present invention is to provide a fault prediction method based on distributed training of a hierarchical neural network, enabling the device to dynamically select the number of layers of the training and inference networks, thereby reasonably utilizing the computing resources of the device or adapting to the storage space.

[0005] The technical solution for achieving the purpose of the present invention is: a fault prediction method based on distributed training of a hierarchical neural network, comprising the following steps:

[0006] Step 1: Divide the convolutional neural network into different network layers. The vector dimension of the output of each sub-layer network is the same as the input vector dimension of the next layer network, obtaining a model training network that includes multiple sub-layers and each layer can output model parameters and model training accuracy.

[0007] Step 2: Calculate the maximum number of network layers that the device can train according to the device parameters and the size of the input data with fault labels and non-fault labels.

[0008] Step 3: Under the condition that the device computing power resources and storage capacity are limited, use the convolutional neural network processed by layer division in Step 1 and the maximum number of training network layers obtained in Step 2 to extract features from the input data in Step 2. The input data is trained to convergence in each layer of the network. The model parameters obtained during the training process are passed to the next layer of the network for training. After completing the specified maximum number of training network layers, a corresponding fault detection model is obtained.

[0009] Step 4: Divide the same convolutional neural network as in Step 1 into three sub-layer convolutional neural networks, and deploy the three sub-layer neural networks on the device side, the edge side, and the cloud side respectively. Each layer outputs a prediction value on whether the input data contains a fault. The neural networks deployed on the device side, the edge side, and the cloud side can communicate with each other, obtaining a fault prediction network.

[0010] Step 5: Use the fault prediction network obtained in Step 4 and the fault detection model obtained in Step 3 to perform fault prediction on the data without fault labels and non-fault labels. First, perform prediction on the device side to obtain a prediction value on whether the input data contains a fault. According to the size of this value and the actual requirements, determine whether to perform prediction on the edge side and the cloud side.

[0011] Further, the division of the convolutional neural network into different network layers in Step 1 is specifically as follows: Each layer of the network includes a convolutional layer, an activation function, a pooling layer, and a fully connected layer. In each sub-layer neural network, first use a convolutional layer to obtain the feature output of the input fault data. The activation function uses the rectified linear unit function. The pooling layer selects the max pooling, and accesses the fully connected layer to integrate the parameters output by the pooling layer. The output of the fully connected layer is passed to the softmax layer for classification.

[0012] Further, the statement that the vector dimension of the output of each sub-layer network is the same as the input vector dimension of the next layer network in Step 1 means that the parameter size of the input convolutional layer of the latter layer is the same as the output parameter size of the fully connected layer of the previous layer.

[0013] Further, in step 1, the model training network includes multiple sub-layers, and each layer can output model parameters and model training accuracy. The model training accuracy refers to the ratio of the amount of data that the fault detection model trained based on data with fault and non-fault labels can correctly distinguish between faults and non-faults to the total input data volume.

[0014] Further, the maximum number of network layers that the computing device in step 2 can train refers to the maximum number of network layers that can be trained. Limited by the device's computing power, memory, and input data volume, the specific formula is as follows:

[0015]

[0016] Where, is a constant coefficient, f is the number of computing cycles per second of the device's central processing unit, R is the memory size of the device, is the size of the input data volume, and k is the size of the computing cycles of the central processing unit required to process one piece of data.

[0017] Further, in step 3, the input data needs to be trained to convergence in each layer of the network, which means that multiple iteration processes are performed in each layer until the descent rate of the loss function is 0.

[0018] Further, in step 4, the neural networks deployed on the device side, edge side, and cloud side can communicate with each other, which means that the device side can transmit the output model parameters to the edge side through a wireless channel for prediction, and the model parameters output by the edge side are transmitted to the cloud side through a wireless channel for prediction. The prediction values of whether the data contains faults obtained at the edge side and the cloud side can be transmitted to the device side through a wireless channel; the model parameters refer to the parameter values obtained from inside the convolutional neural network during the fault prediction process.

[0019] Compared with the prior art, the significant advantages of the present invention are: (1) By dividing the convolutional neural network into multiple sub-layer neural networks, the global fault detection model and the fault prediction and diagnosis accuracy can be obtained relatively quickly during the actual operation of the device, which not only saves computing resources but also improves the working efficiency of the device; (2) The device can adaptively select the number of layers of the convolutional neural network for model training and inference according to the computing resources and storage space, so as to reasonably utilize the computing resources of the device or adapt to the storage space. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 is a schematic flowchart of the fault prediction method based on distributed training of a hierarchical neural network of the present invention.

[0021] Figure 2 is a schematic diagram of the system during the model training process in deep learning of the present invention.

[0022] Figure 3 It is a schematic diagram of the system for the data inference process in the deep learning of the present invention. Specific embodiments

[0023] For traditional deep learning, if it contains a large training and inference network, for devices with insufficient computing power, it is impossible to complete the training of the feature extraction model and the accuracy prediction inference process, or the time taken is long and the efficiency is low. In view of the current situation that due to limited computing resources or insufficient storage space of device terminals, the extraction of fault data features using convolutional neural networks is insufficient or the fault prediction diagnosis of fault data cannot be completed according to known fault detection models, the present invention proposes a fault prediction method based on distributed training of hierarchical neural networks, combined with Figures 1 to 3 , specifically including the following steps:

[0024] Step 1: Divide the convolutional neural network into different network layers, and the vector dimension output by each sub-layer network is the same as the input vector dimension of the next layer network, so as to obtain a model training network including multiple sub-layers and each layer can output model parameters and model training accuracy;

[0025] Step 2: Calculate the maximum number of network layers that the device can train according to the device parameters and the size of the input data with fault labels and non-fault labels;

[0026] Step 3: Under the condition of limited computing power resources and storage capacity of the device, use the convolutional neural network processed by layering in Step 1 and the maximum number of trained network layers obtained in Step 2 to extract features from the input data in Step 2. The input data needs to be trained until convergence in each layer of the network, and the model parameters obtained during the training process are only transmitted to the next layer of the network for training. After completing the specified maximum number of trained network layers, the corresponding fault detection model is obtained;

[0027] Step 4: Divide the same convolutional neural network as in Step 1 into three sub-layer convolutional neural networks, and deploy the three sub-layer neural networks on the device side, the edge side, and the cloud side respectively. Each layer outputs the predicted value of whether the input data contains a fault. The neural networks deployed on the device side, the edge side, and the cloud side can communicate with each other to obtain a fault prediction network;

[0028] Step 5: Use the fault prediction network obtained in Step 4 and the fault detection model obtained in Step 3 to perform fault prediction on the data without fault labels and non-fault labels. First, perform prediction on the device side to obtain the predicted value of whether the input data contains a fault, and judge whether to perform prediction on the edge side and the cloud side according to the size of this value and the actual requirements.

[0029] As a specific example, in step 1, the convolutional neural network is divided into different network layers, specifically: each network layer includes a convolutional layer, an activation function, a pooling layer, and a fully connected layer. In each sub-layer neural network, first, a convolutional layer is used to obtain the feature output of the input fault data. The activation function uses the rectified linear unit function. The pooling layer selects max pooling, and the fully connected layer is connected to integrate the parameters output by the pooling layer. The output of the fully connected layer is passed to the softmax layer for classification.

[0030] As a specific example, the vector dimension of the output of each sub-layer network in step 1 is the same as the input vector dimension of the next layer network, which means that the parameter size of the input convolutional layer of the latter layer is the same as the output parameter size of the fully connected layer of the previous layer.

[0031] As a specific example, in step 1, a model training network that includes multiple sub-layers and each layer can output model parameters and model training accuracy is specifically: for the traditional convolutional neural network, only a set of model parameters and a model training accuracy can be output in each communication round, while this hierarchical neural network can output the corresponding model parameters and model training accuracy in each layer network. Among them, the model training accuracy refers to the ratio of the amount of data that can correctly distinguish faults and non-faults to the total input data volume by the fault detection model trained according to the data with fault and non-fault labels.

[0032] As a specific example, the maximum number of network layers that the computing device can train in step 2 refers to the maximum number of network layers that can be trained. Limited by the device's computing power, memory, and input data volume, the specific formula is as follows:

[0033]

[0034] Among them, f is the number of computing cycles per second of the device's central processing unit, R is the memory size of the device, is the size of the input data volume, k is the size of the computing cycles of the central processing unit required to process one piece of data; is a constant coefficient, The value of should satisfy that when the device data is fixed and the input data volume is the largest, is less than or equal to the maximum number of layers of this neural network.

[0035] As a specific example, the input data in step 3 needs to be trained until convergence in each network layer, which means that in each layer, multiple iteration processes are performed until the descent rate of the loss function is 0.

[0036] As a specific example, in step 4, the ability of the neural networks deployed on the device side, edge side, and cloud side to communicate with each other means that the device side can transmit the output model parameters to the edge side through a wireless channel for prediction, and the model parameters output by the edge side are transmitted to the cloud side through a wireless channel for prediction. The prediction values of whether the data contains faults obtained on the edge side and the cloud side can be transmitted to the device side through the wireless channel; the model parameters refer to the parameter values obtained from inside the convolutional neural network during the fault prediction process.

[0037] The present invention will be further described in detail below with reference to specific embodiments. Embodiment

[0038] This embodiment provides a method for efficiently extracting feature models and predicting the accuracy of fault data of a hierarchical neural network based on cloud-edge-device collaboration. This method means that by dividing the model training network and the model inference network into multiple sub-network layers, and outputting models separately for each layer of the network, the device can dynamically select the number of layers of the training and inference networks, so as to reasonably utilize the computing resources of the device or adapt to the storage space.

[0039] In an industrial scenario, a company has a large amount of normal working data and fault data of devices internally. The company can train corresponding fault detection models based on this data. During centralized training, the computing resources and storage space of the devices are limited and the data volume is large. When using a larger neural network for training, problems such as being unable to complete the training will occur. At this time, we divide the total neural network for training. When adding a softmax layer to each layer after division, it is also ensured that adjacent layers are connected to each other. The output model parameters of the upper layer network are used as the input of the lower layer network. According to the computing resources, storage and other conditions of the device as prior knowledge, it is transmitted into the neural network. The neural network obtains the number of layers to be trained through this prior knowledge. During the training process, when the decrease amplitude of the loss function output by each layer is within a certain threshold, the training of the next layer is carried out.

[0040] After the training is completed, the output model obtained from the training is deployed to the fault detection device. Due to the limitations of the computing resources and other conditions of the fault detection device, it is impossible to complete the inference process of a larger neural network. At this time, we deploy the first layer of the neural network on the device side. When the device working data is uploaded to the neural network, a fault prediction accuracy for the input data can be obtained as soon as possible. If a high inference accuracy has been obtained on the device side, it is selected not to perform the next stage of inference, to determine whether the working device has a fault, and to issue an alarm. If the inference accuracy on the device side does not meet the requirements and further fault diagnosis is required, at this time, the output model parameters of the device side are transmitted into the neural network for edge-side inference through a wireless channel. Then, repeat the above operations until a better inference accuracy is obtained or the entire inference process is completed, and this process ends.

[0041] The present invention designs a method for efficiently extracting feature models and predicting the accuracy of fault data of a hierarchical neural network based on cloud-edge-end collaboration, enabling the device to adaptively select the number of convolutional neural network layers for model training and inference according to computing resources and storage space.

[0042] The present invention hierarchically divides the convolutional neural network for model training. Each layer includes a convolutional layer, a pooling layer, and a fully connected layer. At the same time, a classifier is added after the fully connected layer of each divided convolutional neural network layer, so that the accuracy of the training feature extraction model can be output in each layer of the network. The input image or tabular data is trained using the stochastic gradient descent method until the loss function is kept within a certain range, so that each layer of the convolutional neural network can independently output the feature extraction model obtained from the sample data.

[0043] The present invention also hierarchically processes the convolutional neural network structure in the prediction and inference process. The previous network layers are deployed to the device side, the middle network layers are deployed to the edge side, and the last network layer is deployed to the cloud side. The three are connected through a wireless channel to implement a distributed prediction and inference process.

[0044] At the same time, a classifier is added after the fully connected layer of each layer, so that the prediction accuracy of the input data using the existing feature model can be output in each layer of the network. By adding a classifier in each layer and connecting adjacent two layers of the network, different numbers of layers can be trained and inferred according to the device's own needs during the model training or inference process, so as to reasonably utilize the computing resources of the device or adapt to the storage space.

Claims

1. A fault prediction method based on distributed training of a hierarchical neural network, characterized in that, It includes the following steps: Step 1: Divide the convolutional neural network into different network layers. The vector dimension output by each sub-layer network is the same as the input vector dimension of the next layer network, obtaining a model training network that contains multiple sub-layers and each layer can output model parameters and model training accuracy; Step 2: Calculate the maximum number of network layers that the device can train according to the device parameters and the size of the input data with fault labels and non-fault labels; Step 3: Under the condition that the device computing power resources and storage capacity are limited, use the convolutional neural network processed by layer in Step 1 and the maximum number of training network layers obtained in Step 2 to extract features from the input data in Step 2. The input data is trained to convergence in each layer of the network. The model parameters obtained during the training process are passed to the next layer of the network for training. After completing the specified maximum number of network layers of training, the corresponding fault detection model is obtained; Step 4: Divide the same convolutional neural network as in Step 1 into three sub-layer convolutional neural networks, and deploy the three sub-layer neural networks on the device side, the edge side, and the cloud side respectively. Each layer outputs a prediction value on whether the input data contains a fault. The neural networks deployed on the device side, the edge side, and the cloud side can communicate with each other, obtaining a fault prediction network; Step 5: Use the fault prediction network obtained in Step 4 and the fault detection model obtained in Step 3 to perform fault prediction on the data without fault labels and non-fault labels. First, perform prediction on the device side to obtain a prediction value on whether the input data contains a fault. According to the size of this value and the actual requirements, judge whether to perform prediction on the edge side and the cloud side; The division of the convolutional neural network into different network layers in Step 1 is specifically as follows: Each layer of the network includes a convolutional layer, an activation function, a pooling layer, and a fully connected layer. In each sub-layer neural network, first use a convolutional layer to obtain the feature output of the input fault data. The activation function uses the rectified linear unit function. The pooling layer selects max pooling, accesses the fully connected layer to integrate the parameters output by the pooling layer, and the output of the fully connected layer is passed to the softmax layer for classification; The statement that the vector dimension output by each sub-layer network in Step 1 is the same as the input vector dimension of the next layer network means that the parameter size of the input convolutional layer of the latter layer is the same as the output parameter size of the fully connected layer of the previous layer; In the model training network that contains multiple sub-layers and each layer can output model parameters and model training accuracy in Step 1, the model training accuracy refers to the ratio of the amount of data that can correctly distinguish faults and non-faults to the total input data amount of the fault detection model trained according to the data with fault and non-fault labels; The maximum number of network layers that the computing device described in step 2 can train refers to the maximum number of network layers Layer max Limited by the computing power and memory of the device and the amount of input data, the specific formula is as follows: Where, μ is a constant coefficient, f is the number of computing cycles per second of the device's central processing unit, R is the memory size of the device, |D| is the size of the input data amount, and k is the size of the computing cycles of the central processing unit required to process one piece of data; The statement that the input data in Step 3 needs to be trained to convergence in each layer of the network means that multiple iteration processes are performed in each layer until the descent rate of the loss function is 0; The ability of the neural networks deployed on the device side, edge side, and cloud side described in step 4 to communicate with each other means that the device side can transmit the output model parameters to the edge side through a wireless channel for prediction, and the model parameters output by the edge side are transmitted to the cloud side through a wireless channel for prediction. The prediction values of whether the data contains faults obtained at the edge side and the cloud side can be transmitted to the device side through the wireless channel; the model parameters refer to the parameter values obtained from within the convolutional neural network during the fault prediction process.

Images