Fault prediction method and system based on single path unit neural network architecture search

Abstract

The present invention belongs to the field of neural network fault prediction and specifically discloses a fault prediction method and system based on a single-path unit neural network architecture search. The method comprises: collecting equipment fault signals to obtain a labeled fault prediction dataset, which is divided into a training set and a validation set; constructing a hypernetwork model based on the single-path unit, wherein the hypernetwork model can generate weights for all subnetwork models in the search space according to different operations in the single-path unit; using the training set, the hypernetwork model is trained using a hypernetwork training strategy based on a balance constraint to obtain a trained hypernetwork model; using the validation set, the optimal subnetwork model is searched for on the trained hypernetwork model; and the optimal subnetwork model is trained using the training set to obtain a trained optimal subnetwork model, which is then applied to fault prediction. The present invention realizes the automatic design of the neural network fault prediction model structure, and the model has higher performance in mechanical intelligent fault prediction tasks.

Description

Technical Field

[0001] The present invention belongs to the field of neural network fault prediction, and more specifically, relates to a fault prediction method and system based on single-path unit neural network architecture search. Background Art

[0002] Fault prediction is an integral component of intelligent machinery health monitoring systems. By monitoring the real-time operating status of machinery and issuing timely risk warnings, fault prediction can mitigate the economic losses caused by mechanical failures to a certain extent. Thanks to the massive amounts of industrial data generated by the development of information technology, data-driven fault prediction methods are gaining popularity among researchers and have achieved remarkable results on a wide range of mechanical equipment.

[0003] Deep learning, capable of automatically learning fault signatures from large-scale monitoring data end-to-end, has become a dominant force in data-driven fault prediction methods. However, the design of deep learning models determines their performance in fault prediction tasks. Manually designing a suitable neural network model for a specific fault prediction task not only relies on extensive prior knowledge but is also time-consuming and labor-intensive, requiring repeated debugging and exhibiting significant limitations. Therefore, a new method for automatically designing neural network models is urgently needed to achieve this goal. Summary of the Invention

[0004] In response to the above-mentioned defects or improvement needs of the prior art, the present invention provides a fault prediction method and system based on single-path unit neural network architecture search, the purpose of which is to automatically design the specific structure of the neural network fault prediction model, significantly reduce the time cost of manually designing the model, and improve the performance of the neural network model in mechanical intelligent fault prediction tasks.

[0005] To achieve the above objectives, according to a first aspect of the present invention, a fault prediction method based on a single-path unit neural network architecture search is proposed, comprising the following steps:

[0006] S1. Collect equipment fault signals, preprocess them, obtain labeled fault prediction datasets, and divide them into training and validation sets.

[0007] S2. Construct a super-network model based on a single path unit. According to the different operations selected in the single path unit, the super-network model can generate the weights of all sub-network models in the search space;

[0008] S3. Using the training set, a hypernetwork model is trained using a hypernetwork training strategy based on balance constraints to obtain a trained hypernetwork model.

[0009] S4. Use the validation set to search for the optimal sub-network model on the trained super-network model;

[0010] S5. Use the training set to train the optimal sub-network model to obtain a trained optimal sub-network model;

[0011] S6. Realize equipment failure prediction through the trained optimal sub-network model.

[0012] As a further preferred embodiment, the hypernetwork model includes three single-path units and a common part; each single-path unit includes two input nodes and three intermediate nodes, the two input nodes are connected to the three intermediate nodes respectively, and the three intermediate nodes are connected in pairs, i.e., nine unidirectional edges;

[0013] Specifically, the two input nodes respectively receive the feature maps input from the outside and transmit them to the intermediate node through a unidirectional edge; the intermediate node receives the feature maps obtained by all previous input nodes and intermediate nodes after a unidirectional edge, and superimposes the feature maps; the unidirectional edge outputs a feature map after the input feature map is processed by a certain operation in the operation set; the output of the single-path unit is obtained by connecting the three intermediate nodes.

[0014] As a further preferred embodiment, step S3, training the hypernetwork model, specifically includes the following steps:

[0015] S3.1. Based on the balance constraint, four sub-network models are sampled sequentially on the super-network model. When sampling sequentially, on the unidirectional edge at the same position, the sub-network model cannot select the operation selected by the previous sub-network model.

[0016] S3.2. Each sub-network model inherits the weight of the super-network model and is trained using the training set to update the weight of each sub-network model.

[0017] S3.3. Evaluate the sub-network models using the validation set to obtain the accuracy of each sub-network model.

[0018] S3.4. Based on the balance constraint, update the weights in the super-network model. The weights in the single-path units are updated by the weights in the single-path units of all sub-network models, and the weights in the common parts are updated by the weights in the common parts of the sub-network model with the highest accuracy.

[0019] S3.5. Repeat steps S3.1 to S3.4 until the predetermined number of iterations is reached to obtain a trained hypernetwork model.

[0020] As a further preference, the balance constraint is expressed as:

[0021]

[0022] X ε =X β

[0023] Where In T(n,E,B,η) represents the number of weight updates during the training process of the hypernetwork model, E represents the set of all convolution kernels in all single-path units, B represents the set of all convolution kernels in the common part, and η represents the number of operations contained in the operation set; x∈N + Indicates that the parameter x is a positive integer; X ε represents the number of times the weight of the convolution kernel ε is updated during the training process, X β Indicates the number of times the convolution kernel β weights are updated during the training process.

[0024] As a further preference, the operation set includes four operations: a 1×1 convolution operation, a 3×3 convolution operation, a 3×3 depth-separable convolution operation, and a 5×5 depth-separable convolution operation.

[0025] As a further preferred embodiment, the common part includes five convolution modules and two downsampling modules, wherein:

[0026] Convolutional module 1 and convolutional module 2 are connected to the input of the hypernetwork model and output a feature map;

[0027] Convolution module 3, single path unit 1, and downsampling module 1 are connected in sequence, convolution module 4, single path unit 2, and downsampling module 2 are connected in sequence, and convolution module 5 and single path unit 3 are connected in sequence; convolution modules 3 to 5 each receive two input feature maps and output two feature maps. Specifically, convolution module 3 receives the feature maps output by convolution module 1 and convolution module 2, convolution module 4 receives the feature maps output by convolution module 2 and downsampling module 1, and convolution module 5 receives the feature maps output by downsampling module 1 and downsampling module 2.

[0028] As a further preferred embodiment, in step S4, a genetic algorithm with architecture constraints is used as a search strategy for the sub-network model.

[0029] As a further preference, when training the super network model and the optimal sub-network model, a cross entropy loss function is adopted, and the parameters of the model are optimized by the Adam optimization algorithm during training.

[0030] According to a second aspect of the present invention, a fault prediction system based on a single-path unit neural network architecture search is provided, comprising a processor configured to execute the above-mentioned fault prediction method based on a single-path unit neural network architecture search.

[0031] According to a third aspect of the present invention, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the fault prediction method based on the single-path unit neural network architecture search is implemented.

[0032] In general, the above technical solutions conceived by the present invention have the following technical advantages compared with the existing technology:

[0033] 1. The present invention proposes a fault prediction method based on a neural network architecture search of a single path unit, which can automatically design the structure of a neural network fault prediction model. Compared with the traditional method of manually designing a neural network fault prediction model, this method can significantly reduce the time cost of manually designing the model and improve the performance of the neural network model in mechanical intelligent fault prediction tasks.

[0034] 2. The present invention constructs a super network model based on a single path unit as the search space for neural network architecture search. Compared with super network models of other structures, the super network model based on a single path unit narrows the search task to a single path unit. At the same time, by fixing the connection method between nodes, it only searches for candidate operations on each unidirectional edge, which can reduce the search cost of the algorithm and improve the scalability of the fault prediction model.

[0035] 3. The present invention proposes a hypernetwork training strategy based on balance constraints. Compared with other hypernetwork training strategies, by satisfying the balance constraint conditions, the weights in all convolution kernels in the hypernetwork model are updated to the same extent, which can improve the prediction ability of the hypernetwork model based on single-path units and further improve the performance of the fault prediction model searched by the algorithm. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 This is a flow chart of a fault prediction method based on a single-path unit neural network architecture search according to an embodiment of the present invention;

[0037] Figure 2 Schematic diagram of the structure of a super network model based on a single path unit according to an embodiment of the present invention. DETAILED DESCRIPTION

[0038] In order to make the objectives, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely for the purpose of explaining the present invention and are not intended to limit the present invention. In addition, the technical features involved in the various embodiments of the present invention described below may be combined with each other as long as they do not conflict with each other.

[0039] The embodiment of the present invention provides a fault prediction method based on a neural network architecture search of a single path unit, such as Figure 1 As shown, the following steps are included:

[0040] S1. Use a sliding window of length 1024 to intercept the signal from the equipment fault signal. After normalization, reshape it into a 32×32 two-dimensional matrix. This serves as the data sample of the fault signal to form a labeled fault prediction dataset. 60% of the samples in the dataset are divided into a training set and a validation set, ensuring that there are no duplicate samples in the training and validation sets. The remaining samples can be used as a test set.

[0041] S2. Construct a hypernetwork model based on a single path unit as the search space for neural network architecture search.

[0042] The super network model includes three single path units and the rest of the common parts, such as Figure 2 As shown, where:

[0043] A single-path unit internally consists of two input nodes, three intermediate nodes, and nine unidirectional edges. The input nodes are feature maps passed in from the external input node. An intermediate node is the sum of the feature maps obtained by each of the preceding nodes after a unidirectional edge. Each unidirectional edge applies a candidate operation from the operation set to the input feature map and outputs a feature map. The output of the single-path unit is obtained by concatenating the three intermediate nodes. The operation set includes four operations: 1×1 convolution, 3×3 convolution, 3×3 depthwise separable convolution, and 5×5 depthwise separable convolution. For example, the input to intermediate node 4 is the feature map obtained by applying an operation from each of the unidirectional edges to input node 1, input node 2, and intermediate node.

[0044] The common part includes five convolution modules and two downsampling modules. Convolution modules 1 and 2 are directly connected to the input of the hypernetwork model and output a feature map, which contains a 3×3 convolution filter, a batch normalization layer, and a ReLU activation function layer. Convolution modules 3, 4, and 5 accept two input feature maps and output two feature maps, which contain two convolution filters, two batch normalization layers, and two ReLU activation function layers to balance the resolution and number of channels of the two input feature maps, and then pass them to the single-path unit. The downsampling module contains a maximum pooling layer to halve the resolution of the feature map output by the single-path unit and double the number of channels. For example, for convolution module 4, its input is the feature map output by convolution module 2 and downsampling module 1. After processing, convolution module 4 inputs the two feature maps into single-path unit 1, which then outputs a feature map to downsampling module 2.

[0045] The super-network model based on single-path units includes all sub-network models by activating all candidate operations on unidirectional edges, and for each sub-network model, only one operation is activated on the unidirectional edges in the single-path unit. Therefore, the super-network model and the sub-network model have the same common parts, but the single-path unit of the sub-network model is only a branch of the single-path unit of the super-network model. At the same time, the three single-path units in the sub-network model have the same structure, which can exponentially reduce the search space and increase the possibility of the algorithm searching for a high-performance architecture. Finally, the entire super-network model contains 4 9 =262144 sub-network models.

[0046] S3. Use the hypernetwork training strategy based on balance constraints to train the hypernetwork model to obtain the trained hypernetwork model. The specific expression is:

[0047]

[0048] Among them, L train (·) is the loss function, S(A, W) is the hypernetwork model, W is the weight of the hypernetwork model, and W * is the weight of the trained hypernetwork model.

[0049] The balance constraint ensures that the weights of all convolution kernels in the hypernetwork model are updated to the same extent during training. Specifically, it is expressed as:

[0050]

[0051] X ε =X β

[0052] Where T(n, E, B, η) represents the training process of the hypernetwork model, n represents the number of weight updates during the training process, E represents the set of all convolution kernels in all single-path units, B represents the set of all convolution kernels in the common part, η represents the number of operations contained in the operation set, X represents the number of operations in the operation set, and ε represents the number of times the weight of the convolution kernel ε is updated during the training process, X β Indicates the number of times the convolution kernel β weights are updated during the training process.

[0053] Specifically, training the hypernetwork model based on the balance-constrained hypernetwork training strategy includes the following steps:

[0054] S3.1. Based on the balance constraint, the four sub-network models are sampled sequentially. On the unidirectional edge at the same location, the sub-network model cannot select the operation selected by the previous sub-network model. Specifically, the balance constraint during sampling ensures that all updateable weights in the single-path unit of the super-network model are updated to the same extent.

[0055] S3.2. Each sub-network model inherits the weight of the super-network model. Each sub-network model is trained on the training set based on the above training strategy. After training, the weight of the sub-network model is updated. It should be noted that since the weights of different operations are different, and each sub-network model selects different operations, the weights of each sub-network model are different after the sub-network model inherits the weight of the super-network model.

[0056] S3.3. Evaluate the sub-network model on the validation set to obtain the accuracy of the sub-network model;

[0057] S3.4. Update the weights in the hypernetwork model based on the balance constraint, where the weights in the single-path units are updated by the weights in the single-path units of all subnetwork models, and the weights in the common part are updated by the weights in the common part of the subnetwork model with the highest accuracy. Specifically, the balance constraint is used when updating the weights in the hypernetwork model to ensure that the updateable weights in the single-path units and the common part of the hypernetwork model are updated to the same extent.

[0058] S3.5. Repeat steps S3.1 to S3.4 until the predetermined number of iterations is reached to obtain a trained hypernetwork model.

[0059] During the training process of the hypernetwork model, the cross entropy loss function is used, and the parameters of the model are optimized through the Adam optimization algorithm during training.

[0060] S4. Use the trained super-network model to search for sub-network models on the validation set to obtain the optimal sub-network model. The specific expression is:

[0061]

[0062] Among them, ACC val represents the accuracy; S(α, W * (α)) is the sub-network model, α is the network structure of the sub-network model, W * (α) is the weight inherited by the sub-network model α from the trained super-network model, α * is the network structure of the optimal sub-network model.

[0063] A genetic algorithm with architectural constraints is used as the search strategy for sub-network models. The architectural constraints are specifically the total number of parameters that need to be trained in the model. First, the sub-network model is encoded using an integer-level encoding strategy. Second, a population that satisfies the architectural constraints is initialized. Then, the population is iterated using crossover and mutation operators. The accuracy of the sub-network model on the validation set is used as the fitness metric, and individuals with higher fitness that meet the architectural constraints are continuously searched. Finally, the individual with the highest accuracy in the population is used as the algorithm's output, thereby finding the optimal sub-network model that satisfies the architectural constraints and has the highest accuracy.

[0064] S5. Use the training set to train the optimal sub-network model from scratch; use the cross entropy loss function during the training process, and optimize the parameters of the sub-network model through the Adam optimization algorithm during training to obtain the trained sub-network model.

[0065] S6. Apply the trained sub-network model to equipment failure prediction.

[0066] The following are specific embodiments:

[0067] A bearing fault dataset collected by Case Western Reserve University (CWRU) was used. The test bench used to collect this dataset includes a 2-horsepower motor, torque sensor, dynamometer, and control electronics. The dataset contains three different fault states: inner race fault (IF), outer race fault (OF), and roller fault (RF). Each fault state has three different fault diameters: 0.18mm, 0.36mm, and 0.54mm. A total of nine different fault types, plus the normal state, result in a total of ten different health categories. Furthermore, the dataset was collected under four different loads: 0, 1, 2, and 3 hp. For each health category under each load, 1,000 data samples were collected. The dataset contains a total of 40,000 samples, with 24,000 in the training set and 8,000 in the validation set.

[0068] The optimal model searched by the neural network architecture search algorithm proposed in the present invention on the fault prediction dataset was compared with seven other deep learning fault prediction methods. They are autoencoder (AE), denoising autoencoder (DAE), sparse autoencoder (SAE), multi-layer perceptron (MLP), convolutional neural network (CNN), residual network (Resnet18), and bidirectional long short-term memory network (BILSTM). The comparison results are shown in Table 1. The evaluation index uses the average prediction accuracy of 5 repeated experiments, and the number of model parameters is used as a measure of the model size. As can be seen from the table, the model of this application achieved an average prediction accuracy of 100% with only 0.292M parameters, which achieved better prediction results than other methods. The average prediction accuracies of AE, DAE, SAE, MLP, CNN, Resnet18, and BiLSTM are 73.95%, 80.84%, 76.25%, 75.48%, 96.93%, 100%, and 89.27%, respectively, and the model sizes are 3.514M, 1.4M, 3.514M, 1.751M, 2.328M, 11.175M, 0.974M, and 0.292M, respectively.

[0069] Table 1 Comparison of prediction accuracy and model size of each method

[0070]

[0071] It can be seen from the experimental results that compared with other manually designed neural network models, the method proposed in the present invention not only realizes the automatic design of the fault prediction model and avoids the limitations of manual design, but also achieves higher prediction accuracy with fewer model parameters, achieving obvious advantages and being able to better meet the needs of modern mechanical intelligent fault prediction.

[0072] It will be easily understood by those skilled in the art that the above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A fault prediction method based on single path unit neural network architecture search, characterized in that: The steps include: S1. Collect equipment fault signals, preprocess them, obtain labeled fault prediction datasets, and divide them into training and validation sets. S2. Construct a super-network model based on a single path unit. According to the different operations selected in the single path unit, the super-network model can generate the weights of all sub-network models in the search space; The super network model includes three single path units and a common part; each single path unit includes two input nodes and three intermediate nodes, the two input nodes are connected to the three intermediate nodes respectively, and the three intermediate nodes are connected in pairs, that is, nine unidirectional edges; Specifically, the two input nodes receive feature maps from the outside and pass them to the intermediate node via a unidirectional edge. The intermediate node receives the feature maps obtained by all previous input nodes and intermediate nodes after a unidirectional edge, and superimposes the feature maps. The unidirectional edge performs an operation on the input feature map and outputs a feature map. The output of a single path unit is obtained by connecting three intermediate nodes; S3. Using the training set, a hypernetwork model is trained using a hypernetwork training strategy based on a balance constraint to obtain a trained hypernetwork model. The balance constraint is expressed as: in, Indicates that during the training process of the hypernetwork model, n Indicates the number of weight updates during training, E represents the set of all convolution kernels in all single-path units, B represents the set of all convolution kernels in the common part, Indicates the number of operations contained in the operation set; Representation parameters is a positive integer; Represents the convolution kernel The number of times the weights are updated during training, Represents the convolution kernel The number of times the weights are updated during training; S4. Use the validation set to search for the optimal sub-network model on the trained super-network model; S5. Use the training set to train the optimal sub-network model to obtain a trained optimal sub-network model; S6. Realize equipment failure prediction through the trained optimal sub-network model.

2. The fault prediction method based on single path unit neural network architecture search according to claim 1, characterized in that: Step S3, training the hypernetwork model, specifically includes the following steps: S3.

1. Based on the balance constraint, four sub-network models are sampled sequentially on the super-network model. When sampling sequentially, on the unidirectional edge at the same position, the sub-network model cannot select the operation selected by the previous sub-network model. S3.

2. Each sub-network model inherits the weight of the super-network model and is trained using the training set to update the weight of each sub-network model. S3.

3. Evaluate the sub-network models using the validation set to obtain the accuracy of each sub-network model. S3.

4. Based on the balance constraint, update the weights in the super-network model. The weights in the single-path units are updated by the weights in the single-path units of all sub-network models, and the weights in the common parts are updated by the weights in the common parts of the sub-network model with the highest accuracy. S3.

5. Repeat steps S3.1 to S3.4 until the predetermined number of iterations is reached to obtain a trained hypernetwork model.

3. The fault prediction method based on single path unit neural network architecture search according to claim 1, characterized in that: The operation set includes four operations: 1×1 convolution operation, 3×3 convolution operation, 3×3 depth-separable convolution operation, and 5×5 depth-separable convolution operation.

4. The fault prediction method based on single path unit neural network architecture search according to claim 1, characterized in that: The common part includes five convolution modules and two downsampling modules, where: Convolutional module 1 and convolutional module 2 are connected to the input of the hypernetwork model and output a feature map; Convolution module 3, single path unit 1, and downsampling module 1 are connected in sequence, convolution module 4, single path unit 2, and downsampling module 2 are connected in sequence, and convolution module 5 and single path unit 3 are connected in sequence; convolution modules 3 to 5 each receive two input feature maps and output two feature maps. Specifically, convolution module 3 receives the feature maps output by convolution module 1 and convolution module 2, convolution module 4 receives the feature maps output by convolution module 2 and downsampling module 1, and convolution module 5 receives the feature maps output by downsampling module 1 and downsampling module 2.

5. The fault prediction method based on single path unit neural network architecture search according to claim 1, characterized in that: Step S4: adopting a genetic algorithm with architecture constraints as the search strategy for the sub-network model.

6. The fault prediction method based on single path unit neural network architecture search according to any one of claims 1 to 5, characterized in that: When training the super network model and the optimal sub-network model, the cross entropy loss function is used, and the Adam optimization algorithm is used to optimize the model parameters during training.

7. A fault prediction system based on single path unit neural network architecture search, characterized in that: It includes a processor, which is used to execute the fault prediction method based on single-path unit neural network architecture search as described in any one of claims 1-6.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the fault prediction method based on single-path unit neural network architecture search according to any one of claims 1 to 6 is implemented.

Images