Small-sample Structural Fault Diagnosis Method Based on Transformer Neural Network

Abstract

The present invention discloses a few-shot mechanical structure fault diagnosis method based on the Transformer neural network. The impedance signals of the mechanical structure under different fault conditions are collected by piezoelectric sensors, and the collected signals are collected and processed by the PSD method. Then, the signals are synthesized to obtain amplitude-phase signal data. After sampling, the data is sent into the Transformer neural network for network training to obtain the Transformer classification network. The faults of the mechanical structure can be diagnosed by the Transformer by inputting the signal to be classified. When performing mechanical structure fault diagnosis and classification, the present invention can train an effective classification model to classify a large amount of detection data under the condition of using a few-shot Transformer neural network training data, and can diagnose the fault types of the mechanical structure with high accuracy, having high practicability and feasibility.

Description

Technical Field

[0001] The present invention relates to a fault diagnosis method for a mechanical structure, and particularly to a fault diagnosis method for a mechanical structure based on a Transformer neural network. Background Art

[0002] Beam frame mechanical structures widely exist in various machines, and the cantilever beam structure is indispensable in all aspects of production and life. However, due to long-term use, possible improper operations, and the influence of the natural environment, etc., the mechanical structure may gradually be damaged, or even serious structural failures may occur, posing serious safety hazards. Therefore, it is very important to regularly check the condition of the mechanical structure and establish an effective safety monitoring system for safe production and life. Based on this requirement, it is very important to conveniently perform non-destructive monitoring on the mechanical structure and efficiently judge the fault condition of the mechanical structure. In the past, researchers usually used convolutional neural networks to diagnose faults in mechanical structures. However, convolutional neural networks focus on local data features, pay insufficient attention to global features, and cannot utilize the position information existing in the data, which is very important for beam frame structures. At the same time, convolutional neural networks will forget earlier data information as the input sequence grows, and have insufficient ability to capture information of long sequences. Some researchers have also adopted recurrent neural networks to effectively capture long-sequence information through a sequence structure. However, the sequence structure adopted by recurrent neural networks causes the network not to be able to perform parallel computing, and it takes a lot of time to use, with low efficiency. At the same time, in practical applications, we often face diagnosing a large amount of structural data using limited samples, and both convolutional neural networks and recurrent neural networks have poor effects when using small samples to diagnose a large amount of data. Therefore, there is an urgent need for a new solution to solve the above problems. Summary of the Invention

[0003] Object of the Invention: Aiming at the above problems, the present invention proposes a fault diagnosis method for a mechanical structure based on a Transformer neural network, which improves the accuracy of using limited samples for mechanical structure fault diagnosis and realizes effective diagnosis of the fault types of the mechanical structure.

[0004] Technical solution: The technical solution adopted by the present invention is a mechanical structure fault diagnosis system based on a Transformer neural network, including a data acquisition module, a data processing module, a model training module, and a data classification module. Among them, the data acquisition module uses piezoelectric sensors to collect response signal data under different fault conditions of the mechanical structure; the data processing module uses the PSD method to process the collected signals, and after synthesizing, sampling, and labeling the signals, an amplitude-phase signal data set is obtained; the model training module uses the amplitude-phase signal data set to train the Transformer neural network to obtain a Transformer classification model; the data classification module processes the signal to be classified measured by the piezoelectric sensor and inputs it into the Transformer classification model to perform mechanical structure fault diagnosis and obtain the mechanical structure fault type.

[0005] The data acquisition module includes two piezoelectric sensors. An alternating current excitation voltage is applied to one of them, and the other collects the response voltage and obtains the required response mechanical impedance signal data through signal processing.

[0006] The data processing module processes through phase-sensitive detection. The collected response signals are respectively input into two multipliers after passing through an amplifier. At the same time, the excitation signal is used as a reference signal. One way is directly input into the multiplier, and the other way is input into another multiplier after passing through a phase shifter. Then, the outputs of the two multipliers are subjected to root mean square after passing through a low-pass filter to obtain the response signal data to be measured. Then, the data signal is sliced and processed into a two-dimensional matrix form and input into the Transformer neural network.

[0007] The Transformer neural network adopted in the model training and data classification modules mainly consists of a linear and position encoding layer, an encoding layer, a combination layer, and a linear classification layer. Among them, the linear and position encoding layer is used to retain the position information of the data; the encoding layer is used to extract the features of the response signal data from two different directions of the input data signal; the combination layer is used to combine the data features extracted from different directions; the linear classification layer is used to linearly map the extracted data features to diagnose the fault type of the mechanical structure. First, the Transformer neural network is trained using small sample data, and then the data to be classified is input and classified using the Transformer model. The Transformer neural network used includes the following parts:

[0008] First, when the linear and position encoding layer processes the input data signal matrix, it processes from two directions of rows and columns respectively. Among them, the row processing is consistent with the original order of the signal, and the additional encoding information retains the data position information, and the column processing supplements the features.

[0009] Then, an encoding layer composed of several identical encoding modules is used. Each encoding module includes multi-head self-attention and a feed-forward network layer. Among them, multi-head self-attention is used to capture the relationships between feature data. Each multi-head self-attention mechanism is composed of multiple self-attention mechanisms in parallel. After each self-attention mechanism independently and parallelly obtains feature values and splices them, they are linearly transformed into the output values of the multi-head self-attention mechanism; the feed-forward network averages the output in the normalization summation into a form that is easier to process for the next layer; within each layer, residual connections are adopted and normalization operations are performed to smooth the loss, solve the degradation problem caused by deepening the depth, and improve the network efficiency.

[0010] Finally, a combination layer is used to combine the data features extracted in different directions, and a linear classification layer is used to perform linear mapping on the data features extracted by the encoding layer and then perform classification diagnosis to obtain the fault types of the mechanical structure.

[0011] Beneficial effects: Compared with the prior art, the advantages of the present invention are as follows: A method for diagnosing mechanical structure faults based on a Transformer neural network is proposed. Compared with traditional convolutional neural networks and recurrent neural networks, it effectively solves the problems that convolutional neural networks focus on local features and ignore global features and the problem of forgetting information of long sequences, as well as the problem that recurrent neural networks cannot perform parallel computing, resulting in low model efficiency and low utilization rate; it solves the problem that in practice, due to limited available samples and a large amount of data to be classified and diagnosed, the accuracy of convolutional neural networks and recurrent neural networks is low when using small samples to classify a large amount of data. Description of the Drawings

[0012] Figure 1 is a flowchart of the method for diagnosing mechanical structure faults based on a Transformer neural network according to the present invention;

[0013] Figure 2 is a diagram of the signal acquisition device according to the present invention;

[0014] Figure 3 is a signal processing flowchart according to the present invention;

[0015] Figure 4 is a structural diagram of the Transformer neural network according to the present invention. Detailed Embodiments

[0016] The technical solutions of the present invention will be further described below with reference to the drawings and embodiments.

[0017] Such as Figure 1As shown in the figure, it is a flowchart of a mechanical structure fault diagnosis method based on a Transformer neural network, including a data acquisition module, a data processing module, a model training module, and a data classification module. The data acquisition module is used to collect impedance response signal data of the mechanical structure under different fault conditions by using piezoelectric sensors; secondly, the data processing module uses the PSD method to process the collected signals, and after signal synthesis, sampling, and labeling, an amplitude-phase signal dataset is obtained; then, the model training module uses the amplitude-phase signal dataset to train the Transformer neural network to obtain a Transformer classification model; finally, the data classification module processes the data signals to be diagnosed and classified collected by the piezoelectric sensors and inputs them into the Transformer classification model for mechanical structure fault diagnosis to obtain the mechanical structure fault type.

[0018] As Figure 2 shown in the figure, it is a signal acquisition device diagram, including two piezoelectric sensors, a cantilever beam, an excitation voltage, and a response voltage signal to be collected. Two piezoelectric sensors are attached to the surface of the mechanical structure in parallel. One of the piezoelectric sensors is used as a driver to apply an alternating excitation voltage, and the other is used as a receiver to measure the collected response voltage signal from it.

[0019] As Figure 3 shown in the figure, it is a signal processing flowchart. The collected response signal is input into two multipliers after passing through an amplifier. At the same time, the excitation signal is used as a reference signal. One path is directly input into the multiplier, and the other path is input into another multiplier after passing through a phase shifter. Then, the outputs of the two multipliers are subjected to root mean square after passing through a low-pass filter to obtain the amplitude data of the response signal to be measured.

[0020] As Figure 4 shown in the figure, it is the structure diagram of the Transformer neural network used. The adopted Transformer neural network mainly consists of a linear and position encoding layer, an encoding layer, a combination layer, and a linear classification layer.

[0021] The position encoding layer slices the input one-dimensional data signal into a two-dimensional matrix, maps the signal features row by row (i.e., following the original data position order) and adds position encoding information to input into the encoding layer, and maps the signal features column by column and directly inputs into the encoding layer. The position encoding can be expressed as follows:

[0022]

[0023]

[0024] where pos represents the position and i represents the dimension.

[0025] The encoding layers for row features and column features are identical in structure, both being stacked by 8 identical encoding structures. Each encoding structure consists of two parts: multi-head self-attention and a feed-forward network. Residual connections are used within each part, and the output can be expressed as:

[0026] output = LN(x + function(x)) (3)

[0027] where output is the output of each part, x is the input, LN is the normalization function, and function is the function of each part.

[0028] Multi-head self-attention is used to capture the relationships between feature data. Each multi-head self-attention mechanism is composed of multiple self-attention mechanisms in parallel. After each self-attention mechanism independently and parallelly obtains the feature values, they are concatenated and then linearly transformed into the output value of the multi-head self-attention mechanism. Each self-attention mechanism is only responsible for calculating one subspace of the multi-head self-attention, and a total of 8 self-attentions are used in parallel to form the multi-head self-attention. Each self-attention encodes the input feature sequence into a vector sequence, effectively retaining all the information of the long sequence. At the same time, parallel computing can effectively improve the computing efficiency. Then, the feature values of each subspace are concatenated into the output value of the multi-head self-attention, and the multi-head mechanism can effectively capture various relationships within the sequence and fully describe the various different relationships existing within the sequence. The calculation of multi-head self-attention can be expressed as:

[0029]

[0030] head i = Attention(Q·W i Q , K·W i K , V·W i V ) (5)

[0031] MultiHead(Q, K, V) = Concat(head1,..., head8)W O (6)

[0032] where Q, K, V come from the same input, d k is the dimension of the key, d model = 512, h = 8, d k = d v = d model / h = 64, softmax is used to obtain the weights of the values, and the Concat function concatenates the self-attention mechanisms of different subspaces to obtain the output value of the multi-head self-attention.

[0033] The combined layer and the linear classification layer combine the output features of the encoding layers of the row features and the column features and then perform classification diagnosis through linear mapping. It can be expressed as follows:

[0034] W(1,2) = softmax(Concat(Encoding1, Encoding2)) (7)

[0035] Encoding = Concat(Encoding1·W(1), Encoding2·W(2)) (8)

[0036] Among them, Encoding1 and Encoding2 are the output values of the encoding layers of the row features and the column features respectively. The softmax calculates the corresponding weights W(1,2). The Concat function concatenates the output values of the encoding layers of the row features and the column features. Encoding is the final output value of the encoding layer. Then, the output value is input into the linear layer to perform classification diagnosis.

[0037] Compare the mechanical structure fault diagnosis method based on the Transformer neural network proposed in the present invention with the convolutional neural network to verify the effectiveness of the mechanical structure fault diagnosis method based on the Transformer neural network proposed in the present invention:

[0038] In this embodiment, the response signals measured by the cantilever beam mechanical structure under different conditions are used as the data set. There are 4 categories in total, with 1000 data in each category, which are divided into 800 training data and 3200 test data. The learning rate is set to 0.0001, the batch size is set to 10, and the number of iteration rounds is set to 200. The obtained classification accuracy rates are as follows:

[0039] Table 1 Comparison of Fault Diagnosis Accuracy Rates of Cantilever Beam Mechanical Structures

[0040]

[0041] It can be seen that in this embodiment, the accuracy rate of the small-sample fault diagnosis of the cantilever beam mechanical structure by the Transformer neural network structure proposed in the present invention is significantly better than that of the other two convolutional neural networks, which shows that the mechanical structure fault diagnosis method based on the Transformer neural network proposed in the present invention has obvious advantages and more practical application value.

[0042] Taking the above ideal embodiment based on the present invention as an inspiration, through the above description, relevant staff can completely make various changes and modifications within the scope not deviating from the technical idea of this invention. The technical scope of this invention is not limited to the content in the specification, and its technical scope must be determined according to the scope of the claims.

Claims

1. A few-shot mechanical structure fault diagnosis method based on the Transformer neural network, characterized in that It includes the following steps: Data acquisition, using piezoelectric sensors to acquire impedance signal data under different fault conditions of the mechanical structure; Data processing, using the PSD method to process the acquired signals, and obtaining an amplitude-phase signal dataset after signal synthesis, sampling, and labeling; Model training, using the amplitude-phase signal dataset to train a Transformer neural network to obtain a Transformer classification model; Data classification, processing the signal to be classified measured by the piezoelectric sensor and inputting it into the Transformer classification model for mechanical structure fault diagnosis to obtain the mechanical structure fault type; The Transformer neural network adopted in the model training consists of a linear and position encoding layer, an encoding layer, a combination layer, and a linear classification layer; Among them, the encoding layer is used to retain the position information of the data; the linear and position encoding layer is used to extract the features of the response signal data from two different directions of the input data signal; the combination layer is used to combine the data features extracted from different directions; the linear classification layer is used to linearly map the extracted data features and then diagnose the mechanical structure fault type; the position encoding layer slices the input one-dimensional data signal into a two-dimensional matrix, maps the signal features row by row and adds position encoding information to input into the encoding layer, and maps the signal features column by column and directly inputs into the encoding layer.

2. The small-sample mechanical structure fault diagnosis method based on the Transformer neural network according to claim 1, wherein: The data processing is specifically carried out through phase-sensitive detection. The acquired response signal is amplified and then input into two multipliers respectively. At the same time, the excitation signal is used as the reference signal, one is directly input into the multiplier, and the other is input into the other multiplier after passing through a phase shifter, and then the outputs of the two multipliers are subjected to root mean square after passing through a low-pass filter to obtain the response signal data to be measured, and then the data signal is sliced and processed into a two-dimensional matrix form and input into the Transformer neural network.

3. The small-sample mechanical structure fault diagnosis method based on the Transformer neural network according to claim 1, characterized in that: When processing the input data signal matrix, the linear and position encoding layer processes it from two directions of row and column respectively. Among them, the row processing is consistent with the original order of the signal, and the additional encoding information retains the data position information, and the column processing supplements the features.

4. The small-sample mechanical structure fault diagnosis method based on the Transformer neural network according to claim 1, characterized in that: The encoding layer consists of several identical encoding modules, and each encoding module includes multi-head self-attention and a feed-forward network layer; among them, multi-head self-attention is used to capture the relationship between feature data; the feed-forward network averages the output in the normalization summation into a form that is easier to process for the next layer.

5. The small-sample mechanical structure fault diagnosis method based on the Transformer neural network according to claim 4, wherein: Both the multi-head self-attention and the feed-forward network layer adopt residual connections and normalization operations inside, smooth the loss, solve the degradation problem caused by deepening, and improve the network efficiency.

6. The small-sample mechanical structure fault diagnosis method based on the Transformer neural network according to claim 4, characterized in that: The multi-head self-attention is composed of multiple self-attention mechanisms in parallel. Each self-attention mechanism independently obtains the eigenvalue in parallel and then splices and linearly transforms it into the output value of the multi-head self-attention mechanism.

Images