Fault Diagnosis Method and Device Based on Multi-Head Convolution and Differential Self-Attention

Abstract

The application relates to a fault diagnosis method based on multi-head convolution and differential self-attention, first, a data sample set obtained is processed through a multi-head convolution embedding algorithm to expand the dimension, then a layer of learnable parameters is added on the top of the sample set, each element of the sample set is added with a learnable parameter, then the sample set is processed through an improved transformer encoder module to obtain the final output, only the top layer of the final output is taken to be classified through a classifier to obtain the final classification result, and then a fault diagnosis result is obtained based on the classification result; under the condition of lacking a large number of fault samples, the application can still realize high-accuracy fault diagnosis of bearings; compared with other benchmark models and existing methods, the application has higher diagnosis precision and stability, can provide effective fault diagnosis under a strong noise environment, has good robustness, and can be used for fault diagnosis under various working conditions and has strong generalization ability.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of artificial intelligence, and in particular to a fault diagnosis method and device based on multi-head convolution and differential self-attention. BACKGROUND

[0002] In order to ensure safe and efficient operation of industrial equipment, it is necessary to diagnose and monitor the operating conditions of the equipment. In the conventional fault diagnosis, a large number of fault samples are usually needed to train the model to make the model identification accuracy meet the standard. However, in reality, it is difficult to collect a large number of fault samples due to various reasons. How to still perform high-accuracy fault diagnosis in the case of lacking a large number of fault samples is a problem that needs to be solved by those skilled in the art. SUMMARY

[0003] The present application aims to at least solve one of the problems of the prior art, and provides a fault diagnosis method and device based on multi-head convolution and differential self-attention.

[0004] In order to achieve the above-mentioned purpose, the present application adopts the following technical solutions:

[0005] Specifically, a fault diagnosis method based on multi-head convolution and differential self-attention is proposed, which comprises the following steps:

[0006] Step 110: obtaining target data, and cutting samples from the target data to form a sample set D0, wherein the dimension of the sample set D0 is [n, b], n is the number of samples, and b is the length of a single sample;

[0007] Step 120: performing multi-head convolution embedding algorithm processing on the sample set D0 to form a sample set D1, wherein the dimension of the sample set D1 is [n, b, d], and d is the expanded dimension;

[0008] Step 130: adding a layer of learnable parameters [n, 1, d] on the top of the sample set D1 to obtain a sample set X, wherein the dimension of the sample set X becomes [n, b+1, d], and then a learnable parameter is added to each element of the sample set X, and the dimension is kept unchanged;

[0009] Step 140: processing the sample set X[n, b+1, d] through an improved transformer encoder module to obtain a final output Xout;

[0010] Step 150: only taking the top layer of Xout, denoted as Xtop, wherein the dimension of Xtop is [n, 1, d], and processing Xtop through a classifier composed of two fully connected layers to obtain an output denoted as Xclass, wherein the dimension of Xclass is [n, nc];

[0011] Step 160: Based on the Xclass, obtain the final classification result, and then obtain the fault diagnosis result.

[0012] Furthermore, specifically, the operation of the improved transformer encoder module includes the following:

[0013] Step 210: Apply four fully connected layers with different parameters to the sample set X to obtain outputs Xi with dimensions [n, b+1, d / 4], where i is 1-4. Input each Xi into the differential self-attention algorithm module to obtain the corresponding output Xi-out, with the dimension unchanged. Then, concatenate the Xi-outs on the third dimension to form the output Xout_1, with the dimension of Xout_1 becoming [n, b+1, d].

[0014] Step 220: Add the obtained Xout_1 to the previous X and then standardize it to obtain Xout_2. The dimensions of Xout_2 are still [n, b+1, d].

[0015] Step 230: Map Xout_2 through two fully connected layers to obtain Xout_3, without changing the dimension;

[0016] Step 240: Add the obtained Xout_3 to the previous Xout_2 and then standardize it to obtain Xout_4, with the dimension unchanged;

[0017] Step 250: Take Xout_4 as the new sample set X" and process it through steps 210 to 240, repeating the process at least twice to obtain the final output Xout.

[0018] Furthermore, specifically, step 250 is repeated twice.

[0019] Furthermore, specifically, the operation process of the differential self-attention algorithm module includes the following:

[0020] Extract a feature layer q from Xi. k ;

[0021] q k Subtracting Xi from the result A via a broadcast mechanism yields the result A;

[0022] Squaring A and taking its opposite, then performing a softmax operation in the vertical direction, yields B.

[0023] Take the Hadamard product of B and Xi to obtain C;

[0024] The output q is obtained by summing C vertically. k ′ ;

[0025] Since Xi has a total of b+1 features qk Each feature q k Each corresponds to an output q k ′ Therefore, b+1 q k ′ This allows them to be stitched together to form Xi-out, while maintaining the same dimensions.

[0026] Specifically, the processing steps of the multi-head convolutional embedding algorithm include:

[0027] The input sample set D0 is processed by one-dimensional convolution with three different paths, and D0_1, D0_2, and D0_3 with dimensions [n, b / 3, d] are output respectively. Then, each D0_i, i is taken as 1 to 3, and they are concatenated on the second dimension to form the output D1 with dimensions [n, b, d].

[0028] This invention also proposes a fault diagnosis device based on multi-head convolution and differential self-attention, comprising the following:

[0029] The data acquisition module is used to acquire target data, extract samples from the target data, and form a sample set D0. The dimension of the sample set D0 is [n, b], where n is the number of samples and b is the length of a single sample.

[0030] The multi-head convolutional embedding algorithm module is used to process the sample set D0 using the multi-head convolutional embedding algorithm to form a sample set D1, wherein the dimension of the sample set D1 is [n, b, d], and d is the extended dimension.

[0031] The dimension calculation module is used to add a learnable parameter [n,1,d] to the top of the sample set D1 to obtain the sample set X, the dimension of which becomes [n,b+1,d]. Then, a learnable parameter is added to each element of the sample set X, while keeping the dimension unchanged.

[0032] An improved transformer encoder module is used to process the sample set X[n,b+1,d] to obtain the final output Xout;

[0033] The classification module is used to take only the top layer of Xout, denoted as Xtop, with dimensions [n, 1, d]. Xtop is processed by a classifier consisting of two fully connected layers, and the output is denoted as Xclass, with dimensions [n, nc].

[0034] The fault diagnosis module is used to obtain the final classification result based on the Xclass, and then obtain the fault diagnosis result.

[0035] This invention also proposes a bearing fault diagnosis method, which applies the aforementioned fault diagnosis method based on multi-head convolution and differential self-attention, including:

[0036] Accelerometers are installed in bearing housings during the operation of industrial equipment.

[0037] Data from the accelerometer is collected via a data acquisition card and uploaded to the host computer.

[0038] The host computer reads the data and uses it as a sample set to train a fault diagnosis model using a method based on multi-head convolution and differential self-attention.

[0039] Save and load the training model into the computer used for detection, input the operating data of the target device into the computer used for detection, and the computer performs online fault diagnosis on the target device.

[0040] The beneficial effects of this invention are as follows:

[0041] The fault diagnosis method based on multi-head convolution and differential self-attention provided by this invention first expands the dimensionality of the acquired data sample set through a multi-head convolution embedding algorithm. Then, a learnable parameter layer is added to the top of the sample set, with a learnable parameter added to each element of the sample set. The sample set is then processed by an improved transformer encoder module to obtain the final output. Only the top layer of the final output is taken and classified by a classifier to obtain the final classification result. The fault diagnosis result is then obtained based on the classification result. This invention can still achieve high-accuracy fault diagnosis of bearings even in the absence of a large number of fault samples. Compared with other benchmark models and existing methods, this invention has higher diagnostic accuracy and stability, and can provide effective fault diagnosis in strong noise environments. It has good robustness and can be used for fault diagnosis under various working conditions, with strong generalization ability. Attached Figure Description

[0042] The above and other features of this disclosure will become more apparent from the detailed description of the embodiments shown in conjunction with the accompanying drawings. In the accompanying drawings, the same reference numerals denote the same or similar output voltages. Obviously, the drawings described below are merely some embodiments of this disclosure. For those skilled in the art, other drawings can be obtained from these drawings without any creative effort. In the drawings:

[0043] Figure 1 The flowchart shown is a fault diagnosis method based on multi-head convolution and differential self-attention according to the present invention.

[0044] Figure 2The diagram shows a flowchart of the differential self-attention algorithm module of the fault diagnosis method based on multi-head convolution and differential self-attention of the present invention.

[0045] Figure 3 The diagram shows a flowchart of the bearing fault diagnosis method of the present invention.

[0046] Figure 4 The diagram shows the principle of the multi-head convolution embedding algorithm of the fault diagnosis method based on multi-head convolution and differential self-attention of the present invention. Detailed Implementation

[0047] The following will provide a clear and complete description of the concept, specific structure, and technical effects of the present invention in conjunction with embodiments and accompanying drawings, so as to fully understand the purpose, solution, and effects of the present invention. It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. The same reference numerals used throughout the accompanying drawings indicate the same or similar parts.

[0048] Reference Figure 1 as well as Figure 4 Example 1: This invention proposes a fault diagnosis method based on multi-head convolution and differential self-attention, comprising the following:

[0049] Step 110: Obtain target data, extract samples from the target data to form a sample set D0, where the dimension of the sample set D0 is [n, b], where n is the number of samples and b is the length of a single sample;

[0050] Step 120: Process the sample set D0 using a multi-head convolutional embedding algorithm to form a sample set D1, where the dimension of the sample set D1 is [n, b, d], and d is the extended dimension.

[0051] After D0 enters the module corresponding to the multi-head convolution embedding algorithm, three different paths of one-dimensional convolution are applied to D0. The parameters of each path convolution are shown in Table 1 below:

[0052]

[0053] Table 1

[0054] The outputs are D0_1, D0_2, and D0_3, each with dimensions [n, b / 3, d]. Then, the individual D0_i values ​​are concatenated along the second dimension to form the output D1, with dimensions [n, b, d].

[0055] Step 130: Add a learnable parameter [n,1,d] to the top of the sample set D1 to obtain the sample set X. The dimension of the sample set X becomes [n,b+1,d]. Then, add a learnable parameter to each element of the sample set X, keeping the dimension unchanged.

[0056] Step 140: Process the sample set X[n,b+1,d] using the improved transformer encoder module to obtain the final output Xout;

[0057] Step 150: Take only the top layer of Xout, denoted as Xtop, with dimensions [n, 1, d]. Process Xtop through a classifier consisting of two fully connected layers, and output it as Xclass, with dimensions [n, nc].

[0058] Step 160: Based on the Xclass, obtain the final classification result, and then obtain the fault diagnosis result.

[0059] In a preferred embodiment of the present invention, the operation of the improved transformer encoder module specifically includes the following:

[0060] Step 210: Apply four fully connected layers with different parameters to the sample set X to obtain outputs Xi with dimensions [n, b+1, d / 4], where i is 1-4. Input each Xi into the differential self-attention algorithm module to obtain the corresponding output Xi-out, with the dimension unchanged. Then, concatenate the Xi-outs on the third dimension to form the output Xout_1, with the dimension of Xout_1 becoming [n, b+1, d].

[0061] Step 220: Add the obtained Xout_1 to the previous X and then standardize it to obtain Xout_2. The dimensions of Xout_2 are still [n, b+1, d].

[0062] Step 230: Map Xout_2 through two fully connected layers to obtain Xout_3, without changing the dimension;

[0063] Step 240: Add the obtained Xout_3 to the previous Xout_2 and then standardize it to obtain Xout_4, with the dimension unchanged;

[0064] Step 250: Take Xout_4 as the new sample set X" and process it through steps 210 to 240, repeating the process at least twice to obtain the final output Xout.

[0065] In a preferred embodiment of the present invention, step 250 is repeated twice.

[0066] As a preferred embodiment of the present invention, refer to Figure 2 Specifically, the operation of the differential self-attention algorithm module includes the following steps:

[0067] Extract a feature layer q from Xi.k ;

[0068] q k Subtracting Xi from the result A via a broadcast mechanism yields the result A;

[0069] Squaring A and taking its opposite, then performing a softmax operation in the vertical direction, yields B.

[0070] Take the Hadamard product of B and Xi to obtain C;

[0071] The output q is obtained by summing C vertically. k ′ ;

[0072] Since Xi has a total of b+1 features q k Each feature q k Each corresponds to an output q k ′ Therefore, b+1 q k ′ This allows them to be stitched together to form Xi-out, while maintaining the same dimensions.

[0073] This invention also proposes a fault diagnosis device based on multi-head convolution and differential self-attention, comprising the following:

[0074] The data acquisition module is used to acquire target data, extract samples from the target data, and form a sample set D0. The dimension of the sample set D0 is [n, b], where n is the number of samples and b is the length of a single sample.

[0075] The multi-head convolutional embedding algorithm module is used to process the sample set D0 using the multi-head convolutional embedding algorithm to form a sample set D1, wherein the dimension of the sample set D1 is [n, b, d], and d is the extended dimension.

[0076] The dimension calculation module is used to add a learnable parameter [n,1,d] to the top of the sample set D1 to obtain the sample set X, the dimension of which becomes [n,b+1,d]. Then, a learnable parameter is added to each element of the sample set X, while keeping the dimension unchanged.

[0077] An improved transformer encoder module is used to process the sample set X[n,b+1,d] to obtain the final output Xout;

[0078] The classification module is used to take only the top layer of Xout, denoted as Xtop, with dimensions [n, 1, d]. Xtop is processed by a classifier consisting of two fully connected layers, and the output is denoted as Xclass, with dimensions [n, nc].

[0079] The fault diagnosis module is used to obtain the final classification result based on the Xclass, and then obtain the fault diagnosis result.

[0080] Combination Figure 3 This invention also proposes a bearing fault diagnosis method, which applies the aforementioned fault diagnosis method based on multi-head convolution and differential self-attention, including:

[0081] Step 310: During the operation of the industrial equipment, an acceleration sensor is installed in the bearing housing;

[0082] Step 320: Collect data from the accelerometer using the data acquisition card and upload it to the host computer;

[0083] Step 330: The host computer reads the data and uses it as a sample set to train the model using a fault diagnosis method based on multi-head convolution and differential self-attention.

[0084] Step 340: Save and load the training model into the computer used for detection, input the operating data of the target device into the computer used for detection, and the computer performs online fault diagnosis on the target device.

[0085] The specific application process is as follows:

[0086] The validation was performed using a public bearing dataset from Western Reserve University. The specific procedure for this experiment is as follows:

[0087] 1. The dataset contains 10 fault types. A number of samples are extracted from each type to form a sample set with a length of 4960. The samples are then input into the model for training with a batch size of 150. The sample data D0 for each training session is [150, 4960].

[0088] 2. Input D0 into the multi-head convolution module and expand the dimension d to 96 to obtain D1[150,155,96].

[0089] 3. Add a layer of learnable parameters [150,1,96] on top of D1, and then add another batch of learnable parameters [1,156,96] to D1 through a broadcast mechanism to obtain X, with dimensions [150,156,96].

[0090] 4. Input X into the encoder module mentioned above. X is mapped to X1, X2, X3, and X4. The operation process is the same for each Xi, only the parameters are different. Figure 3 The formula for C in the formula is:

[0091] C = softmax(-1×(q) k -X i ) 2 )⊙X i(1)

[0092] Where ⊙ represents the Hadamard product, C is the dimension [150, 156, 24], and q′ is obtained by summing along the second dimension. k Its dimensions are [150, 1, 24], and 1 q k Corresponding to a q′ k Therefore, there are a total of 156 q′. k X is formed by splicing together in the second dimension. i-out Its dimensions are [150, 156, 24]. Let X... 1-out X 2-out X 3-out and X 4-out By piecing them together in the third dimension, we get X. out_1 The dimensions are [150, 156, 96].

[0093] 5. Place X out_1 Adding it to X and then standardizing it, we get X. out_2 With the dimension unchanged, the formula is as follows:

[0094] X out_2 =Ln(X out_1 +X) (2)

[0095] Ln() represents the layer normalization method.

[0096] 6. Place X out_2 X is obtained through two fully connected layers. out_3 With the dimension unchanged, the formula is as follows:

[0097] X out_3 =ReLU(X) out_2 (3) ×W1+b1)×W2+b2

[0098] Where W1, W2 and b1, b2 are the weights and biases of the two fully connected layers, respectively. ReLU is an activation function.

[0099] 7. Place X out_3 With X out_2 After adding and standardizing, we get X. out_4 The dimension remains unchanged, similar to formula (2).

[0100] 8.X out_4 After repeating steps 4 to 7 twice, X is obtained. out .

[0101] 9. Take X out The top layer is X top The dimensions are [150, 1, 96]. X top X is obtained through the classifier. classWith dimensions [150, 10], the classification of the samples is now complete. X top The formula is as follows; GeLU is also an activation function.

[0102] X top =softmax(GeLU(X) out (×W1+b1)×W2+b2).

[0103] The modules described as separate components may or may not be physically separate. The components shown as modules may or may not be physical modules; that is, they may be located in one place or distributed across multiple network modules. Some or all of the modules can be selected to achieve the purpose of the solution in this embodiment, depending on actual needs.

[0104] Furthermore, the functional modules in the various embodiments of the present invention can be integrated into one processing module, or each module can exist physically separately, or two or more modules can be integrated into one module. The integrated modules described above can be implemented in hardware or as software functional modules.

[0105] If the integrated module is implemented as a software functional module and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments of the present invention can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include: any entity or device capable of carrying the computer program code, a recording medium, a USB flash drive, a portable hard drive, a magnetic disk, an optical disk, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical carrier signal, a telecommunication signal, and a software distribution medium, etc.

[0106] Although the description of the invention has been quite detailed and particularly of several described embodiments, it is not intended to limit it to any of these details or embodiments or any particular embodiment, but should be considered as providing a broad possible interpretation of the claims by referring to the appended claims and taking into account the prior art, thereby effectively covering the intended scope of the invention. Furthermore, the invention has been described above with respect to embodiments foreseeable by the inventors in order to provide a useful description, and non-substantial modifications to the invention that have not yet been foreseen may still represent equivalent modifications.

[0107] The above description is merely a preferred embodiment of the present invention. The present invention is not limited to the above-described embodiments. Any embodiment that achieves the technical effects of the present invention using the same means should fall within the protection scope of the present invention. Within the protection scope of the present invention, various modifications and variations can be made to the technical solutions and / or implementation methods.

Claims

1. A fault diagnosis method based on multi-head convolution and differential self-attention, characterized in that, The method comprises the following steps: Step 110, obtaining target data, and cutting samples from the target data to form a sample set D0, wherein the dimension of the sample set D0 is [n, b], n is the number of samples, and b is the length of a single sample; Step 120, performing multi-head convolution embedding algorithm processing on the sample set D0 to form a sample set D1, wherein the dimension of the sample set D1 is [n, b, d], and d is the expanded dimension; Step 130, adding a layer of learnable parameters [n, 1, d] on the top of the sample set D1 to obtain a sample set X, wherein the dimension of the sample set X becomes [n, b+1, d], and then a learnable parameter is added to each element of the sample set X, and the dimension remains unchanged; Step 140, performing processing on the sample set X [n, b+1, d] through an improved transformer encoder module to obtain a final output Xout; Step 150, only taking the top layer of the Xout, denoted as Xtop, wherein the dimension of the Xtop is [n, 1, d], and performing processing on the Xtop through a classifier composed of two fully connected layers to obtain an output denoted as Xclass, wherein the dimension of the Xclass is [n, nc]; Step 160, obtaining a final classification result based on the Xclass, and then performing fault diagnosis; Specifically, the action process of the improved transformer encoder module comprises the following steps, Step 210, performing action on the sample set X using four different parameters of fully connected layers to obtain outputs Xi with the dimension of [n, b+1, d / 4], i takes 1-4, inputting each Xi into a differential self-attention algorithm module to obtain a corresponding output Xi-out, the dimension remains unchanged, then splicing the Xi- outs in the third dimension to form an output Xout_1, and the dimension of the Xout_1 becomes [n, b+1, d]; Step 220, adding the obtained Xout_1 and the previous X and then performing standardization to obtain Xout_2, wherein the dimension of the Xout_2 is still [n, b+1, d]; Step 230, mapping the Xout_2 through two fully connected layers to obtain Xout_3, and the dimension does not change; Step 240, adding the obtained Xout_3 and the previous Xout_2 and then performing standardization to obtain Xout_4, and the dimension does not change; Step 250, Xout_4 as the new sample set Further processing through steps 210 to 240 and repeating at least twice, the final output Xout is obtained; Specifically, the action process of the differential self-attention algorithm module comprises the following steps, extracting a layer of features from Xi ; Xi is subtracted from X2 through a broadcast mechanism to obtain result A; Taking the square of A, taking the inverse, and then performing softmax in the vertical direction to obtain B; Performing Hadamard product on B and Xi to obtain C; C is summed longitudinally to obtain output ; Since Xi shares b+1 features Each feature Corresponds to an output So b+1 features Can be concatenated to form Xi-out with the same dimension.

2. The fault diagnosis method based on multi-head convolution and differential self-attention according to claim 1, characterized in that, Specifically, the number of repetitions in step 250 is 2 times.

3. The fault diagnosis method based on multi-head convolution and differential self-attention according to claim 1, characterized in that, Specifically, the processing process of the multi-head convolution embedding algorithm comprises, Performing processing on the input sample set D0 using one-dimensional convolutions of three different paths to respectively output D0_1, D0_2 and D0_3 with the dimension of [n, b / 3, d], then splicing the D0_i, i takes 1 to 3, in the second dimension to form an output D1, and the dimension is [n, b, d].

4. The fault diagnosis device based on multi-head convolution and differential self-attention, characterized in that, The device comprises the following modules by applying the steps of the method in any one of claims 1-3: a data acquisition module, configured to acquire target data, and cut samples from the target data to form a sample set D0, wherein the sample set D0 has a dimension of [n, b], n is the number of samples, and b is the length of a single sample; a multi-head convolution embedding algorithm module, configured to perform multi-head convolution embedding algorithm processing on the sample set D0 to form a sample set D1, wherein the sample set D1 has a dimension of [n, b, d], and d is an expanded dimension; a dimension operation module, configured to add a layer of learnable parameters [n, 1, d] on top of the sample set D1 to obtain a sample set X, wherein the dimension of the sample set X becomes [n, b+1, d], and then a learnable parameter is added to each element of the sample set X, and the dimension remains unchanged; an improved transformer encoder module, configured to process the sample set X[n, b+1, d] to obtain a final output Xout; a classification module, configured to take only a top layer of the Xout, denoted as Xtop, wherein the dimension of the Xtop is [n, 1, d], and the Xtop is processed by a classifier composed of two fully connected layers to obtain an output denoted as Xclass, wherein the dimension of the Xclass is [n, nc]; a fault diagnosis module, configured to obtain a final classification result based on the Xclass, and then perform fault diagnosis.

5. A method of diagnosing a failure of a bearing, characterized by, The method for fault diagnosis based on multi-head convolution and differential self-attention in any one of claims 1-3 comprises the following steps: industrial equipment is running, and an acceleration sensor is installed on a bearing seat; data of the acceleration sensor is collected by a collection card and uploaded to an upper computer; the upper computer reads the data as a sample set, and the sample set is trained by the method for fault diagnosis based on multi-head convolution and differential self-attention to obtain a training model; the training model is saved and loaded into a computer for detection, and running data of a target equipment is input into the computer for detection, and the computer performs online fault diagnosis on the target equipment.

Images