Transform neural network and method for multi-feature grouping

Abstract

The invention discloses a multi-feature grouping Transform neural network and a method, and relates to the technical field of motor testing. The neural network comprises a multi-feature extraction module, a Transform feature extraction module and a feature grouping fusion classification module which are connected in sequence, the multi-feature extraction module comprises n types of MFCC modules, and the Transform feature extraction module comprises n types of Transform modules and an embedded full-connection layer; the feature grouping fusion classification module comprises a splicing module, a feature comprehensive full connection layer, an average pooling layer and a dichotomy full connection layer; the method comprises the steps of S1, multi-feature extraction, S2, deep feature extraction through Transform, and S3, feature grouping, fusion and classification to obtain a dichotomy result. The splicing embedding feature vector overcomes the problems of information mixing and strong interference, reduces the parameter quantity, and reduces the calculation quantity.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of motor testing, and in particular to a multi-feature grouping Transformer neural network and method. BACKGROUND

[0002] The steps of the machine identifying motor abnormal equipment are feature extraction and model prediction. In the feature extraction stage, the common method is to convert the time domain data of the motor into a mel frequency cepstral coefficient (MFCC) feature matrix. Different numbers of mel coefficients have advantages and disadvantages: a MFCC with a smaller number of mel frequency cepstral coefficients can remove redundant information, reduce computational cost, and improve efficiency; a MFCC with a larger number of mel frequency cepstral coefficients can capture more rich speech spectrum features and improve classification effect. Therefore, in the feature extraction stage, using only one MFCC dimension information may reduce the accuracy of motor sound classification.

[0003] In the model prediction stage, the commonly used model is the Transformer model. Compared with RNN and CNN, the Transformer model has significant advantages in processing sequence data such as sound and text.

[0004] In the present application, n kinds of MFCCs, i.e., MFCCs with different numbers of mel frequency cepstral coefficients (n_mfcc), are used to extract features from the original audio, obtaining n kinds of MFCC feature matrices. Different kinds of MFCC feature matrices represent different frequency resolutions and time perception abilities of the audio signal, and are directly concatenated into the Transformer, resulting in mixed information and enhanced interference.

[0005] The n kinds of embedding feature vectors e1, e2, ……, en are obtained through a fully connected layer to obtain a comprehensive feature vector.

[0006] Suppose the dimension of each embedding vector is (10, 1), then the overall input dimension is (10 x n, 1). For example, e1, e2 to e10, a total of ten embeddings, are all input into the fully connected layer. The shape of each embedding feature vector is (10, 1), and the overall input dimension is (100, 1). The shape of the fully connected layer is (100, out), and the shape of the 1 feature vector obtained by the fully connected layer output is (out, 1). Since the shape of the fully connected layer is (100, out), the number of parameters is too large, resulting in a large amount of calculation and low computational efficiency.

[0007] The value of n is reduced to 3, and the three embeddings e1, e2 and e3 are all input into the full connection layer, the shape of each embedding feature vector is (10, 1), the input dimension 10*3 is too large, the shape of the full connection layer is (30, out), the shape of the obtained 1 feature vector output by the full connection layer is (out, 1), since the shape of the full connection layer is (30, out), the number of parameters is too large, and the calculation amount is still large, and the calculation efficiency is low.

[0008] Therefore, information mixing, strong interference and large calculation amount become technical problems to be solved. SUMMARY

[0009] The application provides a multi-feature grouping Transformer neural network and method, which solve the technical problems of information mixing, strong interference and large calculation amount.

[0010] To solve the above technical problems, the technical solution adopted by the application is as follows: A multi-feature grouping Transformer neural network, comprising a multi-feature extraction module, a Transformer feature extraction module and a feature grouping fusion classification module connected in sequence, the multi-feature extraction module comprises n first to nth MFCC modules, the Transformer feature extraction module comprises n first to nth Transformer modules and n first to nth embedding full connection layers, one MFCC module is connected with a corresponding Transformer module, and one Transformer module is connected with a corresponding embedding full connection layer; the feature grouping fusion classification module comprises first to n* (n-1) / 2 splicing modules, a shared weight feature comprehensive full connection layer, an average pooling layer and a binary classification full connection layer, each two embedding full connection layers are combined and fused with one splicing module, all splicing modules are connected with the shared weight feature comprehensive full connection layer, the shared weight feature comprehensive full connection layer is connected with the average pooling layer, and the average pooling layer is connected with the binary classification full connection layer.

[0011] Further technical solutions are that the number of mel frequency cepstrum coefficients of the MFCC module n_mfcc is in the range of 13-128.

[0012] Further technical solutions are that the value of n is in the range of 2-6.

[0013] Further, the technical scheme is that one MFCC module is connected with a corresponding Transformer module, one Transformer module is connected with a corresponding embedded full connection layer, that is, the first MFCC module is connected with the first Transformer module, the first Transformer module is connected with the first embedded full connection layer, the second MFCC module is connected with the second Transformer module, the second Transformer module is connected with the second embedded full connection layer, and the n-th MFCC module is connected with the n-th Transformer module, and the n-th Transformer module is connected with the n-th embedded full connection layer.

[0014] Further, the technical scheme is that one MFCC module is connected with a corresponding Transformer module, one Transformer module is connected with a corresponding embedded full connection layer, that is, the first MFCC module is connected with the first Transformer module, the first Transformer module is connected with the first embedded full connection layer, the second MFCC module is connected with the second Transformer module, the second Transformer module is connected with the second embedded full connection layer, and the n-th MFCC module is connected with the n-th Transformer module, and the n-th Transformer module is connected with the n-th embedded full connection layer.

[0015] A method for detecting a motor according to the multi-feature grouping Transformer neural network described above, comprising the following steps, Step S1: obtaining a motor sound time domain signal, and obtaining a first MFCC feature matrix with a shape of (n_frame, n_mfcc1) to an n-th MFCC feature matrix with a shape of (n_frame, n_mfccn) from the motor sound time domain signal through a multi-feature extraction module; Step S2: obtaining a first embedded feature vector e1 to an n-th embedded feature vector en with a shape of (transformer_dim, 1) from the first MFCC feature matrix with a shape of (n_frame, n_mfcc1) to the n-th MFCC feature matrix with a shape of (n_frame, n_mfccn) through a Transformer feature extraction module; Step S3: the first embedding feature vector e1 to the n-th embedding feature vector en in the shape of (transformer_dim, 1) are fused by the feature grouping fusion classification module to obtain a binary classification result, which is normal or abnormal.

[0016] Further technical solutions are as follows: in the step S1, the motor sound time domain signal is extracted by the first MFCC module to obtain a first MFCC feature matrix in the shape of (n_frame, n_mfcc1), the motor sound time domain signal is extracted by the second MFCC module to obtain a second MFCC feature matrix in the shape of (n_frame, n_mfcc2), and the motor sound time domain signal is extracted by the n-th MFCC module to obtain an n-th MFCC feature matrix in the shape of (n_frame, n_mfccn).

[0017] Further technical solutions are as follows: in the step S2, the first MFCC feature matrix in the shape of (n_frame, n_mfcc1) is extracted by the first Transformer module to obtain a first Transformer feature matrix in the shape of (n_frame, n_mfcc1), and the first Transformer feature matrix in the shape of (n_frame, n_mfcc1) is output by the first embedded full connection layer to obtain a first embedding feature vector e1 in the shape of (transformer_dim, 1); the second MFCC feature matrix in the shape of (n_frame, n_mfcc2) is extracted by the second Transformer module to obtain a second Transformer feature matrix in the shape of (n_frame, n_mfcc2), and the second Transformer feature matrix in the shape of (n_frame, n_mfcc2) is output by the second embedded full connection layer to obtain a second embedding feature vector e2 in the shape of (transformer_dim, 1); and the n-th MFCC feature matrix in the shape of (n_frame, n_mfccn) is extracted by the n-th Transformer module to obtain an n-th Transformer feature matrix in the shape of (n_frame, n_mfccn), and the n-th Transformer feature matrix in the shape of (n_frame, n_mfccn) is output by the n-th embedded full connection layer to obtain an n-th embedding feature vector en in the shape of (transformer_dim, 1).

[0018] Further, in the step S3, the first embedding feature vector e1 of shape (transformer_dim, 1) and the second embedding feature vector e2 of shape (transformer_dim, 1) are spliced by the first splicing module to obtain a first spliced feature vector of shape (2*transformer_dim, 1) after splicing, the first embedding feature vector e1 of shape (transformer_dim, 1) and the third embedding feature vector e3 of shape (transformer_dim, 1) are spliced by the second splicing module to obtain a second spliced feature vector of shape (2*transformer_dim, 1) after splicing, and so on until the first embedding feature vector e1 of shape (transformer_dim, 1) and the n-th embedding feature vector en of shape (transformer_dim, 1) are spliced by the (n-1)-th splicing module to obtain an (n-1)-th spliced feature vector of shape (2*transformer_dim, 1) after splicing; The second embedding feature vector e2 of shape (transformer_dim, 1) and the third embedding feature vector e3 of shape (transformer_dim, 1) are spliced by the n-th splicing module to obtain an n-th spliced feature vector of shape (2*transformer_dim, 1) after splicing, the second embedding feature vector e2 of shape (transformer_dim, 1) and the fourth embedding feature vector e4 of shape (transformer_dim, 1) are spliced by the (n+1)-th splicing module to obtain an (n+1)-th spliced feature vector of shape (2*transformer_dim, 1) after splicing, and so on until the second embedding feature vector e2 of shape (transformer_dim, 1) and the n-th embedding feature vector en of shape (transformer_dim, 1) are spliced by the 2n-3-th splicing module to obtain a 2n-3-th spliced feature vector of shape (2*transformer_dim, 1) after splicing; And so on until the (n-1)-th embedding feature vector en-1 of shape (transformer_dim, 1) and the n-th embedding feature vector en of shape (transformer_dim, 1) are spliced by the n×(n-1) / 2-th splicing module to obtain an n×(n-1) / 2-th spliced feature vector of shape (2*transformer_dim, 1) after splicing.

[0019] Further technical solutions are that in the step S3, all spliced feature vectors are obtained n x (n-1) / 2 comprehensive feature vectors with a shape of (transformer_dim, 1) through a shared weight feature comprehensive full connection layer, all comprehensive feature vectors are obtained a feature vector with a shape of (transformer_dim, 1) through an average pooling layer, and the feature vector with a shape of (transformer_dim, 1) is obtained a binary classification result through a binary classification full connection layer.

[0020] The beneficial effects produced by the above technical solutions are that: A multi-feature grouping Transformer neural network comprises a multi-feature extraction module, a Transformer feature extraction module and a feature grouping fusion classification module connected in sequence, the multi-feature extraction module comprises n first to nth MFCC modules, the Transformer feature extraction module comprises n first to nth Transformer modules and n first to nth embedded full connection layers, one MFCC module is connected with a corresponding Transformer module, and one Transformer module is connected with a corresponding embedded full connection layer; the feature grouping fusion classification module comprises n x (n-1) / 2 first to nth splicing modules, a shared weight feature comprehensive full connection layer, an average pooling layer and a binary classification full connection layer, each two embedded full connection layers are combined and fused with one splicing module, all splicing modules are connected with the shared weight feature comprehensive full connection layer, the shared weight feature comprehensive full connection layer is connected with the average pooling layer, and the average pooling layer is connected with the binary classification full connection layer. The feature matrix output by each MFCC module is separately subjected to one Transformer module to fully extract its unique sequence and obtain n embedded feature vectors. The n embedded feature vectors are spliced two by two, the spliced embedded feature vectors can avoid mutual interference between features, and through the shared weight feature comprehensive full connection layer, the parameter amount can be reduced and the generalization is improved. The spliced embedded feature vectors overcome the problems of mixed information and strong interference, reduce the parameter amount and reduce the calculation amount.

[0021] A method for detecting a motor according to the multi-feature grouping Transformer neural network described above, comprising the following steps: step S1: obtaining a motor sound time domain signal, and obtaining a first MFCC feature matrix with a shape of (n_frame, n_mfcc1) to an n-th MFCC feature matrix with a shape of (n_frame, n_mfccn) from the motor sound time domain signal through a multi-feature extraction module; step S2: obtaining a first embedding feature vector e1 with a shape of (transformer_dim, 1) to an n-th embedding feature vector en through a Transformer feature extraction module from the first MFCC feature matrix with a shape of (n_frame, n_mfcc1) to the n-th MFCC feature matrix with a shape of (n_frame, n_mfccn); and step S3: obtaining a binary classification result through a feature grouping fusion classification module from the first embedding feature vector e1 with a shape of (transformer_dim, 1) to the n-th embedding feature vector en, the binary classification result being normal or abnormal. The splicing of the embedding feature vectors overcomes the problems of mixed information and strong interference, reduces the parameter quantity, and reduces the calculation amount. BRIEF DESCRIPTION OF DRAWINGS

[0022] Figure 1 is a principle block diagram of the present application. DETAILED DESCRIPTION

[0023] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments. The description of the at least one exemplary embodiment is actually only illustrative, but not as any limitation on the present application and its application or use. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without making any creative efforts fall within the scope of protection of the present application.

[0024] In the following description, many specific details are set forth in order to provide a thorough understanding of the present application. However, the present application can be practiced without the specific details, other than those described herein, and it is understood that the present application is not limited to the embodiments described herein and can be practiced with or without other apparatuses, systems, structures, methodologies, procedures, components, materials and so on. Therefore, the present application is not limited to the specific embodiments disclosed below, but includes all possible embodiments that are within the scope of the present application. Embodiment 1

[0025] As Figure 1As shown, the application discloses a multi-feature grouping Transformer neural network, comprising a multi-feature extraction module, a Transformer feature extraction module and a feature grouping fusion classification module connected in turn, the multi-feature extraction module comprises n first to nth MFCC modules, each MFCC module is an MFCC module, so that each MFCC module can output an MFCC feature matrix of a shape, the number of mel-frequency cepstral coefficients n_mfcc of the MFCC module is in the range of 13-128.

[0026] The value range of n is 2-6.

[0027] The Transformer feature extraction module comprises n first to nth Transformer modules and n first to nth embedded fully connected layers, each Transformer module is a Transformer module, so that each Transformer module can output a Transformer feature matrix of a shape, and there are n shapes of Transformer feature matrices.

[0028] Each embedded fully connected layer is an embedded fully connected layer, so that each embedded fully connected layer can output an embedded feature vector of a shape. The weight size of the first embedded fully connected layer is (n_mfcc1, transformer_dim), the weight size of the second embedded fully connected layer is (n_mfcc2, transformer_dim), and the weight size of the nth embedded fully connected layer is (n_mfccn, transformer_dim).

[0029] One MFCC module is connected with a corresponding Transformer module, one Transformer module is connected with a corresponding embedded fully connected layer, the first MFCC module is connected with the first Transformer module, the first Transformer module is connected with the first embedded fully connected layer, the second MFCC module is connected with the second Transformer module, the second Transformer module is connected with the second embedded fully connected layer, and the nth MFCC module is connected with the nth Transformer module, and the nth Transformer module is connected with the nth embedded fully connected layer.

[0030] The feature group fusion classification module comprises first to nth×(n-1) / 2 splicing modules, a shared weight feature comprehensive full connection layer, an average pooling layer and a binary classification full connection layer, and each two embedded full connection layers are combined with one splicing module to be fused and connected, that is, the first embedded full connection layer is connected with the first splicing module, the second embedded full connection layer is connected with the first splicing module, the first embedded full connection layer is connected with the second splicing module, the third embedded full connection layer is connected with the second splicing module, and so on until the first embedded full connection layer is connected with the n-1 splicing module, and the n embedded full connection layer is connected with the n-1 splicing module; the second embedded full connection layer is connected with the n splicing module, the third embedded full connection layer is connected with the n splicing module, the second embedded full connection layer is connected with the n+1 splicing module, the fourth embedded full connection layer is connected with the n+1 splicing module, and so on until the second embedded full connection layer is connected with the 2n-3 splicing module, and the n embedded full connection layer is connected with the 2n-3 splicing module; and so on until the n-1 embedded full connection layer is connected with the nth×(n-1) / 2 splicing module, and the n embedded full connection layer is connected with the nth×(n-1) / 2 splicing module.

[0031] All the splicing modules are connected with the shared weight feature comprehensive full connection layer, the shared weight feature comprehensive full connection layer is connected with the average pooling layer, and the average pooling layer is connected with the binary classification full connection layer.

[0032] The motor sound time domain signal is subjected to feature extraction by the first MFCC module to obtain a first MFCC feature matrix with a shape of (n_frame, n_mfcc1), the motor sound time domain signal is subjected to feature extraction by the second MFCC module to obtain a second MFCC feature matrix with a shape of (n_frame, n_mfcc2), and so on until the motor sound time domain signal is subjected to feature extraction by the n-th MFCC module to obtain an n-th MFCC feature matrix with a shape of (n_frame, n_mfccn).

[0033] The first MFCC feature matrix with a shape of (n_frame, n_mfcc1) is extracted by a first Transformer module to obtain a first Transformer feature matrix with a shape of (n_frame, n_mfcc1), and the first Transformer feature matrix with a shape of (n_frame, n_mfcc1) is output by a first embedded full connection layer to obtain a first embedded feature vector e1 with a shape of (transformer_dim, 1); the second MFCC feature matrix with a shape of (n_frame, n_mfcc2) is extracted by a second Transformer module to obtain a second Transformer feature matrix with a shape of (n_frame, n_mfcc2), and the second Transformer feature matrix with a shape of (n_frame, n_mfcc2) is output by a second embedded full connection layer to obtain a second embedded feature vector e2 with a shape of (transformer_dim, 1); until the nth MFCC feature matrix with a shape of (n_frame, n_mfccn) is extracted by the nth Transformer module to obtain the nth Transformer feature matrix with a shape of (n_frame, n_mfccn), and the nth Transformer feature matrix with a shape of (n_frame, n_mfccn) is output by the nth embedded full connection layer to obtain the nth embedded feature vector en with a shape of (transformer_dim, 1).

[0034] The first embedded feature vector e1 with a shape of (transformer_dim, 1) and the second embedded feature vector e2 with a shape of (transformer_dim, 1) are spliced by a first splicing module to obtain a spliced first splicing feature vector with a shape of (2*transformer_dim, 1), the first embedded feature vector e1 with a shape of (transformer_dim, 1) and the third embedded feature vector e3 with a shape of (transformer_dim, 1) are spliced by a second splicing module to obtain a spliced second splicing feature vector with a shape of (2*transformer_dim, 1), and until the first embedded feature vector e1 with a shape of (transformer_dim, 1) and the nth embedded feature vector en with a shape of (transformer_dim, 1) are spliced by an (n-1)th splicing module to obtain a spliced (n-1)th splicing feature vector with a shape of (2*transformer_dim, 1).

[0035] The second embedding feature vector e2 of shape (transformer_dim, 1) and the third embedding feature vector e3 of shape (transformer_dim, 1) are spliced by the nth splicing module to obtain a spliced nth splicing feature vector of shape (2 x transformer_dim, 1), the second embedding feature vector e2 of shape (transformer_dim, 1) and the fourth embedding feature vector e4 of shape (transformer_dim, 1) are spliced by the n+1th splicing module to obtain a spliced n+1th splicing feature vector of shape (2 x transformer_dim, 1), and so on, until the second embedding feature vector e2 of shape (transformer_dim, 1) and the nth embedding feature vector en of shape (transformer_dim, 1) are spliced by the 2n-3th splicing module to obtain a spliced 2n-3th splicing feature vector of shape (2 x transformer_dim, 1).

[0036] until the n-1th embedding feature vector en-1 of shape (transformer_dim, 1) and the nth embedding feature vector en of shape (transformer_dim, 1) are spliced by the n x (n-1) / 2th splicing module to obtain a spliced n x (n-1) / 2th splicing feature vector of shape (2 x transformer_dim, 1).

[0037] All spliced splicing feature vectors obtain n x (n-1) / 2 comprehensive feature vectors of shape (transformer_dim, 1) through the feature comprehensive fully connected layer with shared weights, that is, the first splicing feature vector of shape (2 x transformer_dim, 1) obtains a first comprehensive feature vector of shape (transformer_dim, 1) through the feature comprehensive fully connected layer with shared weights, the second splicing feature vector of shape (2 x transformer_dim, 1) obtains a second comprehensive feature vector of shape (transformer_dim, 1) through the feature comprehensive fully connected layer with shared weights, and so on, until the n-1th splicing feature vector of shape (2 x transformer_dim, 1) obtains an n-1th comprehensive feature vector of shape (transformer_dim, 1) through the feature comprehensive fully connected layer with shared weights.

[0038] The n-th spliced feature vector with a shape of (2*transformer_dim, 1) obtains an n-th integrated feature vector with a shape of (transformer_dim, 1) through the feature integrated fully connected layer with shared weights, an (n+1)-th spliced feature vector with a shape of (2*transformer_dim, 1) obtains an (n+1)-th integrated feature vector with a shape of (transformer_dim, 1) through the feature integrated fully connected layer with shared weights, and so on until a 2n-3-th spliced feature vector with a shape of (2*transformer_dim, 1) obtains a 2n-3-th integrated feature vector with a shape of (transformer_dim, 1) through the feature integrated fully connected layer with shared weights.

[0039] until an n*x(n-1) / 2-th spliced feature vector with a shape of (2*transformer_dim, 1) obtains an n*x(n-1) / 2-th integrated feature vector with a shape of (transformer_dim, 1) through the feature integrated fully connected layer with shared weights.

[0040] All integrated feature vectors obtain a feature vector with a shape of (transformer_dim, 1) through an average pooling layer, and the feature vector with a shape of (transformer_dim, 1) obtains a binary classification result of motor normal or abnormal through a fully connected layer of binary classification. Embodiment 2

[0041] The application discloses a method for detecting a motor, a multi-feature grouping Transformer neural network based on embodiment 1, including steps S1: multi-feature extraction, S2: deep feature extraction based on Transformer, and S3: feature grouping fusion classification to obtain a binary classification result.

[0042] Step S1: multi-feature extraction.

[0043] The time-domain sound signals collected during the operation of the motor are respectively processed through the MFCC method with different numbers of mel frequency cepstral coefficients (n_mfcc) to extract n different MFCC feature matrices with shapes of (n_frame, n_mfcc1), (n_frame, n_mfcc2),..., (n_frame, n_mfccn).

[0044] Step S2: deep feature extraction based on Transformer.

[0045] Different kinds of MFCC feature matrices respectively represent the feature information of the audio signal in different frequency resolutions and time perception levels. If all MFCC feature matrices are directly spliced and input into the same Transformer module, it may cause mutual interference between information and reduce the discrimination performance of the model. Therefore, the method uses an independent feature extraction structure, that is, for each kind of MFCC feature matrix, a Transformer module is introduced for feature coding. The module includes m multi-head attention mechanisms and K encoder layers, which can extract the time sequence representation ability of each group of features.

[0046] Since the Transformer module is a sequence-to-sequence structure, its output maintains the same two-dimensional shape as the input. Therefore, after the n kinds of MFCC feature matrices pass through their respective Transformer modules, they output n kinds of Transformer feature matrices with shapes of (n_frame, n_mfcc1), (n_frame, n_mfcc2), …, (n_frame, n_mfccn).

[0047] In order to unify the feature dimension and facilitate subsequent feature grouping and fusion processing, each Transformer feature matrix is further processed through an embedded fully connected layer. The weight size of the embedded fully connected layer is (n_mfcc1, transformer_dim), (n_mfcc2, transformer_dim), …, (n_mfccn, transformer_dim) respectively. Finally, each feature matrix is compressed into an embedded feature vector with a shape of (transformer_dim, 1), denoted as e1, e2, …, en.

[0048] Step S3: Feature grouping and fusion classification to obtain a binary classification result.

[0049] The n embedded feature vectors obtained in the foregoing are denoted as e1, e2, …, en. Each two embedded feature vectors are combined and spliced, for example, e1 and e2, e1 and e3, e1 and e4, …, e1 and en, e2 and e3, e2 and e4, e2 and e5, …, e2 and en, …, en-1 and en. A total of n×(n-1) / 2 spliced feature vectors can be obtained. This splicing method is similar to the idea of grouped convolution, which aims to enhance the information expression ability while reducing the mutual interference between different features. The shape of these spliced feature vectors is (2×transformer_dim, 1).

[0050] Next, these concatenated feature vectors are fed into a weight-shared feature integration fully-connected layer. This fully-connected layer, used for feature integration, has an input dimension of 2 times transformer_dim and an output dimension of transformer_dim. Because this feature integration fully-connected layer shares weights across all concatenated vectors, it effectively reduces the number of parameters and improves the model's generalization. This process yields n × (n-1) / 2 integrated feature vectors with a shape of (transformer_dim, 1).

[0051] Next, these comprehensive feature vectors are averaged and averaged over the dimensions to obtain a unified feature vector with a shape of transformer_dim rows and 1 column.

[0052] Finally, this final unified feature vector is input into a binary classification fully connected layer to output the final classification result.

[0053] The technical concept of this application is summarized as follows.

[0054] In this application, n types of MFCCs, that is, MFCCs with different numbers of Mel-frequency cepstral coefficients (n_mfcc), are used to extract features from the original audio to obtain n types of MFCC feature matrices. Different types of MFCC feature matrices represent different frequency resolutions and time perception capabilities of audio signals. Direct splicing and input into the Transformer may cause information mixing and interference enhancement. Therefore, this application adopts an independent feature extraction method, that is, each MFCC feature matrix passes through a Transformer module alone to fully extract its unique sequence representation capability and obtain n types of embedded feature vectors.

[0055] Secondly, the n embedded feature vectors e1, e2, …, en are spliced ​​in pairs, such as e1 and e2, e1 and e3, etc., for a total of n*(n-1) / 2 spliced ​​embedded feature vectors. Similar to the idea of ​​grouped convolution, splicing embedded feature vectors can avoid mutual interference between features and pass through a shared fully connected layer. Since the fully connected layer has many parameters, this shared fully connected layer can not only reduce the number of parameters but also improve generalization.

[0056] Finally, average pooling is performed, and the two-class fully connected layer obtains the classification result.

[0057] Data Example: In order to make the content, purpose and advantages of this application clearer and more detailed, this application is described in more detail in conjunction with the following data examples.

[0058] The present application uses the public dataset MIMII Dataset to verify the superiority of the Transformer neural network based on multi-feature grouping for industrial motor equipment anomaly detection method. MIMII Dataset is a reliable dataset for fault industrial machine investigation and inspection. It contains the sound produced by four industrial machines, i.e. valve, pump, fan and slide rail. Four models (id0, id2, id4, id6) are disclosed for each valve, pump, fan and slide rail, respectively, and each model discloses -6db, 0db, 6db signal-to-noise ratio sound data, and the data of each model contains normal sound and abnormal sound.

[0059] The present application selects the -6db signal-to-noise ratio fan motor sound of id2 as experimental data, and cuts the training set and test set in the ratio of 8:2.

[0060] In feature extraction: n_frame=5, n=3, n_mfcc1=32, n_mfcc2=64, n_mfcc3=128.

[0061] The Transformer module encodes the features. The module has m=4 multi-head attention mechanisms and K=2 encoder layers.

[0062] The embedded fully connected layer has transformer_dim=64.

[0063] The experimental results are shown as follows.

[0064] Compare n=1, i.e. only use one MFCC model, n=2, use 2 MFCC models, n=3 use 3 MFCC models.

[0065] When n=1, n_mfcc=32, the accuracy is 97.5%; when n=1, n_mfcc=64, the accuracy is 99.2%; when n=1, n_mfcc=128, the accuracy is 99.2%.

[0066] When n=2, integrate n_mfcc1=32 and n_mfcc2=64, the accuracy is 99.3%; when n=2, integrate n_mfcc1=32 and n_mfcc2=128, the accuracy is 99.3%; when n=2, integrate n_mfcc1=64 and n_mfcc2=128, the accuracy is 99.4%.

[0067] When n=3, integrate n_mfcc1=32, n_mfcc2=64 and n_mfcc3=128, the accuracy is 99.3%.

[0068] Therefore, we can conclude that Transformer neural networks based on multi-feature grouping are more effective than single-feature methods for detecting anomalies in industrial motor equipment. However, low accuracy of individual MFCCs can also affect the multi-MFCC grouping method. Here, a single n_mfcc=32 achieves the lowest accuracy of 97.5%. Therefore, instead of using n_mfcc=32, a model combining n_mfcc1=64 and n_mfcc2=128 achieves an accuracy of 99.4%. This model achieves the highest accuracy.

Claims

1. A Transformer neural network for multi-feature grouping, characterized by: It includes a multi-feature extraction module, a Transformer feature extraction module and a feature grouping fusion classification module connected in sequence, wherein the multi-feature extraction module includes n types of first to nth MFCC modules, the Transformer feature extraction module includes n types of first to nth Transformer modules and n types of first to nth embedded fully connected layers, one MFCC module is connected to a corresponding Transformer module, and one Transformer module is connected to a corresponding embedded fully connected layer; the feature grouping fusion classification module includes the first to n×(n-1) / 2 splicing modules, a feature comprehensive fully connected layer with shared weights, an average pooling layer and a binary fully connected layer, and every two embedded fully connected layers are fused and connected with a splicing module, all splicing modules are connected to the feature comprehensive fully connected layer with shared weights, the feature comprehensive fully connected layer with shared weights is connected to the average pooling layer, and the average pooling layer is connected to the binary fully connected layer.

2. The multi-feature grouping Transformer neural network according to claim 1, characterized in that: The number of Mel-frequency cepstral coefficients n_mfcc of the MFCC module ranges from 13 to 128.

3. The multi-feature grouping Transformer neural network according to claim 1, characterized in that: The value of n ranges from 2 to 6.

4. The multi-feature grouping Transformer neural network according to claim 1, characterized in that: The one MFCC module is connected to the corresponding one Transformer module, and the one Transformer module is connected to the corresponding one embedded fully connected layer, that is, the first MFCC module is connected to the first Transformer module, the first Transformer module is connected to the first embedded fully connected layer, the second MFCC module is connected to the second Transformer module, the second Transformer module is connected to the second embedded fully connected layer, until the nth MFCC module is connected to the nth Transformer module, and the nth Transformer module is connected to the nth embedded fully connected layer.

5. The multi-feature grouping Transformer neural network according to claim 1, characterized in that: The method comprises the following steps: every two embedded fully-connected layers are combined and fused with one splicing module, that is, the first embedded fully-connected layer is connected to the first splicing module, the second embedded fully-connected layer is connected to the first splicing module, the first embedded fully-connected layer is connected to the second splicing module, and the third embedded fully-connected layer is connected to the second splicing module, until the first embedded fully-connected layer is connected to the n-1th splicing module and the n-1th embedded fully-connected layer is connected to the n-1th splicing module; the second embedded fully-connected layer is connected to the n-1th splicing module, the third embedded fully-connected layer is connected to the n-1th splicing module, the second embedded fully-connected layer is connected to the n+1th splicing module, and the fourth embedded fully-connected layer is connected to the n+1th splicing module, until the second embedded fully-connected layer is connected to the 2n-3 splicing module and the n-1th embedded fully-connected layer is connected to the 2n-3 splicing module; until the n-1th embedded fully-connected layer is connected to the n×(n-1) / 2th splicing module and the n-1th embedded fully-connected layer is connected to the n×(n-1) / 2th splicing module.

6. A method for detecting a motor, comprising: The following steps are included: Step S1: Obtain the motor sound time domain signal, and extract the motor sound time domain signal through the multi-feature extraction module to obtain the first MFCC feature matrix with a shape of (n_frame, n_mfcc1) to the nth MFCC feature matrix with a shape of (n_frame, n_mfccn); Step S2: The first MFCC feature matrix with a shape of (n_frame, n_mfcc1) to the nth MFCC feature matrix with a shape of (n_frame, n_mfccn) are subjected to the Transformer feature extraction module to obtain the first embedded feature vector e1 to the nth embedded feature vector en with a shape of (transformer_dim, 1); Step S3: The first embedded feature vector e1 to the nth embedded feature vector en of shape (transformer_dim, 1) are subjected to a feature grouping and fusion classification module to obtain a binary classification result, which is normal or abnormal.

7. The method for detecting a motor according to claim 6, wherein: In step S1, the time domain signal of the motor sound is subjected to feature extraction by the first MFCC module to obtain a first MFCC feature matrix with a shape of (n_frame, n_mfcc1), the time domain signal of the motor sound is subjected to feature extraction by the second MFCC module to obtain a second MFCC feature matrix with a shape of (n_frame, n_mfcc2), until the time domain signal of the motor sound is subjected to feature extraction by the nth MFCC module to obtain an nth MFCC feature matrix with a shape of (n_frame, n_mfccn).

8. The method for detecting a motor according to claim 6, wherein: In step S2, the first MFCC feature matrix with a shape of (n_frame, n_mfcc1) is subjected to feature extraction by the first Transformer module to obtain a first Transformer feature matrix with a shape of (n_frame, n_mfcc1), and the first Transformer feature matrix with a shape of (n_frame, n_mfcc1) is subjected to the first embedded fully connected layer to output a first embedded feature vector e1 with a shape of (transformer_dim, 1); the second MFCC feature matrix with a shape of (n_frame, n_mfcc2) is subjected to feature extraction by the second Transformer module to obtain a second Transformer feature matrix with a shape of (n_frame, n_mfcc2), and the second Transformer feature matrix with a shape of (n_frame, n_mfcc2) is subjected to the second embedded fully connected layer to output a second embedded feature vector e2 with a shape of (transformer_dim, 1); until the nth MFCC feature matrix with a shape of (n_frame, n_mfccn) is subjected to feature extraction by the nth Transformer module to obtain a second Transformer feature matrix with a shape of (n_frame, The nth Transformer feature matrix with a shape of (n_frame, n_mfccn) is output through the nth embedded fully connected layer as the nth embedded feature vector en with a shape of (transformer_dim, 1).

9. The method for detecting a motor according to claim 6, wherein: In step S3, the first embedded feature vector e1 with a shape of (transformer_dim, 1) and the second embedded feature vector e2 with a shape of (transformer_dim, 1) are spliced ​​by the first splicing module to obtain a spliced ​​first spliced ​​feature vector with a shape of (2×transformer_dim, 1), the first embedded feature vector e1 with a shape of (transformer_dim, 1) and the third embedded feature vector e3 with a shape of (transformer_dim, 1) are spliced ​​by the second splicing module to obtain a spliced ​​second spliced ​​feature vector with a shape of (2×transformer_dim, 1), until the first embedded feature vector e1 with a shape of (transformer_dim, 1) and the nth embedded feature vector en with a shape of (transformer_dim, 1) are spliced ​​by the n-1th splicing module to obtain a spliced ​​n-1th spliced ​​feature vector with a shape of (2×transformer_dim, 1); The second embedded feature vector e2 with a shape of (transformer_dim, 1) and the third embedded feature vector e3 with a shape of (transformer_dim, 1) are spliced ​​by the nth splicing module to obtain the nth spliced ​​feature vector with a shape of (2×transformer_dim, 1). The second embedded feature vector e2 with a shape of (transformer_dim, 1) and the fourth embedded feature vector e4 with a shape of (transformer_dim, 1) are spliced ​​by the n+1th splicing module to obtain the n+1th spliced ​​feature vector with a shape of (2×transformer_dim, 1), until the second embedded feature vector e2 with a shape of (transformer_dim, 1) and the nth embedded feature vector en with a shape of (transformer_dim, 1) are spliced ​​by the 2n-3th splicing module to obtain the 2n-3th spliced ​​feature vector with a shape of (2×transformer_dim, 1). Until the n-1th embedded feature vector en-1 with a shape of (transformer_dim, 1) and the nth embedded feature vector en with a shape of (transformer_dim, 1) are spliced ​​by the n×(n-1) / 2 splicing module to obtain the spliced ​​n×(n-1) / 2th spliced ​​feature vector with a shape of (2×transformer_dim, 1).

10. The method for detecting a motor according to claim 9, characterized in that: In step S3, all the spliced ​​feature vectors are passed through a weighted feature synthesis fully connected layer to obtain n× (n-1) / 2 integrated feature vectors with a shape of (transformer_dim, 1). All the integrated feature vectors are passed through an average pooling layer to obtain a feature vector with a shape of (transformer_dim, 1). The feature vector with a shape of (transformer_dim, 1) is passed through a binary classification fully connected layer to obtain a binary classification result.

Images