Multi-feature grouping transformer neural network and method

Abstract

The application discloses a multi-feature grouping Transformer neural network and method, and relates to the technical field of motor testing; the neural network comprises a multi-feature extraction module, a Transformer feature extraction module and a feature grouping fusion classification module which are connected in sequence; the multi-feature extraction module comprises n MFCC modules; the Transformer feature extraction module comprises n Transformer modules and an embedded full connection layer; the feature grouping fusion classification module comprises a splicing module, a feature comprehensive full connection layer, an average pooling layer and a binary classification full connection layer; the method comprises the following steps: S1, multi-feature extraction; S2, Transformer deep feature extraction; and S3, feature grouping fusion classification to obtain a binary classification result; the spliced embedded feature vector overcomes the problems of mixed information and strong interference, reduces the parameter quantity and lowers the calculation amount.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of motor testing, and in particular to a multi-feature grouping Transformer neural network and method. BACKGROUND

[0002] The steps of the machine identifying motor abnormal equipment are feature extraction and model prediction. In the feature extraction stage, the common method is to convert the time domain data of the motor into a mel frequency cepstral coefficient (MFCC) feature matrix. Different numbers of mel coefficients have advantages and disadvantages: a MFCC with a smaller number of mel frequency cepstral coefficients can remove redundant information, reduce computational cost, and improve efficiency; a MFCC with a larger number of mel frequency cepstral coefficients can capture more rich speech spectrum features and improve classification effect. Therefore, in the feature extraction stage, using only one MFCC dimension information may reduce the accuracy of motor sound classification.

[0003] In the model prediction stage, the commonly used model is the Transformer model. Compared with RNN and CNN, the Transformer model has significant advantages in processing sequence data such as sound and text.

[0004] In the present application, n kinds of MFCCs, i.e., MFCCs with different numbers of mel frequency cepstral coefficients (n_mfcc), are used to extract features from the original audio, obtaining n kinds of MFCC feature matrices. Different kinds of MFCC feature matrices represent different frequency resolutions and time perception abilities of the audio signal, and are directly concatenated into the Transformer, resulting in mixed information and enhanced interference.

[0005] The n kinds of embedding feature vectors e1, e2, ……, en are obtained through a fully connected layer to obtain a comprehensive feature vector.

[0006] Suppose the dimension of each embedding vector is (10, 1), then the overall input dimension is (10 x n, 1). For example, e1, e2 to e10, a total of ten embeddings, are all input into the fully connected layer. The shape of each embedding feature vector is (10, 1), and the overall input dimension is (100, 1). The shape of the fully connected layer is (100, out), and the shape of the 1 feature vector obtained by the fully connected layer output is (out, 1). Since the shape of the fully connected layer is (100, out), the number of parameters is too large, resulting in a large amount of calculation and low computational efficiency.

[0007] The value of n is reduced to 3, and the total of three embedding e1, e2 and e3 are input into the full connection layer, the shape of each embedding feature vector is (10, 1), the input dimension 10*3 is too large, the shape of the full connection layer is (30, out), the shape of the obtained 1 feature vector output by the full connection layer is (out, 1), since the shape of the full connection layer is (30, out), the number of parameters is too large, and the calculation amount is still large, and the calculation efficiency is low.

[0008] Therefore, the information is mixed, the interference is strong, and the calculation amount is large, which becomes a technical problem to be solved. SUMMARY

[0009] The application provides a multi-feature grouping Transformer neural network and method, which solves the technical problems of mixed information, strong interference and large calculation amount.

[0010] To solve the above technical problems, the technical solution adopted by the application is as follows:

[0011] A multi-feature grouping Transformer neural network comprises a multi-feature extraction module, a Transformer feature extraction module and a feature grouping fusion classification module connected in sequence, the multi-feature extraction module comprises n first to nth MFCC modules, the Transformer feature extraction module comprises n first to nth Transformer modules and n first to nth embedding full connection layers, one MFCC module is connected with a corresponding Transformer module, and one Transformer module is connected with a corresponding embedding full connection layer; the feature grouping fusion classification module comprises first to n*(n-1) / 2 splicing modules, a shared weight feature comprehensive full connection layer, an average pooling layer and a binary classification full connection layer, each two embedding full connection layers are combined and fused with one splicing module, all splicing modules are connected with the shared weight feature comprehensive full connection layer, the shared weight feature comprehensive full connection layer is connected with the average pooling layer, and the average pooling layer is connected with the binary classification full connection layer.

[0012] Further technical solutions are that the number of mel frequency cepstral coefficients of the MFCC module n_mfcc is in the range of 13-128.

[0013] Further technical solutions are that the value of n is in the range of 2-6.

[0014] Further, the technical scheme is that one MFCC module is connected with a corresponding Transformer module, one Transformer module is connected with a corresponding embedded full connection layer, that is, the first MFCC module is connected with the first Transformer module, the first Transformer module is connected with the first embedded full connection layer, the second MFCC module is connected with the second Transformer module, the second Transformer module is connected with the second embedded full connection layer, and the nth MFCC module is connected with the nth Transformer module, and the nth Transformer module is connected with the nth embedded full connection layer.

[0015] Further, the technical scheme is that one MFCC module is connected with a corresponding Transformer module, one Transformer module is connected with a corresponding embedded full connection layer, that is, the first MFCC module is connected with the first Transformer module, the first Transformer module is connected with the first embedded full connection layer, the second MFCC module is connected with the second Transformer module, the second Transformer module is connected with the second embedded full connection layer, and the nth MFCC module is connected with the nth Transformer module, and the nth Transformer module is connected with the nth embedded full connection layer.

[0016] A method for detecting a motor according to the multi-feature grouping Transformer neural network described above, comprising the following steps,

[0017] Step S1: obtaining a motor sound time domain signal, and obtaining a first MFCC feature matrix with a shape of (n_frame, n_mfcc1) to an nth MFCC feature matrix with a shape of (n_frame, n_mfccn) from the motor sound time domain signal through a multi-feature extraction module;

[0018] Step S2: obtaining a first embedded feature vector e1 to an nth embedded feature vector en with a shape of (transformer_dim, 1) from the first MFCC feature matrix with a shape of (n_frame, n_mfcc1) to the nth MFCC feature matrix with a shape of (n_frame, n_mfccn) through a Transformer feature extraction module;

[0019] Step S3: the first embedding feature vector e1 to the n-th embedding feature vector en in the shape of (transformer_dim, 1) are fused by the feature grouping fusion classification module to obtain a binary classification result, which is normal or abnormal.

[0020] Further technical solutions are as follows: in the step S1, the motor sound time domain signal is extracted by the first MFCC module to obtain a first MFCC feature matrix in the shape of (n_frame, n_mfcc1), the motor sound time domain signal is extracted by the second MFCC module to obtain a second MFCC feature matrix in the shape of (n_frame, n_mfcc2), and the motor sound time domain signal is extracted by the n-th MFCC module to obtain an n-th MFCC feature matrix in the shape of (n_frame, n_mfccn).

[0021] Further technical solutions are as follows: in the step S2, the first MFCC feature matrix in the shape of (n_frame, n_mfcc1) is extracted by the first Transformer module to obtain a first Transformer feature matrix in the shape of (n_frame, n_mfcc1), and the first Transformer feature matrix in the shape of (n_frame, n_mfcc1) is output by the first embedded full connection layer to obtain a first embedding feature vector e1 in the shape of (transformer_dim, 1); the second MFCC feature matrix in the shape of (n_frame, n_mfcc2) is extracted by the second Transformer module to obtain a second Transformer feature matrix in the shape of (n_frame, n_mfcc2), and the second Transformer feature matrix in the shape of (n_frame, n_mfcc2) is output by the second embedded full connection layer to obtain a second embedding feature vector e2 in the shape of (transformer_dim, 1); and the n-th MFCC feature matrix in the shape of (n_frame, n_mfccn) is extracted by the n-th Transformer module to obtain an n-th Transformer feature matrix in the shape of (n_frame, n_mfccn), and the n-th Transformer feature matrix in the shape of (n_frame, n_mfccn) is output by the n-th embedded full connection layer to obtain an n-th embedding feature vector en in the shape of (transformer_dim, 1).

[0022] Further, in the step S3, the first embedding feature vector e1 of shape (transformer_dim, 1) and the second embedding feature vector e2 of shape (transformer_dim, 1) are spliced by the first splicing module to obtain a first spliced feature vector of shape (2 x transformer_dim, 1) after splicing, the first embedding feature vector e1 of shape (transformer_dim, 1) and the third embedding feature vector e3 of shape (transformer_dim, 1) are spliced by the second splicing module to obtain a second spliced feature vector of shape (2 x transformer_dim, 1) after splicing, and so on until the first embedding feature vector e1 of shape (transformer_dim, 1) and the n-th embedding feature vector en of shape (transformer_dim, 1) are spliced by the (n-1)-th splicing module to obtain an (n-1)-th spliced feature vector of shape (2 x transformer_dim, 1) after splicing;

[0023] The second embedding feature vector e2 of shape (transformer_dim, 1) and the third embedding feature vector e3 of shape (transformer_dim, 1) are spliced by the n-th splicing module to obtain an n-th spliced feature vector of shape (2 x transformer_dim, 1) after splicing, the second embedding feature vector e2 of shape (transformer_dim, 1) and the fourth embedding feature vector e4 of shape (transformer_dim, 1) are spliced by the (n+1)-th splicing module to obtain an (n+1)-th spliced feature vector of shape (2 x transformer_dim, 1) after splicing, and so on until the second embedding feature vector e2 of shape (transformer_dim, 1) and the n-th embedding feature vector en of shape (transformer_dim, 1) are spliced by the 2n-3-th splicing module to obtain a 2n-3-th spliced feature vector of shape (2 x transformer_dim, 1) after splicing;

[0024] until the (n-1)-th embedding feature vector en-1 of shape (transformer_dim, 1) and the n-th embedding feature vector en of shape (transformer_dim, 1) are spliced by the n x (n-1) / 2-th splicing module to obtain an n x (n-1) / 2-th spliced feature vector of shape (2 x transformer_dim, 1) after splicing.

[0025] Further, in step S3, all spliced feature vectors are obtained n x (n-1) / 2 comprehensive feature vectors with a shape of (transformer_dim, 1) through a shared weight feature comprehensive full connection layer, all comprehensive feature vectors are obtained a feature vector with a shape of (transformer_dim, 1) through an average pooling layer, and the feature vector with a shape of (transformer_dim, 1) is obtained a binary classification result through a binary classification full connection layer.

[0026] The above technical scheme has the following beneficial effects:

[0027] A multi-feature grouping Transformer neural network comprises a multi-feature extraction module, a Transformer feature extraction module and a feature grouping fusion classification module connected in sequence, the multi-feature extraction module comprises n first to nth MFCC modules, the Transformer feature extraction module comprises n first to nth Transformer modules and n first to nth embedded full connection layers, one MFCC module is connected with a corresponding Transformer module, and one Transformer module is connected with a corresponding embedded full connection layer; the feature grouping fusion classification module comprises n x (n-1) / 2 first to nth splicing modules, a shared weight feature comprehensive full connection layer, an average pooling layer and a binary classification full connection layer, each two embedded full connection layers are combined to be fused with one splicing module, all splicing modules are connected with the shared weight feature comprehensive full connection layer, the shared weight feature comprehensive full connection layer is connected with the average pooling layer, and the average pooling layer is connected with the binary classification full connection layer. The feature matrix output by each MFCC module is separately subjected to one Transformer module to sufficiently extract the unique sequence thereof, and n embedded feature vectors are obtained. The n embedded feature vectors are spliced two by two, the spliced embedded feature vectors can avoid mutual interference between features, and through the shared weight feature comprehensive full connection layer, the parameter amount can be reduced, and the generalization is improved. The spliced embedded feature vectors overcome the problems of mixed information and strong interference, reduce the parameter amount, and reduce the calculation amount.

[0028] A method for detecting a motor according to the multi-feature grouping Transformer neural network described above, comprising the following steps: step S1: obtaining a motor sound time domain signal, and obtaining a first MFCC feature matrix with a shape of (n_frame, n_mfcc1) to an n-th MFCC feature matrix with a shape of (n_frame, n_mfccn) from the motor sound time domain signal through a multi-feature extraction module; step S2: obtaining a first embedding feature vector e1 with a shape of (transformer_dim, 1) to an n-th embedding feature vector en through a Transformer feature extraction module from the first MFCC feature matrix with a shape of (n_frame, n_mfcc1) to the n-th MFCC feature matrix with a shape of (n_frame, n_mfccn); and step S3: obtaining a binary classification result through a feature grouping fusion classification module from the first embedding feature vector e1 with a shape of (transformer_dim, 1) to the n-th embedding feature vector en, the binary classification result being normal or abnormal. The splicing of the embedding feature vectors overcomes the problems of mixed information and strong interference, reduces the parameter quantity, and reduces the calculation amount. BRIEF DESCRIPTION OF DRAWINGS

[0029] Figure 1 is a principle block diagram of the present application. DETAILED DESCRIPTION

[0030] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application. The following description of at least one exemplary embodiment is actually only illustrative, but not as any limitation on the present application and its application or use. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without making creative efforts fall within the scope of protection of the present application.

[0031] In the following description, many specific details are set forth in order to provide a thorough understanding of the present application, but the present application can also be implemented in other ways different from those described herein, and those skilled in the art can make similar generalizations without departing from the connotation of the present application, therefore the present application is not limited to the specific embodiments disclosed below. Embodiment 1

[0032] As Figure 1As shown, the application discloses a multi-feature grouping Transformer neural network, comprising a multi-feature extraction module, a Transformer feature extraction module and a feature grouping fusion classification module connected in turn, the multi-feature extraction module comprises n first to nth MFCC modules, each MFCC module is an MFCC module, so that each MFCC module can output an MFCC feature matrix of a shape, the number of mel-frequency cepstral coefficients n_mfcc of the MFCC module is in the range of 13-128.

[0033] The value range of n is 2-6.

[0034] The Transformer feature extraction module comprises n first to nth Transformer modules and n first to nth embedded fully connected layers, each Transformer module is a Transformer module, so that each Transformer module can output a Transformer feature matrix of a shape, and there are n shapes of Transformer feature matrices.

[0035] Each embedded fully connected layer is an embedded fully connected layer, so that each embedded fully connected layer can output an embedded feature vector of a shape. The weight size of the first embedded fully connected layer is (n_mfcc1, transformer_dim), the weight size of the second embedded fully connected layer is (n_mfcc2, transformer_dim), and the weight size of the nth embedded fully connected layer is (n_mfccn, transformer_dim).

[0036] One MFCC module is connected with a corresponding Transformer module, one Transformer module is connected with a corresponding embedded fully connected layer, the first MFCC module is connected with the first Transformer module, the first Transformer module is connected with the first embedded fully connected layer, the second MFCC module is connected with the second Transformer module, the second Transformer module is connected with the second embedded fully connected layer, and the nth MFCC module is connected with the nth Transformer module, and the nth Transformer module is connected with the nth embedded fully connected layer.

[0037] The feature group fusion classification module comprises first to nth×(n-1) / 2 splicing modules, a shared weight feature comprehensive full connection layer, an average pooling layer and a binary classification full connection layer, and each two embedded full connection layers are combined with one splicing module to be fused and connected, that is, the first embedded full connection layer is connected with the first splicing module, the second embedded full connection layer is connected with the first splicing module, the first embedded full connection layer is connected with the second splicing module, the third embedded full connection layer is connected with the second splicing module, and so on until the first embedded full connection layer is connected with the n-1 splicing module, and the n embedded full connection layer is connected with the n-1 splicing module; the second embedded full connection layer is connected with the n splicing module, the third embedded full connection layer is connected with the n splicing module, the second embedded full connection layer is connected with the n+1 splicing module, the fourth embedded full connection layer is connected with the n+1 splicing module, and so on until the second embedded full connection layer is connected with the 2n-3 splicing module, and the n embedded full connection layer is connected with the 2n-3 splicing module; and so on until the n-1 embedded full connection layer is connected with the nth×(n-1) / 2 splicing module, and the n embedded full connection layer is connected with the nth×(n-1) / 2 splicing module.

[0038] All the splicing modules are connected with the shared weight feature comprehensive full connection layer, the shared weight feature comprehensive full connection layer is connected with the average pooling layer, and the average pooling layer is connected with the binary classification full connection layer.

[0039] The motor sound time domain signal is subjected to feature extraction by the first MFCC module to obtain a first MFCC feature matrix with a shape of (n_frame, n_mfcc1), the motor sound time domain signal is subjected to feature extraction by the second MFCC module to obtain a second MFCC feature matrix with a shape of (n_frame, n_mfcc2), and so on until the motor sound time domain signal is subjected to feature extraction by the n-th MFCC module to obtain an n-th MFCC feature matrix with a shape of (n_frame, n_mfccn).

[0040] The first MFCC feature matrix with a shape of (n_frame, n_mfcc1) is extracted by a first Transformer module to obtain a first Transformer feature matrix with a shape of (n_frame, n_mfcc1), and the first Transformer feature matrix with a shape of (n_frame, n_mfcc1) is output by a first embedded full connection layer to obtain a first embedded feature vector e1 with a shape of (transformer_dim, 1); the second MFCC feature matrix with a shape of (n_frame, n_mfcc2) is extracted by a second Transformer module to obtain a second Transformer feature matrix with a shape of (n_frame, n_mfcc2), and the second Transformer feature matrix with a shape of (n_frame, n_mfcc2) is output by a second embedded full connection layer to obtain a second embedded feature vector e2 with a shape of (transformer_dim, 1); until the nth MFCC feature matrix with a shape of (n_frame, n_mfccn) is extracted by the nth Transformer module to obtain the nth Transformer feature matrix with a shape of (n_frame, n_mfccn), and the nth Transformer feature matrix with a shape of (n_frame, n_mfccn) is output by the nth embedded full connection layer to obtain the nth embedded feature vector en with a shape of (transformer_dim, 1).

[0041] The first embedded feature vector e1 with a shape of (transformer_dim, 1) and the second embedded feature vector e2 with a shape of (transformer_dim, 1) are spliced by a first splicing module to obtain a spliced first splicing feature vector with a shape of (2*transformer_dim, 1), the first embedded feature vector e1 with a shape of (transformer_dim, 1) and the third embedded feature vector e3 with a shape of (transformer_dim, 1) are spliced by a second splicing module to obtain a spliced second splicing feature vector with a shape of (2*transformer_dim, 1), and until the first embedded feature vector e1 with a shape of (transformer_dim, 1) and the nth embedded feature vector en with a shape of (transformer_dim, 1) are spliced by an (n-1)th splicing module to obtain a spliced (n-1)th splicing feature vector with a shape of (2*transformer_dim, 1).

[0042] The second embedding feature vector e2 of shape (transformer_dim, 1) and the third embedding feature vector e3 of shape (transformer_dim, 1) are spliced by the nth splicing module to obtain a spliced nth splicing feature vector of shape (2 x transformer_dim, 1), the second embedding feature vector e2 of shape (transformer_dim, 1) and the fourth embedding feature vector e4 of shape (transformer_dim, 1) are spliced by the n+1th splicing module to obtain a spliced n+1th splicing feature vector of shape (2 x transformer_dim, 1), and so on, until the second embedding feature vector e2 of shape (transformer_dim, 1) and the nth embedding feature vector en of shape (transformer_dim, 1) are spliced by the 2n-3th splicing module to obtain a spliced 2n-3th splicing feature vector of shape (2 x transformer_dim, 1).

[0043] until the n-1th embedding feature vector en-1 of shape (transformer_dim, 1) and the nth embedding feature vector en of shape (transformer_dim, 1) are spliced by the n x (n-1) / 2th splicing module to obtain a spliced n x (n-1) / 2th splicing feature vector of shape (2 x transformer_dim, 1).

[0044] All spliced splicing feature vectors obtain n x (n-1) / 2 comprehensive feature vectors of shape (transformer_dim, 1) through the feature comprehensive fully connected layer with shared weights, that is, the first splicing feature vector of shape (2 x transformer_dim, 1) obtains a first comprehensive feature vector of shape (transformer_dim, 1) through the feature comprehensive fully connected layer with shared weights, the second splicing feature vector of shape (2 x transformer_dim, 1) obtains a second comprehensive feature vector of shape (transformer_dim, 1) through the feature comprehensive fully connected layer with shared weights, and so on, until the n-1th splicing feature vector of shape (2 x transformer_dim, 1) obtains an n-1th comprehensive feature vector of shape (transformer_dim, 1) through the feature comprehensive fully connected layer with shared weights.

[0045] The n-th spliced feature vector with a shape of (2*transformer_dim, 1) obtains an n-th integrated feature vector with a shape of (transformer_dim, 1) through a feature integrated fully connected layer with shared weights, an (n+1)-th spliced feature vector with a shape of (2*transformer_dim, 1) obtains an (n+1)-th integrated feature vector with a shape of (transformer_dim, 1) through a feature integrated fully connected layer with shared weights, and so on until a 2n-3-th spliced feature vector with a shape of (2*transformer_dim, 1) obtains a 2n-3-th integrated feature vector with a shape of (transformer_dim, 1) through a feature integrated fully connected layer with shared weights.

[0046] The n-th spliced feature vector with a shape of (2*transformer_dim, 1) obtains an n-th integrated feature vector with a shape of (transformer_dim, 1) through a feature integrated fully connected layer with shared weights, an (n+1)-th spliced feature vector with a shape of (2*transformer_dim, 1) obtains an (n+1)-th integrated feature vector with a shape of (transformer_dim, 1) through a feature integrated fully connected layer with shared weights, and so on until a 2n-3-th spliced feature vector with a shape of (2*transformer_dim, 1) obtains a 2n-3-th integrated feature vector with a shape of (transformer_dim, 1) through a feature integrated fully connected layer with shared weights.

[0047] All integrated feature vectors obtain a feature vector with a shape of (transformer_dim, 1) through an average pooling layer, and the feature vector with a shape of (transformer_dim, 1) obtains a binary classification result of motor normal or abnormal through a fully connected layer of binary classification. Embodiment 2

[0048] The application discloses a method for detecting a motor, based on a multi-feature grouping Transformer neural network of embodiment 1, comprising steps S1: multi-feature extraction, S2: deep feature extraction based on Transformer, and S3: feature grouping fusion classification to obtain a binary classification result.

[0049] Step S1: multi-feature extraction.

[0050] The time-domain sound signals collected during the operation of the motor are respectively processed through MFCC methods with different numbers of mel frequency cepstral coefficients (n_mfcc) to extract n different MFCC feature matrices with shapes of (n_frame, n_mfcc1), (n_frame, n_mfcc2), ……(n_frame, n_mfccn).

[0051] Step S2: deep feature extraction based on Transformer.

[0052] Different kinds of MFCC feature matrices respectively represent the feature information of the audio signal in different frequency resolutions and time perception levels. If all MFCC feature matrices are directly spliced and input into the same Transformer module, it may cause mutual interference between information and reduce the discrimination performance of the model. Therefore, the method uses an independent feature extraction structure, that is, for each kind of MFCC feature matrix, a Transformer module is introduced for feature coding. The module includes m multi-head attention mechanisms and K encoder layers, which can extract the time sequence representation ability of each group of features.

[0053] Since the Transformer module is a sequence-to-sequence structure, its output maintains the same two-dimensional shape as the input. Therefore, after the n kinds of MFCC feature matrices pass through their respective Transformer modules, they output n kinds of Transformer feature matrices with shapes of (n_frame, n_mfcc1), (n_frame, n_mfcc2), …, (n_frame, n_mfccn).

[0054] In order to unify the feature dimension and facilitate subsequent feature grouping and fusion processing, each Transformer feature matrix is further processed through an embedded fully connected layer. The weight size of the embedded fully connected layer is (n_mfcc1, transformer_dim), (n_mfcc2, transformer_dim), …, (n_mfccn, transformer_dim) respectively. Finally, each feature matrix is compressed into an embedded feature vector with a shape of (transformer_dim, 1), denoted as e1, e2, …, en.

[0055] Step S3: Feature grouping and fusion classification to obtain a binary classification result.

[0056] The n embedded feature vectors obtained in the foregoing are denoted as e1, e2, …, en. Each two embedded feature vectors are combined and spliced, for example, e1 and e2, e1 and e3, e1 and e4, …, e1 and en, e2 and e3, e2 and e4, e2 and e5, …, e2 and en, …, en-1 and en. A total of n×(n-1) / 2 spliced feature vectors can be obtained. This splicing method is similar to the idea of grouped convolution, which aims to enhance the information expression ability while reducing the mutual interference between different features. The shape of these spliced feature vectors is (2×transformer_dim, 1).

[0057] Next, these concatenated feature vectors will be uniformly input into a feature synthesis fully connected layer with shared weights. This fully connected layer is used for feature synthesis, and its input dimension is 2 times transformer_dim, and the output dimension is transformer_dim. Since the feature synthesis fully connected layer shares weights in all concatenated vectors, not only does it effectively control the number of parameters, but it also improves the generalization ability of the model. Through this process, n x (n-1) / 2 synthesized feature vectors will be obtained, with a shape of (transformer_dim, 1).

[0058] Next, average pooling is performed on these synthesized feature vectors, that is, their mean values are calculated in the dimension, and finally a unified feature vector is obtained, with a shape of transformer_dim rows and 1 column.

[0059] Finally, this final unified feature vector is input into a two-class fully connected layer for output of the final classification result.

[0060] Summarizing the technical concept of the present application, the following is stated.

[0061] In the present application, n kinds of MFCCs, i.e., MFCCs with different numbers of mel frequency cepstral coefficients (n_mfcc), are used to extract features from the original audio, obtaining n kinds of MFCC feature matrices. Different kinds of MFCC feature matrices represent different frequency resolution and time perception ability of the audio signal, and direct concatenation input into the Transformer may cause information confusion and interference enhancement. Therefore, the present application adopts an independent feature extraction method, i.e., each MFCC feature matrix is individually passed through a Transformer module to fully extract its unique sequential representation ability, obtaining n kinds of embedding feature vectors.

[0062] Secondly, the n kinds of embedding feature vectors e1, e2, ……, en are concatenated two by two, such as e1 and e2, e1 and e3, etc., a total of n* (n-1) / 2 kinds of concatenated embedding feature vectors, similar to the idea of grouped convolution, the concatenated embedding feature vectors can avoid mutual interference between features, and through a shared fully connected layer, since the fully connected layer has many parameters, this shared fully connected layer not only reduces the number of parameters, but also improves the generalization.

[0063] Finally, average pooling is performed, and a two-class fully connected layer is used to obtain the classification result.

[0064] Data Examples:

[0065] In order to make the content, purpose and advantages of the present application more clear and detailed, the present application will be described in more detail in combination with the following data examples.

[0066] The present application uses the public dataset MIMII Dataset to verify the superiority of the industrial motor equipment anomaly detection method based on the multi-feature grouping Transformer neural network. The MIMII Dataset is a reliable dataset for fault industrial machine investigation and inspection. It contains the sounds produced by four industrial machines, i.e. valves, pumps, fans and slides. Four models (id0, id2, id4, id6) are disclosed for each valve, pump, fan and slide, respectively, and each model discloses -6db, 0db, 6db signal-to-noise ratio sound data, and the data of each model contains normal sound and abnormal sound.

[0067] The present application selects the -6db signal-to-noise ratio fan motor sound of id2 as experimental data, and cuts the training set and test set in the ratio of 8:2.

[0068] In feature extraction: n_frame=5, n=3, n_mfcc1=32, n_mfcc2=64, n_mfcc3=128.

[0069] The Transformer module encodes the features. The module has m=4 multi-head attention mechanisms and K=2 encoder layers.

[0070] The embedded fully connected layer has transformer_dim=64.

[0071] The experimental results are shown as follows.

[0072] Compare n=1, i.e. only use one MFCC model, n=2, use two MFCC models, and n=3, use three MFCC models.

[0073] When n=1, n_mfcc=32, the accuracy is 97.5%; when n=1, n_mfcc=64, the accuracy is 99.2%; when n=1, n_mfcc=128, the accuracy is 99.2%.

[0074] When n=2, integrate n_mfcc1=32 and n_mfcc2=64, the accuracy is 99.3%; when n=2, integrate n_mfcc1=32 and n_mfcc2=128, the accuracy is 99.3%; when n=2, integrate n_mfcc1=64 and n_mfcc2=128, the accuracy is 99.4%.

[0075] When n=3, integrate n_mfcc1=32, n_mfcc2=64 and n_mfcc3=128, the accuracy is 99.3%.

[0076] Therefore, it can be concluded that the method of industrial motor equipment anomaly detection based on multi-feature grouping Transformer neural network is better than using a single feature. However, when the accuracy of a single MFCC is low, it will also affect the method of grouping multiple MFCCs. Here, the accuracy of n_mfcc=32 is the lowest, which is 97.5%. Therefore, by not using n_mfcc=32, integrating n_mfcc1=64 and n_mfcc2=128, the model has an accuracy of 99.4%. This model has the highest accuracy.

Claims

1. A method for detecting an electric machine based on a multi-feature grouping Transformer neural network, characterized in that: the multi-feature grouping Transformer neural network comprises: a plurality of feature extraction modules, a Transformer feature extraction module and a feature grouping fusion classification module connected in sequence, the plurality of feature extraction modules comprise n first to nth MFCC modules, the Transformer feature extraction module comprises n first to nth Transformer modules and n first to nth embedded fully connected layers, one MFCC module is connected with a corresponding Transformer module, and one Transformer module is connected with a corresponding embedded fully connected layer; the feature grouping fusion classification module comprises first to n×(n-1) / 2 splicing modules, a shared weight feature comprehensive fully connected layer, an average pooling layer and a binary classification fully connected layer, each two embedded fully connected layers are combined to be fused with one splicing module, all the splicing modules are connected with the shared weight feature comprehensive fully connected layer, the shared weight feature comprehensive fully connected layer is connected with the average pooling layer, and the average pooling layer is connected with the binary classification fully connected layer; the method comprises the following steps: Step S1: obtaining an electric machine sound time domain signal, and obtaining a first MFCC feature matrix with a shape of (n_frame, n_mfcc1) to an nth MFCC feature matrix with a shape of (n_frame, n_mfccn) from the electric machine sound time domain signal through the plurality of feature extraction modules; Step S2: obtaining a first embedded feature vector e1 with a shape of (transformer_dim, 1) to an nth embedded feature vector en with a shape of (transformer_dim, 1) from the first MFCC feature matrix with the shape of (n_frame, n_mfcc1) to the nth MFCC feature matrix with the shape of (n_frame, n_mfccn) through the Transformer feature extraction module; Step S3: obtaining a binary classification result through the feature grouping fusion classification module from the first embedded feature vector e1 with the shape of (transformer_dim, 1) to the nth embedded feature vector en with the shape of (transformer_dim, 1), and the binary classification result is normal or abnormal.

2. A method of detecting an electric machine as claimed in claim 1, characterized in that: In the step S1, the electric machine sound time domain signal is subjected to feature extraction through the first MFCC module to obtain the first MFCC feature matrix with the shape of (n_frame, n_mfcc1), the electric machine sound time domain signal is subjected to feature extraction through the second MFCC module to obtain the second MFCC feature matrix with the shape of (n_frame, n_mfcc2), and the electric machine sound time domain signal is subjected to feature extraction through the nth MFCC module to obtain the nth MFCC feature matrix with the shape of (n_frame, n_mfccn).

3. The method of claim 1, wherein: In the step S2, the first MFCC feature matrix with the shape of (n_frame, n_mfcc1) is input into the first Transformer module to extract features and obtain a first Transformer feature matrix with the shape of (n_frame, n_mfcc1), and the first Transformer feature matrix with the shape of (n_frame, n_mfcc1) is output into the first embedded full connection layer to obtain a first embedded feature vector e1 with the shape of (transformer_dim, 1); the second MFCC feature matrix with the shape of (n_frame, n_mfcc2) is input into the second Transformer module to extract features and obtain a second Transformer feature matrix with the shape of (n_frame, n_mfcc2), and the second Transformer feature matrix with the shape of (n_frame, n_mfcc2) is output into the second embedded full connection layer to obtain a second embedded feature vector e2 with the shape of (transformer_dim, 1); and until the nth MFCC feature matrix with the shape of (n_frame, n_mfccn) is input into the nth Transformer module to extract features and obtain an nth Transformer feature matrix with the shape of (n_frame, n_mfccn), and the nth Transformer feature matrix with the shape of (n_frame, n_mfccn) is output into the nth embedded full connection layer to obtain an nth embedded feature vector en with the shape of (transformer_dim, 1).

4. The method of claim 1, wherein: In the step S3, the first embedded feature vector e1 with the shape of (transformer_dim, 1) and the second embedded feature vector e2 with the shape of (transformer_dim, 1) are input into the first splicing module to obtain a first spliced feature vector with the shape of (2*transformer_dim, 1) after splicing, the first embedded feature vector e1 with the shape of (transformer_dim, 1) and the third embedded feature vector e3 with the shape of (transformer_dim, 1) are input into the second splicing module to obtain a second spliced feature vector with the shape of (2*transformer_dim, 1) after splicing, and until the first embedded feature vector e1 with the shape of (transformer_dim, 1) and the nth embedded feature vector en with the shape of (transformer_dim, 1) are input into the (n-1)th splicing module to obtain an (n-1)th spliced feature vector with the shape of (2*transformer_dim, 1) after splicing; The second embedding feature vector e2 with a shape of (transformer_dim, 1) and the third embedding feature vector e3 with a shape of (transformer_dim, 1) are spliced by the nth splicing module to obtain an spliced nth splicing feature vector with a shape of (2*transformer_dim, 1), the second embedding feature vector e2 with a shape of (transformer_dim, 1) and the fourth embedding feature vector e4 with a shape of (transformer_dim, 1) are spliced by the n+1th splicing module to obtain an spliced n+1th splicing feature vector with a shape of (2*transformer_dim, 1), and so on, until the second embedding feature vector e2 with a shape of (transformer_dim, 1) and the nth embedding feature vector en with a shape of (transformer_dim, 1) are spliced by the 2n-3th splicing module to obtain an spliced 2n-3th splicing feature vector with a shape of (2*transformer_dim, 1); until the n-1th embedding feature vector en-1 with a shape of (transformer_dim, 1) and the nth embedding feature vector en with a shape of (transformer_dim, 1) are spliced by the n*n-1 / 2th splicing module to obtain an spliced n*n-1 / 2th splicing feature vector with a shape of (2*transformer_dim, 1).

5. A method of detecting an electric machine as claimed in claim 4, characterized in that: In the step S3, all the spliced splicing feature vectors obtain n*n-1 / 2 comprehensive feature vectors with a shape of (transformer_dim, 1) through a feature comprehensive fully connected layer with shared weights, all the comprehensive feature vectors obtain a feature vector with a shape of (transformer_dim, 1) through an average pooling layer, and the feature vector with a shape of (transformer_dim, 1) obtains a binary classification result through a binary classification fully connected layer.

6. The method of claim 1, wherein: The number n_mfcc of mel frequency cepstral coefficients of the MFCC module ranges from 13 to 128.

7. The method of claim 1, wherein: The n ranges from 2 to 6.

8. The method of claim 1, wherein: The n ranges from 2 to 6. The n ranges from 2 to 6. The n ranges from 2 to 6.

9. The method of claim 1, wherein: The combination of every two embedded full connection layers is fused and connected with a splicing module, that is, the first embedded full connection layer is connected with the first splicing module, the second embedded full connection layer is connected with the first splicing module, the first embedded full connection layer is connected with the second splicing module, the third embedded full connection layer is connected with the second splicing module, until the first embedded full connection layer is connected with the (n-1)th splicing module, and the nth embedded full connection layer is connected with the (n-1)th splicing module; the second embedded full connection layer is connected with the nth splicing module, the third embedded full connection layer is connected with the nth splicing module, the second embedded full connection layer is connected with the (n+1)th splicing module, the fourth embedded full connection layer is connected with the (n+1)th splicing module, until the second embedded full connection layer is connected with the (2n-3)th splicing module, and the nth embedded full connection layer is connected with the (2n-3)th splicing module; until the (n-1)th embedded full connection layer is connected with the nth×(n-1) / 2 splicing module, and the nth embedded full connection layer is connected with the nth×(n-1) / 2 splicing module.

Images