Unsupervised machine abnormal sound detection method based on bilinear feature fusion
By adopting the combination of bilinear feature fusion and channel space attention mechanism in the abnormal sound detection of unsupervised machines, the problem of low data volume of abnormal sound is solved, and more efficient abnormal sound detection is achieved, improving detection accuracy and performance , making up for the blind spots of video surveillance and improving the safety and efficiency of industrial production.
Patent Information
- Application Number
- CN202411159640.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-22
- Publication Date
- 2024-11-26
AI Technical Summary
In the detection of abnormal sounds, the prior art is difficult to effectively distinguish abnormal sound from normal sounds due to the limited amount of abnormal sound data, which leads to the impact of detection performance, especially in an unsupervised learning environment.
Adopt unsupervised machine anomaly sound detection method based on bilinear feature fusion, and combine logarithmic Meer spectral features and bilinear feature fusion technology to add channel space attention mechanism to improve feature extraction and fusion capabilities to achieve more accurate Abnormal sound detection.
It improves the accuracy and performance of abnormal sound detection of unsupervised machines, can effectively identify machine failures under a small amount of abnormal sound data, improves the timeliness and accuracy of the detection system, reduces dependence on human resources, and reduces costs.
Smart Images

Figure CN119026083A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the field of machine abnormal sound detection, and in particular relates to an unsupervised machine abnormal sound detection method based on bilinear feature fusion. Background Art
[0002] The main goal of anomaly detection is to distinguish abnormal events from normal events. The task of anomaly detection can be achieved in many ways. One of the simplest methods is to treat the problem as a supervised learning task and train classifiers for normal and abnormal sounds. The problem with this method is that abnormal sounds rarely occur and are difficult to collect in bursts. Correspondingly, the amount of data for abnormal classes is limited, which will have a great impact on classification performance. Another unsupervised abnormal sound detection used in the present invention refers to the abnormal sound detection system whose detection target is "abnormal" relative to the normal state. Using an unsupervised method to model normal sounds, this method can predict the sound state of the device without collecting a large amount of abnormal sound data.
[0003] In modern industrial production, the audio abnormal event detection system has become a very important tool. Although traditional video surveillance systems can provide effective monitoring methods in many aspects, they may have blind spots and limitations in certain specific situations. The audio abnormal event detection system makes up for the shortcomings of video monitoring by capturing and analyzing audio signals in the factory environment, making the monitoring system more comprehensive and three-dimensional. Compared with traditional manual inspections, the audio monitoring system not only improves the timeliness and accuracy of fault detection, but also greatly reduces the dependence on human resources. This automated monitoring method saves costs and improves production efficiency. On the technical level, the system deeply explores the subtle features of the machine operation sound, combines the logarithmic Mel spectrum features with the bilinear feature fusion technology, so that the system can more accurately reveal the hidden fault information. In addition, the system also adds an attention mechanism to the feature fusion part, which further improves the ability to extract feature information, thereby achieving more accurate abnormal event detection. This technological innovation not only improves the performance of the detection system, but also provides a new perspective for the safety and efficiency of industrial production. Summary of the invention
[0004] Technical problem to be solved by the present invention: In order to overcome the problems raised in the background technology, the present invention provides an unsupervised machine abnormal sound detection method based on bilinear feature fusion, which can improve the detection capability of unsupervised machine abnormal sound and effectively improve the overall performance of the model.
[0005] The technical solution of the present invention is as follows: an unsupervised machine abnormal sound detection method based on bilinear feature fusion, the method comprising the following steps: Step 1, data preprocessing; Step 2, feature extraction; Step 3, fusion stage.
[0006] As an improvement of the present invention, the data preprocessing in step 1 includes the following steps: Step 1.1, collect sound samples of normal and abnormal operation status of the factory machine to be tested as data sets for model training and testing; Step 1.2, preprocessing the sound sample data to obtain a number of audio frames.
[0007] As an improvement of the present invention, the feature extraction in step 2 includes the following steps: Step 2.1, extracting the logarithmic Mel spectrum features of the audio frame after data preprocessing; Step 2.2, construct an unsupervised machine abnormal sound detection network based on bilinear feature fusion; In step 2.3, the extracted logarithmic Mel spectrum features are sent to the abnormal sound detection network for training.
[0008] As an improvement of the present invention, the step 3 fusion stage includes the following steps: Step 3.1, use the bilinear method to fuse the input features to obtain the fused features; Step 3.2, use the channel space attention mechanism to assign different attention weights to the fused features; Step 3.3: Use the sound sample data in the test set to test the trained abnormal sound detection model and evaluate the overall performance of the model.
[0009] As an improvement of the present invention, in step 1.1, the training set only includes machine sound samples in normal operating state, and the test set includes machine sound samples in both normal and abnormal operating states.
[0010] As an improvement of the present invention, in step 1.2, preprocessing operations including pre-emphasis, framing and windowing are performed on the collected audio samples.
[0011] As an improvement of the present invention, in step 2.1, the logarithmic Mel spectrum features are extracted based on the auditory perception characteristics of the human ear, and the logarithmic Mel spectrum feature matrix is generated by an FFT group with a length of 1024, 128 Mel filters and a window length of 230.
[0012] As an improvement of the present invention, in step 2.2, the classification model used by the unsupervised machine abnormal sound detection network is a machine type classification self-supervised learning method based on the MobileFaceNet neural network. The trained classification model cannot directly identify abnormalities in the test data. An abnormality detection algorithm using the Mahalanobis distance as the abnormality score of the audio is used for subsequent calculations, and the formula is as follows:
[0013] in, is the feature vector of the sample to be detected, is the mean vector of normal samples, is the covariance matrix of the normal sample, is the inverse of the covariance matrix, It represents a given sample Mahalanobis distance to the normal sample distribution.
[0014] As an improvement of the present invention, in step 3.1, a bilinear feature fusion method is used to perform bilinear fusion on two tensors of size [32, 128, 1, 1] at the same position on the last linear layer in the MobileFaceNet neural network. First, the two input tensors are reshaped into a tensor of size [32, 128, 1], and then the two tensors are matrix transposed and multiplied, and the multiplication result is reassigned to a new tensor of size [32, 128, 128], and finally the new feature matrix is expanded into a fused feature vector of size [32, 128*128]. The obtained fused feature vector is first taken square root using the sqrt function, and then normalized using the normalize function, and finally the processed fused feature vector is sent to a fully connected layer of dimension [128*128, 128] to obtain a bilinear fused feature vector of final size [32, 128].
[0015] As an improvement of the present invention, in step 3.2, the channel space attention mechanism is used to first perform spatial dimension information compression on the input features, and the feature map is compressed into a compressed map with unchanged number of channels through two parallel maximum pooling layers and average pooling layers, and then the number of channels is compressed to 1 / 16 of the original number and expanded back to the original number of channels, and the two results after the ReLU activation function are added element by element, and then the final output is obtained through the sigmoid activation function. Then, the information compression of the channel dimension is performed again, and the feature map is sequentially passed through the maximum pooling layer and the average pooling layer to obtain two feature maps with the number of channels compressed to 1, and then the two feature maps are spliced and converted into a feature map of 1 channel through 7*7 convolution, and the result is multiplied by the original image after the sigmoid activation function to return to the original size. Finally, the two attention weights are added to the fused features to obtain the final fused features.
[0016] Compared with the prior art, the beneficial effects of the present invention are: (1) In view of the fact that there are few abnormal sounds in actual applications and only normal sound data can be provided, the present invention performs bilinear feature fusion on the extracted features, performs matrix fusion on different features at the same position from a new perspective, obtains more subtle machine sound signal representation, and improves the accuracy of abnormal sound detection; (2) A channel space attention mechanism is added to the bilinear feature fusion method to add attention weights to two different features at the same position in bilinear fusion, so as to better obtain subtle features related to the machine itself and improve the accuracy of abnormal sound detection. BRIEF DESCRIPTION OF THE DRAWINGS
[0017] Figure 1 It is a flow chart of the unsupervised machine abnormal sound detection method based on bilinear feature fusion of the present invention; Figure 2 The invention discloses a bilinear feature fusion algorithm structure for an unsupervised machine abnormal sound detection method based on bilinear feature fusion. DETAILED DESCRIPTION
[0018] The present invention will be further explained below in conjunction with the accompanying drawings and specific embodiments. It should be understood that the following specific embodiments are only used to illustrate the present invention and are not used to limit the scope of the present invention. Example
[0019] Figure 1 The present invention is a flow chart of an embodiment of an unsupervised machine abnormal sound detection method based on time-frequency feature fusion, and the unsupervised machine abnormal sound detection method comprises the following steps: (1) Collect sound samples of the factory machines under normal and abnormal operating conditions as data sets for model training and testing.
[0020] (2) Perform preprocessing operations on the sound sample data to obtain several audio frames.
[0021] (3) Extract the logarithmic Mel-spectrogram features of the audio frame after data preprocessing.
[0022] (4) Construct an unsupervised machine abnormal sound detection network based on bilinear feature fusion. The classification model used by the unsupervised machine abnormal sound detection network is a machine type classification self-supervised learning method based on the MobileFaceNet neural network.
[0023] (5) The extracted logarithmic Mel-spectrogram features are sent to the abnormal sound detection network for training.
[0024] (6) Using the bilinear feature fusion method, two tensors of size [32, 128, 1, 1] at the same position on the last linear layer of the MobileFaceNet neural network are bilinearly fused. First, the two input tensors are reshaped into a tensor of size [32, 128, 1], and then the two tensors are matrix transposed and multiplied. The multiplication result is reassigned to a new tensor of size [32, 128, 128], and finally the new feature matrix is stretched into a fused feature vector of size [32, 128*128]. The obtained fused feature vector is first taken square root using the sqrt function, and then normalized using the normalize function. Finally, the processed fused feature vector is sent to a fully connected layer of dimension [128*128, 128] to obtain a final bilinear fused feature vector of size [32, 128].
[0025] (7) Using the channel space attention mechanism, the input features are first compressed in the spatial dimension. Through two parallel maximum pooling layers and average pooling layers, the feature map is compressed into a compressed map with the same number of channels. The number of channels is then compressed to 1 / 16 of the original number and expanded back to the original number of channels. The two results after the ReLU activation function are added element by element, and then the final output is obtained through the sigmoid activation function. Then the information compression of the channel dimension is performed again. The feature map is passed through the maximum pooling layer and the average pooling layer in turn to obtain two feature maps with the number of channels compressed to 1. The two feature maps are then concatenated and converted into a 1-channel feature map through a 7*7 convolution. After the sigmoid activation function, the result is multiplied by the original image to return to its original size. Finally, the two attention weights are added to the fused features to obtain the final fused features.
[0026] (8) Use the sound sample data in the test set to test the trained abnormal sound detection model and evaluate the overall performance of the model.
[0027] The above description is an exemplary embodiment of the present invention, and does not limit the scope of patent protection of the present invention. Any equivalent structure or equivalent process transformation made by using the contents of the specification and drawings of the present invention, or directly or indirectly applied in other related technical fields, are also included in the scope of patent protection of the present invention. For ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principle of the present invention, and these improvements and modifications all fall within the scope of protection of the claims of the present invention.
Claims
1. An unsupervised machine abnormal sound detection method based on bilinear feature fusion, It is characterized in that The method comprises the following steps: Step 1, data preprocessing; Step 2, feature extraction; Step 3, fusion stage.
2. According to claim 1, an unsupervised machine abnormal sound detection method based on bilinear feature fusion, It is characterized in that The data preprocessing step 1 includes the following steps: Step 1.1, collect sound samples of normal and abnormal operation status of the factory machine to be tested as data sets for model training and testing; Step 1.2, preprocessing the sound sample data to obtain a number of audio frames.
3. According to the unsupervised machine abnormal sound detection method based on bilinear feature fusion according to claim 1, It is characterized in that The step 2 feature extraction comprises the following steps: Step 2.1, extracting the logarithmic Mel spectrum features of the audio frame after data preprocessing; Step 2.2, construct an unsupervised machine abnormal sound detection network based on bilinear feature fusion; In step 2.3, the extracted logarithmic Mel spectrum features are sent to the abnormal sound detection network for training.
4. According to the unsupervised machine abnormal sound detection method based on bilinear feature fusion according to claim 1, It is characterized in that The step 3 fusion stage includes the following steps: Step 3.1, use the bilinear pooling method to fuse the input features to obtain the fused features; Step 3.2, use the channel space attention mechanism to assign different attention weights to the fused features; Step 3.3: Use the sound sample data in the test set to test the trained abnormal sound detection model and evaluate the overall performance of the model.
5. According to claim 2, the unsupervised machine abnormal sound detection method based on bilinear feature fusion, It is characterized in that In step 1.1, the training set only includes machine sound samples in normal operating state, and the test set includes machine sound samples in both normal and abnormal operating states.
6. According to claim 2, the unsupervised machine abnormal sound detection method based on bilinear feature fusion, It is characterized in that In the step 1.2, preprocessing operations including pre-emphasis, framing and windowing are performed on the collected audio samples.
7. According to claim 3, the unsupervised machine abnormal sound detection method based on bilinear feature fusion, It is characterized in that In the step 2.1, the logarithmic Mel spectrum features are extracted based on the auditory perception characteristics of the human ear, and the logarithmic Mel spectrum feature matrix is generated by an FFT group with a length of 1024, 128 Mel filters and a window length of 230.
8. According to claim 3, the unsupervised machine abnormal sound detection method based on bilinear feature fusion, It is characterized in that In step 2.2, the classification model used by the unsupervised machine abnormal sound detection network is a machine type classification self-supervised learning method based on the MobileFaceNet neural network. The trained classification model cannot directly identify abnormalities in the test data, and an abnormality detection algorithm using the Mahalanobis distance as the abnormality score of the audio is used for subsequent calculations.
9. According to claim 4, the unsupervised machine abnormal sound detection method based on bilinear feature fusion, It is characterized in that In the step 3.1, a bilinear feature fusion method is used to perform bilinear fusion on two tensors of size [32, 128, 1, 1] at the same position on the last linear layer in the MobileFaceNet neural network. First, the two input tensors are reshaped into a tensor of size [32, 128, 1], and then the two tensors are matrix transposed and multiplied, and the multiplication result is reassigned to a new tensor of size [32, 128, 128]. Finally, the new feature matrix is stretched into a fused feature vector of size [32, 128*128]. The obtained fused feature vector is first taken square root using the sqrt function, and then normalized using the normalize function. Finally, the processed fused feature vector is sent to a fully connected layer of dimension (128*128, 128) to obtain a bilinear fused feature vector of size [32, 128].
10. According to claim 4, the unsupervised machine abnormal sound detection method based on bilinear feature fusion, It is characterized in that In the step 3.2, the channel space attention mechanism is used to first perform spatial dimension information compression on the input features, and the feature map is compressed into a compressed map with the same number of channels through two parallel maximum pooling layers and average pooling layers, and then the number of channels is compressed to 1 / 16 of the original number and expanded back to the original number of channels, and the two results after the ReLU activation function are added element by element, and then the final output is obtained through the sigmoid activation function, and then the channel dimension information is compressed again, and the feature map is passed through the maximum pooling layer and the average pooling layer in turn to obtain two feature maps with the number of channels compressed to 1, and then the two feature maps are concatenated and converted into a feature map of 1 channel through 7*7 convolution, and the result is multiplied by the original image after the sigmoid activation function to return to the original size, and finally the two attention weights are added to the fused features to obtain the final fused features.
