Machine abnormal sound detection method based on Conform network

Through the machine abnormal sound detection method based on Conformer network, the improved network structure and loss function are used to extract the characteristics of normal and abnormal sound of the machine, which solves the problems of traditional fault detection and high hardware cost, and improves the successful identification of abnormal sounds The rate is suitable for small and medium-sized enterprises and enhances production safety guarantees.

CN118918921APending Publication Date: 2024-11-08ZHEJIANG UNIV OF TECH
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202410986225.2
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-23
Publication Date
2024-11-08

AI Technical Summary

Technical Problem

Traditional equipment failure detection methods rely on manual detection, which is time-consuming and labor-intensive and susceptible to subjective factors, making it difficult to ensure accuracy, especially when there are fewer faults and a wide variety of problems during normal operation of the machine, it is difficult to effectively identify abnormal sounds. At the same time, the hardware cost of video surveillance methods is high and it is difficult to popularize small and medium-sized enterprises.

Method used

Using the machine anomaly sound detection method based on the Conformer network, the model is trained using only normal sound samples, and the improved Conformer network is used to identify abnormal sound samples. The global and local features are extracted by combining the log-mel spectrum and classification loss function. Calculate the exception score.

Benefits of technology

It improves the recognition success rate of abnormal sound detection, can work effectively when there are fewer abnormal samples, reduces hardware costs, is suitable for small and medium-sized enterprises, and enhances production safety guarantees.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118918921A_ABST
    Figure CN118918921A_ABST
Patent Text Reader

Abstract

The invention relates to a machine abnormal sound detection method based on a Conformer network, and belongs to the technical field of artificial intelligence and anomaly detection. The method comprises the following steps: S1, generating a training set, wherein the training set uses normal operation sounds of a plurality of devices; s2, using the log-Mel spectrogram and a corresponding machine source label as input; s3, constructing a plurality of improved Conformer encoders, and stacking each output value into a new output value according to the channel dimension; s4, outputting a prediction tensor reflecting the probability belonging to each source label by using self-attention pooling and a full connection layer; and S5, calculating an abnormal score according to a probability value of a prior source tag bit in the prediction tensor. According to the method, the equipment fault voiceprint detection can be effectively realized under the condition that abnormal sample data is not used, and the method has important significance in improving the equipment maintenance efficiency and reducing the fault risk.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention belongs to the technical field of artificial intelligence and anomaly detection, and in particular relates to a machine abnormal sound detection method based on a Conformer network. Background Art

[0002] At present, various automation equipment has been widely used in all walks of life. The stable operation of mechanical systems plays a vital role in the efficiency, quality and overall safety of mechanical production. The traditional equipment fault detection method uses manual comprehensive inspection of equipment, which is not only time-consuming and labor-intensive, but also the experience and ability of inspectors vary. The fault detection results are easily affected by the subjective factors of the inspectors, and it is difficult to guarantee their accuracy, resulting in certain economic losses.

[0003] Most of the time, the machine is in a stable operation phase, during which relatively few faults occur. However, the types of these faults are quite diverse, involving all aspects of the mechanical system. Due to the diversity of fault types, collecting all possible fault sounds of the machine and conducting supervised training has become a very challenging task.

[0004] At present, the mainstream method of enterprises is to use video surveillance to monitor the production environment of the factory. However, the processing of video information streams requires high storage capacity and computing power of the equipment. Therefore, the hardware cost of the system is very expensive, which makes it difficult to popularize it to small and medium-sized enterprises with low purchasing power. As an important part of safety monitoring, the sound-based monitoring system can judge whether the machine is abnormal by the sound emitted by the machine. The system can detect abnormal conditions of the machine in time and provide customers with data basis for decision-making, thereby avoiding major accidents and critical situations, and playing a positive role in ensuring the safety of enterprise production. Summary of the invention

[0005] In view of the above problems existing in the prior art, the object of the present invention is to provide a machine abnormal sound detection method based on a Conformer network, which can realize the recognition of abnormal sound samples when only normal sound samples are used to train the model.

[0006] The present invention provides the following technical solution: a method for detecting abnormal sound of a machine based on a Conformer network, comprising the following steps:

[0007] S1: Collect audio data of the machine under normal operating conditions (i.e., the operating parameters of the equipment are within the set range), divide the collected audio data, and generate a training set. The naming of the training set data includes the characters of the source label and a four-digit code number;

[0008] S2: Segment each audio file in the training set and convert it into a log-mel spectrogram;

[0009] S3: Based on the Conformer encoder, the Depthwise Conv convolution layer of the Conformer encoder is modified to the convolution layer with scale in Res2net, and the output values ​​of multiple Conformer encoders are aggregated for pooling to construct an improved Conformer network;

[0010] S4: input the log-mel spectrogram converted from the training set data and the source label of the training set data into the network constructed in step S3, and train the network using the classification loss function;

[0011] S5: Input the audio to be tested and the source label of the audio to be tested into the network trained in S4, and use the output value to calculate the anomaly score.

[0012] Furthermore, in S1, the format of the audio data is a mono wav file with a sampling rate at a set frequency, and the file naming format is idx_xxxx, where x is a number.

[0013] Furthermore, in S2, the audio data file is segmented according to the set window length and moving interval, and when extracting the log-mel spectrogram of each audio segment, the frame size, jump length and the number of n_Mel filters are set as parameters.

[0014] Furthermore, in S3, the calculation formula of the convolutional layer containing the scale is as follows:

[0015]

[0016] Where x i is the tensor after the input tensor is divided into s groups, K i () represents one-dimensional convolution calculation, y i is the corresponding output tensor;

[0017] Aggregating the output values ​​of multiple Conformer encoders is stacking the three-dimensional tensors of multiple encoders according to the second dimension;

[0018] The calculation formula of the pooling is as follows:

[0019]

[0020] m=Softmax((H-μ) T ×(H-μ))

[0021]

[0022] Where t irepresents the i-th column vector of the third dimension of the tensor, T represents the length of the third dimension, Softmax() represents the softmax normalization calculation, H is the input tensor, is the output tensor, μ represents the average value tensor in the time dimension, and m represents the normalized tensor inner product value matrix.

[0023] Furthermore, in S4, the loss function used is the AAM-Softmax loss function.

[0024] Furthermore, the specific process of step S5 is as follows:

[0025] S5.1: segment the audio file to be tested according to the set window length and moving interval, and set the parameters of frame size, jump length and n_Mel filter number when extracting log-mel spectrogram of each audio segment;

[0026] S5.2: Input all log-mel spectrograms into the trained network to obtain a tensor reflecting the probability that the audio belongs to each source label;

[0027] S5.3: Use the source label of the audio file to be tested to index the values ​​at the corresponding positions of all tensors, calculate the average of all values, and then use the difference between the average and 1 as the anomaly score.

[0028] By adopting the above technology, compared with the prior art, the beneficial effects of the present invention are as follows:

[0029] The machine abnormal sound detection method proposed in the present invention can fully play its role when there are fewer abnormal samples; by utilizing the balanced global feature and local feature extraction capabilities of the network, the network can remain sensitive to the differences between normal samples and abnormal samples, thereby improving the recognition success rate of abnormal sound detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 The figure is a flow chart of a method for detecting abnormal machine sound based on a Conformer network in an embodiment of the present invention.

[0031] Figure 2 4 is a structural diagram of a Conformer encoder in an embodiment of the present invention.

[0032] Figure 3 The present invention is a flowchart of a plurality of encoder output tensor fusion in an embodiment. DETAILED DESCRIPTION

[0033] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments of the specification. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0034] On the contrary, the present invention covers any substitution, modification, equivalent method and scheme made on the essence and scope of the present invention as defined by the claims. Further, in order to make the public have a better understanding of the present invention, some specific details are described in detail in the detailed description of the present invention below. Those skilled in the art can fully understand the present invention without the description of these details.

[0035] Reference Figure 1-3 , a machine abnormal sound detection method based on Conformer network, comprising the following steps:

[0036] S1: Collect the normal working audio of four machines of the same model. Each machine collects 1000 wav format audio files with a duration of 10s at a sampling rate of 16Khz. Each audio file is prefixed with id0, id1, id2, and id3, indicating that it comes from machines 0, 1, 2, and 3 respectively. The suffix _xxxx indicates a four-digit code for different files.

[0037] S2: Each audio file is segmented and converted into a log-mel spectrogram; the audio file is segmented with a window length of 4096ms and a moving interval of 512ms. The parameter settings for extracting the log-mel spectrogram of each audio segment are a frame size of 1024, a jump length of 512, and a number of n_Mel filters of 128; finally, the training set tensor size is (128000, 64, 128), and the additional source label tensor size is (128000, 1), where the elements of the second dimension are integers 0, 1, 2, and 3 representing the sources from machines 0, 1, 2, and 3 respectively.

[0038] S3: Based on the Conformer network, an improved Conformer network is constructed. Specifically, the Depthwise Conv convolution layer of the Conformer encoder is modified to the scaled convolution layer in Res2net. The encoder structure is as follows: Figure 2 As shown, the calculation formula of the convolution layer containing the scale is as follows:

[0039]

[0040] Where x i is the tensor after the input tensor is divided into s groups, s is 4, K i () represents the one-dimensional convolution calculation with a convolution kernel size of 15, y i is the corresponding output tensor;

[0041] The output values ​​of the three encoders are aggregated for pooling. The process of aggregation and pooling is as follows: Figure 3As shown, the three-dimensional tensors output by the three encoders are stacked according to the second dimension. The size of the second dimension of the stacked tensor is three times the original size. The pooling calculation formula is as follows:

[0042]

[0043] Where t i represents the i-th column vector of the third dimension of the tensor, T represents the length of the third dimension, Softmax() represents the softmax normalization calculation, H is the input tensor, is the output tensor.

[0044] S4: Input the log-mel spectrogram and source label of the training set into the network described in S3, train the network using the AAM-softmax classification loss function, with a batch size of 32 and 10,000 iterations, and save the network weight file in pt format.

[0045] S5: Input the audio to be tested and the source label into the network trained in S4, and use the output value to calculate the anomaly score, which is specifically achieved through the following sub-steps:

[0046] S5.1: The audio file to be tested is segmented with a window length of 4096ms and a moving interval of 512ms. The parameter settings for extracting log-mel spectrograms from each audio segment are: frame size is 1024, jump length is 512, number of n_Mel filters is 128, and the tensor size of each spectrogram is (64, 128);

[0047] S5.2: Input all log-mel spectrograms into the network after loading the pt file and setting it to evaluation mode, and obtain a tensor reflecting the probability of the audio belonging to each source label. The shape of the prediction tensor is (n, 4), where n represents the number of segments separated by the audio file;

[0048] S5.3: Use the source tag of the file to index the values ​​at the corresponding positions of all tensors, that is, the source tag id0 means to save the element at the 0th position on the second dimension, and so on. The size of the indexed tensor is (n, 1). The average of all the values ​​in the tensor is calculated, and then the difference between the average and 1 is used as the anomaly score. The larger the anomaly score, the more obvious the abnormal state.

[0049] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A machine abnormal sound detection method based on Conformer network, characterized in that: The following steps are involved: S1: Collect audio data of the machine in normal operation, divide the collected audio data, and generate a training set. The name of the training set data includes the characters of the source label and a four-digit code number; S2: Segment each audio file in the training set and convert it into a log-mel spectrogram; S3: Based on the Conformer encoder, the Depthwise Conv convolution layer of the Conformer encoder is modified to the convolution layer with scale in Res2net, and the output values ​​of multiple Conformer encoders are aggregated for pooling to construct an improved Conformer network; S4: input the log-mel spectrogram converted from the training set data and the source label of the training set data into the network constructed in step S3, and train the network using the classification loss function; S5: Input the audio to be tested and the source label of the audio to be tested into the network trained in S4, and use the output value to calculate the anomaly score.

2. According to claim 1, a method for detecting abnormal machine sound based on Conformer, characterized in that: In S1, the format of the audio data is a mono wav file with a sampling rate at a set frequency, and the file naming format is idx_xxxx, where x is a number.

3. The method for detecting abnormal machine sound based on Conformer according to claim 1, characterized in that: In S2, the audio data file is segmented according to the set window length and moving interval, and when extracting the log-mel spectrogram of each audio segment, the frame size, jump length and the number of n_Mel filters are set as parameters.

4. According to claim 1, a method for detecting abnormal machine sound based on Conformer, characterized in that: In S3, the calculation formula of the convolutional layer containing the scale is as follows: Where x i is the tensor after the input tensor is divided into s groups, K i () represents one-dimensional convolution calculation, y i is the corresponding output tensor; Aggregating the output values ​​of multiple Conformer encoders is stacking the three-dimensional tensors of multiple encoders according to the second dimension; The calculation formula of the pooling is as follows: m=Softmax((H-μ) T ×(H-μ)) Where t i represents the i-th column vector of the third dimension of the tensor, T represents the length of the third dimension, Softmax() represents the softmax normalization calculation, H is the input tensor, is the output tensor, μ represents the average value tensor in the time dimension, and m represents the normalized tensor inner product value matrix.

5. The method for detecting abnormal machine sound based on Conformer according to claim 1, characterized in that: In S4, the loss function used is the AAM-Softmax loss function.

6. The method for detecting abnormal machine sound based on Conformer according to claim 1, characterized in that: The specific process of step S5 is as follows: S5.1: segment the audio file to be tested according to the set window length and moving interval, and set the parameters of frame size, jump length and n_Mel filter number when extracting log-mel spectrogram of each audio segment; S5.2: Input all log-mel spectrograms into the trained network to obtain a tensor reflecting the probability that the audio belongs to each source label; S5.3: Use the source label of the audio file to be tested to index the values ​​at the corresponding positions of all tensors, calculate the average of all values, and then use the difference between the average and 1 as the anomaly score.