A Self-Supervised Machine Abnormal Sound Detection Method Based on Dual-Path WaveNet

Abstract

The present application provides a self-supervised machine abnormal sound detection method based on a dual-path WaveNet, including: Step 1: Use WaveNet in the frequency-domain path to model the frequency information of each time frame of the input sound signal and extract frequency-domain features; Step 2: The time-domain path takes the output of the frequency-domain path as input, transposes it, and models the time dimension; Step 3: The outputs of the frequency-domain path and the time-domain path are further fused through a two-dimensional convolutional layer to generate a one-dimensional embedding vector; Step 4: Adopt a self-supervised strategy to construct a classification task using metadata to train the model; Step 5: For the trained model, in the detection stage, determine whether it is normal according to the abnormal score of the model test sample. The dual-path design of the present application realizes the comprehensive modeling of the time-frequency spectrogram, not only retains the local details of the time and frequency dimensions, but also can effectively capture the frequency-domain harmonic information and the time-domain dynamic characteristics, and solves the problem of insufficient modeling of complex features in the time-frequency spectrogram.

Description

Technical Field

[0001] This application relates to the technical field of sound signal processing. Specifically, it relates to a self-supervised machine abnormal sound detection method based on a dual-path WaveNet. Background Art

[0002] Sound, as a carrier of information transmission, contains rich information. Detection based on sound signals has become an indispensable technical means in multiple fields. In industrial production, by monitoring and analyzing the sounds generated by machines during operation, not only can potential equipment failures or problems be detected in a timely manner, but also valuable opportunities are provided for taking timely maintenance measures. This operation method helps reduce equipment downtime and slow down equipment wear, thus significantly improving overall production efficiency. At the same time, by proactively solving potential problems, it is also possible to extend the service life of equipment, reduce unplanned maintenance requirements, and effectively reduce production costs. The machine abnormal sound detection technology not only has significant significance in terms of economic benefits, but also has a direct positive impact on workplace safety. Detecting abnormal situations during equipment operation in a timely manner enables early warning of potential safety risks. By preventing accidents and reducing equipment failures, this technology helps create a safer working environment and effectively protects employees from potential harm.

[0003] Traditional machine abnormal sound detection methods mainly rely on the experience and rules in professional fields, and judge whether the sound is abnormal by setting thresholds or using rules. With the rise of machine learning and deep learning technologies, data-driven methods have gradually become the mainstream. By collecting a large amount of sound data and using machine learning algorithms for training, abnormal sounds can be more accurately identified and classified. The new generation of abnormal sound detection systems, especially deep learning models such as convolutional neural networks (CNNs) and recurrent neural networks (RNNs), can extract abstract features from large-scale and complex sound data, achieve efficient abnormal sound recognition, and improve the accuracy and adaptability of detection. Researchers collect normal and abnormal samples, extract the features of the signals, and then use machine learning methods to extract the features of these samples, so as to achieve the detection of abnormal sounds.

[0004] Although significant progress has been made in abnormal sound detection technology, in a real-world environment, it is relatively difficult to obtain abnormal sounds. Even if abnormal sound samples are obtained by deliberately creating abnormalities, the resulting data often fails to cover all possible abnormal situations; especially in some cases where machines need to be deliberately damaged to collect abnormal sounds, this method is not only costly but also impractical. Faced with extreme data imbalance, using mainstream supervised learning methods for abnormal sound detection will result in high accuracy for normal sounds but a large number of missed detections of abnormal sounds. Therefore, unsupervised anomaly detection is more realistic. It does not require adding normal and abnormal labels to the data, but only uses the daily operating sounds of the machine collected. Based on the distribution characteristics of this information, it judges unknown abnormal sounds to achieve effective anomaly detection. This method avoids the problems of scarce abnormal samples and data imbalance, provides a more practical solution for the anomaly detection system, and enables it to more flexibly adapt to different actual industrial scenarios.

[0005] For this reason, researchers have proposed an unsupervised ASD method that only uses normal sound data during the training phase to let the model learn the rules and characteristics of normal sounds. As Figure 1 shown, during the detection phase, the model determines whether an input sound is abnormal by calculating the degree of deviation of the input sound from the normal pattern. Specifically, the model calculates an anomaly score for each input sound and compares it with a preset threshold . When the anomaly score exceeds the threshold, the input sound is determined to be abnormal.

[0006] The main strategies for implementing unsupervised abnormal sound detection include two categories: reconstruction-based methods and self-supervised methods. Reconstruction-based methods usually use autoencoders to reconstruct the input sound feature map and use the reconstruction error as the anomaly score. During testing, the test samples are judged to be normal or abnormal by comparing the anomaly score with the threshold. However, such methods may encounter problems when dealing with non-stationary environments, and the reconstruction error of normal sounds is sometimes higher than that of abnormal sounds. To solve this problem, researchers have proposed self-supervised methods. This method uses certain attributes in the data to set up a proxy task. By solving the supervised proxy task, it learns the representation of normal signals from the data. For example, through relevant information of audio files, such as machine type or machine ID as labels, the model is trained to classify them, so that the model learns more compact feature representations for each category. During testing, the true class probability value predicted by the model is used as the anomaly score, and it is judged whether the anomaly score exceeds the threshold to measure whether the sample is abnormal.

[0007] As Figure 2As shown, the self-supervised anomaly sound detection method enables the model to learn the feature distribution of normal sounds without real labels by designing predefined pseudo-supervised tasks. In the training phase, metadata (such as device category or environmental information) is usually used as pseudo-labels to construct a classification task, enabling the model to learn the discriminative boundary between normal sounds. In the detection phase, the input sound is subjected to anomaly detection based on the classification probability of the model. Normal sounds are usually classified with high confidence, while anomaly sounds have a lower classification confidence due to deviating from the normal distribution. When the anomaly score is higher than the set threshold, the sample is judged to be anomalous.

[0008] The time-frequency spectrogram can intuitively represent the frequency components and time variations of a sound signal in the form of a two-dimensional image, thereby capturing the complex time-frequency structure features in the sound. Sound signals are usually non-stationary, and their frequency intensity changes dynamically over time, while the time-frequency spectrogram can well describe these characteristics, enabling the differences between normal and anomaly sounds to be fully manifested. Compared with directly processing the one-dimensional original waveform, the time-frequency spectrogram can more clearly reveal the unique frequency distribution, energy changes, and harmonic structure of anomaly sounds. In addition, the image-like characteristics of the time-frequency spectrogram enable it to be directly combined with models in deep learning to extract deep-level features, greatly improving the detection performance. The time-frequency spectrogram not only enriches the feature information but also enhances the model's recognition ability and robustness to anomaly sounds.

[0009] However, currently, most methods usually treat the time-frequency spectrogram as ordinary image processing and use a two-dimensional convolutional neural network for modeling when extracting time-frequency spectrogram features. Due to the limited receptive field of the two-dimensional convolutional network, this limitation makes it difficult to capture specific information crucial for anomaly detection when processing the time-frequency spectrogram. On the one hand, the harmonic structure in the frequency domain often contains important feature information. For example, the harmonics of normal sounds usually exhibit a stable and regular frequency distribution, while anomaly sounds may break this regularity. However, due to the fixed receptive field of the two-dimensional convolutional network, it is difficult to effectively capture these complex cross-frequency-band harmonic relationships. On the other hand, the global changes in the time domain, such as the rhythm, temporal pattern, and dynamic characteristics of the sound signal, are also of great significance for anomaly detection. However, the two-dimensional convolutional network tends to focus more on local region features and cannot effectively model long-range dependencies or dynamic change trends in the time dimension. Therefore, relying solely on the two-dimensional convolutional network to process the time-frequency spectrogram may lead to the neglect of important global information, thus affecting the accuracy and robustness of anomaly detection.

[0010] For this reason, some studies have attempted to use Transformer and WaveNet to model information in specific dimensions. With its powerful global modeling ability and parallel processing efficiency, Transformer can effectively capture long-range dependencies in the frequency domain or time domain through its self-attention mechanism. However, when dealing with spectrograms, Transformer often requires a large amount of computing resources, and its ability to capture local features is relatively limited, making it difficult to fully retain frequency details or subtle changes between time frames.

[0011] As Figure 3 shown, WaveNet is a one-dimensional convolutional neural network originally designed to generate raw audio and has shown remarkable capabilities in signal feature extraction. Its structure is based on residual blocks, combining dilated convolutions, gated units, residual connections, and skip connections, enabling the model to efficiently capture long-range dependencies and retain local information. Dilated convolutions gradually expand the receptive field, allowing the model to extract multi-scale features without increasing computational complexity. Residual connections enhance gradient flow and feature learning, while gated units control the flow of information by combining sigmoid and tanh functions, further optimizing the model performance.

[0012] In contrast, WaveNet gradually expands the receptive field through dilated convolutions, enabling it to efficiently capture long-range dependencies while retaining local information. However, traditional WaveNet is only applicable to one-dimensional signal processing and has difficulty directly modeling the complex correlation between time and frequency when faced with two-dimensional spectrograms. Although there have been methods attempting to extend WaveNet to model frequency domain information, this single-dimensional processing method cannot fully reflect the global characteristics of spectrograms. Especially in anomaly detection tasks, the harmonic features in the frequency domain and the dynamic changes in the time domain are often inseparable. Summary of the Invention

[0013] In view of this, the present application provides a self-supervised machine abnormal sound detection method based on dual-path WaveNet, which realizes the gradual extraction of time-domain and frequency-domain features while effectively maintaining the correlation between time-frequency domain information.

[0014] To achieve the above object, the technical solution adopted by the present application is as follows:

[0015] A self-supervised machine abnormal sound detection method based on dual-path WaveNet, comprising:

[0016] Step 1: Use WaveNet in the frequency domain path to model the frequency information of each time frame of the input sound signal and extract frequency domain features: ;

[0017] Where The WaveNet mapping function representing the frequency-domain path represents the input data The dimension is , is the output frequency-domain feature;

[0018] Step 2: The time-domain path takes the output of the frequency-domain path as the input. After transposing it, it is regarded as one-dimensional sequences of length . Then, WaveNet is used for modeling in the time dimension, and the output is the time-domain feature : ;

[0019] where represents the WaveNet mapping function of the time-domain path;

[0020] Step 3: The outputs of the frequency-domain path and the time-domain path further fuse features through a two-dimensional convolutional layer to generate a one-dimensional embedding vector;

[0021] Step 4: Adopt a self-supervised strategy, use metadata to construct a classification task to train the model, and the classification loss function is ArcFace Loss;

[0022] Step 5: For the trained model, in the detection stage, the model calculates the anomaly score A through the classification probability. If the anomaly score A of the test sample exceeds the set threshold, it is determined as an anomaly; otherwise, it is judged as normal.

[0023] Furthermore, the calculation method of the classification loss function ArcFace Loss in Step 4 is as follows:

[0024] Step 4.1: Normalize both the one-dimensional embedding vector and the class weight vector;

[0025] Step 4.2: Obtain the angular value of each class by calculating the cosine similarity between the normalized one-dimensional embedding vector and the class weight vector ;

[0026] ;

[0027] where represents the one-dimensional embedding vector, represents the class weight vector, represents the input sample and the class the angle between;

[0028] Step 4.3: Introduce the angular margin m and calculate the cosine value of the angle of the true class ​ ;

[0029] Step 4.4: Introduce a scaling factor , and calculate the final logits value , and the calculation formula is: ;

[0030] Step 4.5: Transform the logits value into a probability distribution through the Softmax function, and the transformation formula is: ;

[0031] Among them, represents the predicted probability that the model predicts the sample belongs to the true category , is the logits value of the category ;

[0032] Step 4.6: Use the cross-entropy loss function to calculate the ArcFace loss, and the specific calculation formula is: ;

[0033] Among them, represents the batch size, is the number of categories, represents the true category label of the th sample, represents the angle between the one-dimensional embedding vector of the th sample and its true label vector, while represents the angle between the input sample and the category ; is the additional angular margin parameter, is the scaling factor.

[0034] Furthermore, the calculation method of the anomaly score A in the step 5 is: ;

[0035] Among them, represents the predicted probability that the model predicts the sample belongs to the true category .

[0036] Furthermore, before the step 1, it also includes:

[0037] Transform the input one-dimensional audio signal into a time-frequency spectrogram, and the specific transformation formula is: ;

[0038] Among them, is the Log-Mel spectrogram, denotes the Mel filter bank, is a one-dimensional audio signal with the input length of , and the short-time Fourier transform (STFT) is used to convert the audio signal into a spectrogram with the dimension of , where represents the number of time frames, and represents the filtering process of the STFT spectrogram by the Mel filter bank M, where is the number of Mel filters.

[0039] Compared with the prior art, the beneficial effects of this application are as follows:

[0040] 1. The dual-path design of this application realizes the comprehensive modeling of the time-frequency spectrogram. In the frequency dimension, the frequency-domain path can efficiently extract the frequency features within each time frame; in the time dimension, the time-domain path further integrates the frequency-domain information to capture the long-range dependencies and global dynamic changes of the signal; compared with the problem of limited receptive fields in traditional two-dimensional convolutional networks, the dual-path WaveNet gradually expands the receptive field through dilated convolutions, not only retaining the local details in the time and frequency dimensions, but also effectively capturing the frequency-domain harmonic information and time-domain dynamic characteristics, thus solving the problem of insufficient modeling of complex features in the time-frequency spectrogram;

[0041] 2. In the abnormal sound detection task, this application combines the dual-path WaveNet with the self-supervised learning mechanism, enabling the model to deeply explore the feature distribution of normal sounds. In this way, without relying on a large amount of labeled data, the discriminative boundary of normal sounds can be learned through the pseudo-label classification task; in the detection stage, abnormal sounds show low classification confidence due to deviating from the normal distribution and are thus accurately identified; compared with traditional two-dimensional convolutional networks or single-path methods, the detection accuracy and robustness of this application in abnormal sound detection are significantly improved, especially in unsupervised or pseudo-supervised scenarios;

[0042] 3. This application optimizes the classification objective by introducing ArcFace Loss, making the features of normal samples more concentrated, while the distribution of abnormal samples in the feature space is more discrete, thereby improving the discriminative ability of the model for normal and abnormal samples; this characteristic enables abnormal detection to still maintain high generalization performance among different classes of normal sounds, further enhancing the applicability of the technical solution;

[0043] 4. Through the technical solution of the dual-path WaveNet, this application overcomes the key bottleneck problems in the time-frequency spectrogram feature extraction and anomaly detection of existing methods, providing an efficient and accurate solution for abnormal sound detection. Its innovation and practicality are not only reflected in the in-depth mining of complex features, but also provide important technical support for the field of unsupervised anomaly detection, with broad application value and promotion significance. Description of the Drawings

[0044] To more clearly illustrate the technical solutions of the embodiments of this application, the following will briefly introduce the drawings required in the embodiments. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as limiting the scope. For those of ordinary skill in the art, without creative efforts, other relevant drawings can also be obtained based on these drawings.

[0045] Figure 1 Schematic diagram of the unsupervised ASD method in the background technology;

[0046] Figure 2 Schematic diagram of the self-supervised abnormal sound detection method in the background technology;

[0047] Figure 3 Structural diagram of the WaveNet model in the background technology;

[0048] Figure 4 Flowchart of a self-supervised machine abnormal sound detection method based on dual-path WaveNet in this application;

[0049] Figure 5 Schematic diagram of a self-supervised machine abnormal sound detection method based on dual-path WaveNet in this application. Detailed Embodiments

[0050] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the following will clearly and completely describe the technical solutions in the embodiments of this application in conjunction with the drawings in the embodiments of this application. Obviously, the described embodiments are part of the embodiments of this application, rather than all of them.

[0051] As Figure 4 and 5 shown, a self-supervised machine abnormal sound detection method based on dual-path WaveNet includes:

[0052] Step 1: Use WaveNet in the frequency domain path to model the frequency information of each time frame of the input sound signal and extract frequency domain features;

[0053] Specifically, when the input data is a time-frequency spectrogram, and , it can be regarded as one-dimensional subsequences of length , where and represent the number of time frames and the number of frequency points respectively; subsequently, WaveNet on the frequency domain path models the frequency information in each frame, and the output is the frequency domain feature : ;

[0054] where represents the WaveNet mapping function of the frequency domain path, represents the input data has a dimension of ; the frequency domain path captures the information in the frequency dimension and extracts the key features related to anomaly detection.

[0055] Step 2: The time domain path takes the output of the frequency domain path as the input. After transposing it, it is regarded as one-dimensional sequences of length , and then, WaveNet is used for modeling in the time dimension, and the output is the time domain feature : ;

[0056] where represents the WaveNet mapping function of the time domain path, and the time domain path integrates the extracted frequency information in time.

[0057] Step 3: The outputs of the frequency domain path and the time domain path are further fused by a two-dimensional convolutional layer to generate a one-dimensional embedding vector;

[0058] Step 4: Adopt a self-supervised strategy to construct a classification task using metadata to train the model, and the classification loss function is ArcFace Loss;

[0059] The difference between the predicted probability distribution of the model and the true class distribution is measured by calculating the ArcFace loss. Among them, ArcFace (Additive Angular Margin) represents the additive angular margin.

[0060] Step 5: For the trained model, in the detection stage, the model calculates the anomaly score A through the classification probability. If the anomaly score A of the test sample exceeds the set threshold, it is determined to be abnormal; otherwise, it is judged to be normal.

[0061] Furthermore, the calculation method of the classification loss function ArcFace Loss in step 4 is:

[0062] Step 4.1: Normalize both the one-dimensional embedding vector and the class weight vector;

[0063] Step 4.2: Obtain the angular value for each class by calculating the cosine similarity between the normalized one-dimensional embedding vector and the class weight vector ;

[0064] ;

[0065] where, denotes the one-dimensional embedding vector, denotes the class weight vector, denotes the input sample and the class the angle between them;

[0066] Step 4.3: Introduce an angular margin m, and calculate the cosine value of the angle of the true class ; ;

[0067] This step can force the model to widen the margin between classes in the feature space and improve the discrimination of classes.

[0068] Step 4.4: Introduce a scaling factor , and calculate the final logits value , and the calculation formula is: ;

[0069] Step 4.5: Convert the logits value into a probability distribution through the Softmax function, and the conversion formula is: ;

[0070] where, denotes the predicted probability that the model predicts the sample belongs to the true class , is the logits value of class ;

[0071] Step 4.6: Calculate the ArcFace loss using the cross-entropy loss function, and the specific calculation formula is: ;

[0072] where, denotes the batch size, is the number of classes, denotes the true class label of the th sample, denotes the angle between the one-dimensional embedding vector of the th sample and its true label vector, while Represents the input sample The angle between and the category is the additional angular margin parameter and [[ID=]] is the scaling factor

[0073] By introducing an angular margin between categories, the embedding features of normal samples are more effectively aggregated near the average embedding vector of their respective categories, thus forming a tighter distribution. After training, in the detection stage, abnormal samples can be more clearly distinguished from normal samples because of their larger angular deviation from the average embedding vector of each category in normal samples. This angular-based feature distribution method strengthens the discriminative ability of the model, making the detection of abnormal sounds more accurate and efficient.

[0074] Furthermore, the calculation method of the abnormal score A in step 5 is as follows: ;

[0075] where represents the predicted probability that the model predicts the sample belongs to the true category

[0076] The trained model can usually accurately classify normal sounds with high confidence, while showing low confidence in unseen abnormal sounds. To quantify the degree of abnormality, the classification probability is converted into an abnormal score through negative logarithm, so that normal samples correspond to lower abnormal scores, while abnormal samples correspond to higher abnormal scores. When the abnormal score exceeds the set threshold, the sound is determined to be abnormal. Specifically, for the setting of the threshold, the abnormal scores of normal training samples can be fitted to a gamma distribution, and the point in the gamma distribution that makes the abnormal scores of 90% of the normal samples lower than this value is used as the set threshold (i.e., the 90th percentile).

[0077] Furthermore, before step 1, it also includes:

[0078] Transform the input one-dimensional audio signal into a time-frequency spectrogram. The specific transformation formula is: ;

[0079] where is the Log-Mel spectrogram represents the Mel filter bank is the one-dimensional audio signal with length Input. The short-time Fourier transform (STFT) is used to convert the audio signal into a spectrogram with a dimension of where represents the number of time frames represents the number of frequency bands represents passing through the Mel filter bank ​Perform filtering on the STFT spectrogram, where is the number of Mel filters.

[0080] The Log-Mel spectrogram can effectively capture the time and frequency information of the audio signal; the Log-Mel spectrogram performs time-frequency conversion on the audio signal through the short-time Fourier transform (STFT), then combines the Mel filter bank to compress the frequency axis, and finally the logarithmic transformation makes the features have a better dynamic range.

[0081] This application first models the frequency dimension of the time-frequency spectrogram and extracts the frequency information within each time frame; then it integrates the frequency-domain features in the time dimension, gradually extracts the time and frequency domain features, while maintaining the correlation between the time-frequency domain information. In the application of abnormal sound detection, this application uses the dual-path channel attention WaveNet model to deeply mine the feature distribution of normal sounds, so as to accurately distinguish normal and abnormal sounds, and significantly improves the accuracy and robustness of unsupervised abnormal sound detection.

[0082] The above is only the specific implementation manner of this application, but the protection scope of this application is not limited thereto. Any person skilled in the art within the technical scope disclosed by this application can easily think of changes or substitutions, which should all be covered within the protection scope of this application. Therefore, the protection scope of this application should be subject to the protection scope of the claims.

Claims

1. A self-supervised machine abnormal sound detection method based on dual-path WaveNet, characterized in that: It includes transforming the input one-dimensional audio signal into a time-spectrogram, followed by: Step 1: Use WaveNet in the frequency domain path to model the frequency information of each time frame of the input sound signal and extract frequency domain features: D s =W s (X); Where W s (·) represents the WaveNet mapping function of the frequency domain path, X∈R T×F Indicates that the dimension of the input data X is T×F, D s ∈R T×F is the output frequency domain feature; Step 2: The time domain path is the output D of the frequency domain path s As input, after transposing it, it is regarded as F one-dimensional sequences of length T. Then, WaveNet is used to model it in the time dimension, and the output is the time domain feature D g ∈R F×T , Where W t (·) WaveNet mapping function representing the time domain path; Step 3: Output D of frequency domain path and time domain path g The features are further fused through a two-dimensional convolutional layer to generate a one-dimensional embedding vector; Step 4: Adopt a self-supervised strategy and use metadata to build a classification task to train the model. The classification loss function is ArcFace Loss. Step 5: After the training model, in the detection phase, the model calculates the anomaly score A through the classification probability. If the anomaly score A of the test sample exceeds the set threshold, it is judged as abnormal; otherwise, it is judged as normal; The calculation method of the classification loss function ArcFace Loss in step 4 is: Step 4.1: Normalize the one-dimensional embedding vector and the category weight vector; Step 4.2: Obtain the angle value θ for each category by calculating the cosine similarity between the normalized one-dimensional embedding vector and the normalized category weight vector j ; Step 4.3: Introduce the angle interval m and calculate the true category y i The cosine of the angle; Step 4.4: Introduce scaling factor s and calculate the final logits value Step 4.5: Use the Softmax function to convert the logits value Convert to probability distribution; Step 4.6: Calculate ArcFace loss using the cross entropy loss function.

2. A self-supervised machine abnormal sound detection method based on dual-path WaveNet as claimed in claim 1, characterized in that: The specific formula for calculating the cosine similarity between the normalized one-dimensional embedding vector and the normalized category weight vector in step 4.2 is: Among them, X represents the normalized one-dimensional embedding vector, W j represents the normalized category weight vector, θ j Represents the angle between the input sample x and category j; In step 4.3, the true category y is calculated i The formula for the cosine of an angle is: The final logits value is calculated in step 4.4 The formula is: In step 4.5, the logits value is converted into The conversion formula to probability distribution is: Among them, P(y i |x) indicates that sample x belongs to category y i The predicted probability, z j is the logits value of category j; The specific calculation formula for calculating the ArcFace loss in step 4.6 is: Where L represents ArcFace loss, N represents batch size, n is the number of categories, and y i represents the true category label of the i-th sample, represents the angle between the one-dimensional embedding vector of the i-th sample and its true label vector, and θ j represents the angle between the input sample x and the category j, m is the additional angle margin parameter, and s is the scaling factor.

3. A self-supervised machine abnormal sound detection method based on dual-path WaveNet as described in claim 1 or 2, characterized in that: The calculation method of the anomaly score A in step 5 is: A=-logP(y i |x) Among them, P(y i |x) indicates that the model predicts that sample x belongs to the true category y i Predicted probability.

4. A self-supervised machine abnormal sound detection method based on dual-path WaveNet as described in claim 3, characterized in that: The specific transformation formula for transforming the input one-dimensional audio signal into a time-frequency spectrum is: Z=log(M·‖STFT(x)‖ 2 ); Among them, Z∈R T×F is the Log-Mel spectrum, M∈R F×B represents the Mel filter bank, x∈R L The input is a one-dimensional audio signal with a length of L. STFT represents short-time Fourier transform, and the audio signal is converted into a spectrogram with a dimension of T×B, where T represents the number of time frames, B represents the number of frequency bands, and M· represents filtering of the STFT spectrogram through a Mel filter group M, where F is the number of Mel filters.

Images