Acoustic feature dynamic extraction method based on deep neural network

Abstract

The invention relates to the technical field of acoustic dynamic extraction, in particular to an acoustic feature dynamic extraction method based on a deep neural network. The method comprises the following steps: S1, preprocessing audio data, and framing the audio data; s2, performing Fourier transform on the framed audio signal to convert the framed audio signal from a time domain signal to a frequency domain signal and obtain a spectrogram; s3, preprocessing the spectrogram, and taking the preprocessed spectrogram as the input of a deep neural network; s4, using first-order and second-order differential parameters in the deep neural network to realize dynamic feature extraction, combining the first-order and second-order differential parameters into a feature vector, and inputting the feature vector into the deep neural network; s5, outputting time change information in the audio signal according to the extracted feature sequence through a sequence labeling method; the deep neural network not only can automatically extract acoustic features, but also can capture dynamic changes of the features in a time sequence, and is beneficial to understanding of acoustic scenes and accurate classification.

Description

Technical Field

[0001] The present invention relates to the technical field of dynamic extraction of acoustic features, and in particular to a dynamic extraction method of acoustic features based on a deep neural network. Background Art

[0002] Deep neural networks extract features from local areas, reduce dimensions through pooling layers, and extract local features from speech signals. These methods can automatically learn feature expressions and improve recognition accuracy. Existing technologies only extract parameters or feature vectors that can characterize the essential characteristics of sound from original audio signals, and have certain limitations in the extraction of acoustic features, and the extraction accuracy is not high. Deep neural networks can not only automatically extract acoustic features, but also capture the dynamic changes of these features in time series, which is conducive to the understanding and accurate classification of acoustic scenes. In view of this, a method for dynamic extraction of acoustic features based on deep neural networks is provided. Summary of the invention

[0003] The purpose of the present invention is to provide a method for dynamic extraction of acoustic features based on deep neural networks to solve the problems of limitations in acoustic feature extraction, low extraction accuracy and low classification accuracy proposed in the above background technology.

[0004] To achieve the above object, the present invention provides a method for dynamic extraction of acoustic features based on a deep neural network, comprising the following steps:

[0005] S1, pre-processing the audio data and dividing the audio data into frames;

[0006] S2, performing Fourier transform on the framed audio signal to convert it from a time domain signal to a frequency domain signal and obtain a spectrum diagram;

[0007] S3, preprocessing the spectrum graph, and using the preprocessed spectrum graph as the input of the deep neural network;

[0008] S4. Using first-order and second-order differential parameters in the deep neural network to realize dynamic feature extraction, and then combining the first-order and second-order differential parameters into a feature vector and inputting it into the deep neural network;

[0009] S5. Output the time variation information of the audio signal by using the sequence labeling method using the extracted feature sequence.

[0010] 2. As a further improvement of the technical solution, in S1, the audio data needs to be preprocessed, involving the following steps:

[0011] S2.1, using an A / D converter to convert a continuous analog signal into a discrete digital signal;

[0012] S2.2, perform pre-emphasis processing to increase high-frequency components.

[0013] 3. As a further improvement of the technical solution, in S2, dividing the audio data into frames involves the following steps:

[0014] S3.1. Determine framing parameters;

[0015] S3.2, loading audio data;

[0016] S3.3, calculating the number of frames, and calculating the total number of frames according to the frame length and the total number of audio sampling points;

[0017] S3.4, frame processing, traverse the audio signal, take out a frame of data each time, save each frame of data into a matrix or array, form a two-dimensional data structure, each row represents a frame of data;

[0018] S3.5, perform overlapping processing. Except for the first and last frames, there will be overlapping parts between the frames.

[0019] 4. As a further improvement of the technical solution, in S2, the framed audio signal is subjected to Fourier transform to convert it from the time domain to the frequency domain, which involves the following steps:

[0020] S4.1. Obtain the time domain signal to be analyzed from the audio;

[0021] S4.2, converting the time domain signal into the frequency domain signal using the Fourier transform algorithm;

[0022] Among them, the main advantage of Fourier transform is that it can convert complex time domain signals into frequency domain signals that are easy to analyze. In the frequency domain, the various frequency components of the signal can be clearly displayed, which is convenient for researchers to conduct further analysis and processing. In addition, Fourier transform can also reveal the periodicity and non-periodicity characteristics of the signal, which is very important for understanding the nature of sound signals.

[0023] S4.3, extracting the amplitude spectrum, i.e. the amplitude information of the spectrum, from the Fourier transform result;

[0024] S4.4, convert the frequency axis to conventional units according to the sampling rate and Fourier transform length;

[0025] S4.5. Visualize the spectrum data as a graph.

[0026] 5. As a further improvement of the technical solution, in S4.3, the calculation formula of the Fourier transform algorithm is:

[0027] Fast Fourier Transform

[0028]

[0029] Where X(k) represents the complex value in the frequency domain, k represents the discrete frequency in the frequency domain, x(n) represents the complex value in the time domain, n represents the discrete time in the time domain, N represents the number of sampling points in the time domain, and W N is a complex root of length N and principal value 1.

[0030] 6. As a further improvement of the technical solution, in S2, performing Fourier transform on the framed audio signal to obtain a spectrum diagram involves the following steps:

[0031] S6.1. For one-dimensional signals, use fast Fourier transform; for two-dimensional image signals, use two-dimensional fast Fourier transform;

[0032] S6.2. Adjust the center of the spectrum and move the DC component to the center;

[0033] S6.3. Calculate the amplitude spectrum and phase spectrum. The amplitude spectrum is the intensity part of the spectrum and can be obtained by calculating the absolute value of the complex fast Fourier transform result. The phase spectrum reflects the phase information of each frequency component of the signal and is obtained by calculating the complex angle.

[0034] S6.4, determine the sampling frequency and signal length, calculate the value of the frequency axis; adjust the starting point and step size of the frequency axis according to whether the spectrum shift is performed;

[0035] S6.5. Draw a spectrum graph.

[0036] 7. As a further improvement of the technical solution, in S3, the spectrum graph is preprocessed and the preprocessed spectrum graph is used as the input of the deep neural network, which involves the following steps:

[0037] S7.1, adding a window to each frame in the audio data after framing;

[0038] S7.2, applying fast Fourier transform to each audio frame to obtain a spectrum;

[0039] S7.3, perform the obtained spectrum diagram;

[0040] S7.4, normalize the spectrum graph so that its data is distributed within a uniform range;

[0041] S7.5, pairing the processed spectrogram with the classification label as a training set;

[0042] S7.6. Input the preprocessed spectrogram and classification labels into the deep neural network, use the training set for multiple iterations, calculate the predicted output through forward propagation, and then update the weights through backpropagation.

[0043] 8. As a further improvement of the technical solution, in S4, first-order and second-order differential parameters and feature vectors are used in a deep neural network to extract dynamic features of the audio signal, involving the following steps:

[0044] S8.1. performing differential calculation on the continuity of each audio sample sequence, using the calculated first-order differential sequence as a supplement to the original features, and inputting the calculated first-order differential sequence into the deep neural network together with the original features;

[0045] S8.2. Perform another difference calculation on the first-order difference sequence to obtain the second-order difference sequence;

[0046] S8.3. Design the network input layer to combine the original features, first-order difference features, and second-order difference features into a new feature vector;

[0047] S8.4. Train deep neural networks using labeled datasets.

[0048] 9. As a further improvement of the technical solution, in S4, the first-order and second-order differential parameters are used for dynamic feature extraction and the original features are input into the deep neural network calculation formula as follows:

[0049] S9.1. For a time series X = [x 1 , x 2 , …, x n ], first-order difference D 1 It can be calculated by the following formula

[0050] D 1 =[x 2 -x 1 , x 3 -x 2 , …, x n -x n-1 ]

[0051] S9.2, based on the first-order difference sequence D 1 , second order difference D 2 It can be further calculated as:

[0052] D 2 =[D 1 [2]-D 1 [1], D 1 [3]-D 1 [2], …, D 1 [n-1]-D 1 [n-2]]

[0053] S9.3, the original feature X, the first-order difference feature D 1 and the second-order difference feature D 2 Concatenate them together to form an enhanced feature vector F;

[0054] S9.4. Use the constructed feature vector F as the input layer of the deep neural network.

[0055] 10. As a further improvement of the technical solution, in S5, outputting the time variation information in the audio signal by a sequence labeling method involves the following steps:

[0056] S10.1. In the output layer of the neural network model, design the output dimension;

[0057] S10.2, using the extracted feature sequence as the input of the model, preparing training targets for the labels corresponding to each frame of audio;

[0058] S10.3. Given a new audio signal, repeat the preprocessing steps and input it into the trained model. The model will output one or a group of probability distributions for each frame of audio, indicating the most likely label or state at that time point. The obtained sequence label reflects the change information of the audio signal over time.

[0059] Compared with the prior art, the present invention has the following beneficial effects:

[0060] 1. In this method of dynamic extraction of acoustic features based on deep neural networks, the neural network is used to extract features from local areas, and the dimension is reduced through the pooling layer to extract local features in the speech signal; these methods can automatically learn feature expression and improve recognition accuracy. Deep neural networks can not only automatically extract acoustic features, but also capture the dynamic changes of these features in time series.

[0061] 2. In this method of dynamic extraction of acoustic features based on deep neural networks, dynamic extraction of acoustic features is a key step in speech recognition and processing. It involves extracting features that can reflect the content of speech from speech signals and describing the changes of these features over time. Dynamic extraction of acoustic features means that when processing time series audio data, features are not only extracted from a single fixed-length audio clip, but also the feature representation of the entire audio stream that changes over time is considered. BRIEF DESCRIPTION OF THE DRAWINGS

[0062] Figure 1 The figure is a flow chart of the overall method of the present invention. DETAILED DESCRIPTION

[0063] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0064] Example:

[0065] See also Figure 1 As shown, this embodiment provides a method for dynamic extraction of acoustic features based on a deep neural network, comprising the following steps:

[0066] S1, pre-processing the audio data and dividing the audio data into frames;

[0067] In this embodiment, the audio data needs to be preprocessed, involving the following steps:

[0068] S2.1, using an A / D converter to convert a continuous analog signal into a discrete digital signal;

[0069] S2.2, perform pre-emphasis processing to increase high-frequency components;

[0070] The purpose of pre-emphasis processing is to enhance the high-frequency components in the audio signal to compensate for the high-frequency attenuation that occurs naturally during transmission or recording, and to reduce the spectrum distortion caused by the radiation effects of lips, mouth, etc., involving the following steps:

[0071] S2.21, determine the pre-emphasis coefficient a, which is a value between 0.9 and 1.0, with common values ​​such as 0.97 or 0.98. This coefficient determines the degree of high-frequency enhancement;

[0072] S2.22, design a first-order FIR (finite impulse response) high-pass filter using the selected pre-emphasis coefficient;

[0073] S2.23, split the continuous audio signal into short time frames, usually using a sliding window method, each frame covers, for example, 20 to 30 milliseconds of signal, and there is overlap (frame shift) between adjacent frames to reduce mutations caused by processing boundaries;

[0074] S2.24, applying a pre-emphasis filter to the audio data of each frame;

[0075] S2.25. After processing all frames, reassemble them into a complete audio signal.

[0076] Further, framing the audio data involves the following steps:

[0077] S3.1. Determine the framing parameters, including frame length, frame shift, and window function;

[0078] Window functions take into account the relative positions of rows when they are executed, which enables them to perform grouping, sorting, ranking, or aggregation operations while maintaining the details of each row.

[0079] S3.2, loading audio data;

[0080] S3.3, calculate the number of frames, calculate the total number of frames according to the frame length and the total number of audio sampling points, if the last frame is not enough to constitute a complete frame length, you can choose to discard, fill with zeros or other strategies;

[0081] S3.4, frame processing, traverse the audio signal, take out a frame length of data each time, apply window function processing, if the zero-filling strategy is adopted, fill the insufficient part of the last frame with zeros to reach the set frame length, save each frame of data into a matrix or array, form a two-dimensional data structure, each row represents a frame of data;

[0082] S3.5, perform overlapping processing to ensure smooth transition between consecutive frames. Except for the first and last frames, there will be overlapping parts between frames, which is determined by the frame shift;

[0083] S2, performing Fourier transform on the framed audio signal to convert it from a time domain signal to a frequency domain signal and obtain a spectrum diagram;

[0084] Among them, Fourier transform converts time domain signals into frequency domain signals, allowing us to intuitively understand the component frequencies and amplitudes of the signal, easily identify specific frequency components in the signal, and simplify the complex time domain convolution operations into multiplication operations in the frequency domain, greatly reducing the computational complexity and simplifying the understanding and processing of complex signals;

[0085] Furthermore, the calculation formula of the Fourier transform algorithm is:

[0086] Fast Fourier Transform

[0087]

[0088] Where X(k) represents the complex value in the frequency domain, k represents the discrete frequency in the frequency domain, x(n) represents the complex value in the time domain, n represents the discrete time in the time domain, N represents the number of sampling points in the time domain, and W N is a complex root of length N and principal value 1.

[0089] Among them, performing Fourier transform on the framed audio signal to obtain a spectrum diagram involves the following steps:

[0090] S6.1. For one-dimensional signals, use fast Fourier transform; for two-dimensional image signals, use two-dimensional fast Fourier transform;

[0091] S6.2. Adjust the center of the spectrum and move the DC component to the center;

[0092] S6.3. Calculate the amplitude spectrum and phase spectrum. The amplitude spectrum is the intensity part of the spectrum and can be obtained by calculating the absolute value of the complex fast Fourier transform result. The phase spectrum reflects the phase information of each frequency component of the signal and is obtained by calculating the complex angle.

[0093] The argument θ of the complex number z=a+bi (a and b are the real and imaginary parts respectively) can be calculated by the following formula:

[0094]

[0095] S6.4. Determine the sampling frequency (Fs) and signal length, and calculate the value of the frequency axis; for a one-dimensional signal, the frequency range is usually [0, Fs / 2) Hz; for a two-dimensional image, the frequency is usually expressed in the number of cycles per dimension; adjust the starting point and step size of the frequency axis according to whether the spectrum shift is performed;

[0096] Calculating the frequency axis involves the following steps:

[0097] S6.41. Determine the sampling rate Fs. The sampling rate refers to the number of samples collected per second, and the unit is usually Hz (samples per second);

[0098] S6.42, This is the number of data points used when performing the Fast Fourier Transform (FFT) transformation, which determines the frequency resolution, that is, the interval between adjacent frequency samples;

[0099] S6.43. Calculate the frequency resolution. The frequency resolution is the difference between two adjacent frequency samples in the FFT result. The calculation formula is:

[0100]

[0101] S6.44, set the frequency range, usually [0, Fs / 2) Hz;

[0102] S6.45, generating various values ​​on the frequency axis according to the frequency resolution and the frequency range;

[0103] S6.5. Draw a spectrum, usually using Python and its libraries.

[0104] S3, preprocessing the spectrum graph, and using the preprocessed spectrum graph as the input of the deep neural network;

[0105] In this embodiment, the spectrogram is preprocessed and the preprocessed spectrogram is used as the input of the deep neural network, which involves the following steps:

[0106] S7.1, adding a window to each frame in the audio data after framing to reduce spectrum leakage;

[0107] The purpose of windowing is to reduce or control the Gibbs phenomenon by applying a smooth transition region at both ends of the signal, thereby improving the quality of spectrum estimation, which can avoid truncation effects, improve frequency resolution, and reduce sidelobe leakage;

[0108] S7.2, applying fast Fourier transform to each audio frame to obtain a spectrum;

[0109] S7.3, performing logarithmic transformation on the obtained spectrum, usually taking the logarithm of each frequency component to compress the dynamic range and enhance the distinguishability in the logarithmic domain;

[0110] S7.4. In order to eliminate the intensity differences between different audios, the spectrum graph is normalized so that its data is distributed in a uniform range;

[0111] S7.5, pairing the processed spectrogram with the classification label as a training set;

[0112] S7.6. Input the preprocessed spectrogram and classification labels into the deep neural network, use the training set for multiple iterations, calculate the predicted output through forward propagation, and then update the weights through backpropagation.

[0113] S4. Using first-order and second-order differential parameters in the deep neural network to realize dynamic feature extraction, and then combining the first-order and second-order differential parameters into a feature vector and inputting it into the deep neural network;

[0114] The process of using first-order and second-order differential parameters in the neural network model to extract dynamic features is aimed at capturing the time-varying information in the audio signal. These differential features help describe the rate of change and acceleration of the audio signal over time, which is particularly important for tasks such as speech recognition and music analysis.

[0115] Among them, the first-order and second-order differential parameters are used in the deep neural network to realize dynamic feature extraction, and then the first-order and second-order differential parameters are combined into a feature vector and input into the deep neural network, involving the following steps:

[0116] S8.1. performing differential calculation on the continuity of each audio sample sequence, using the calculated first-order differential sequence as a supplement to the original features, and inputting the calculated first-order differential sequence into the deep neural network together with the original features;

[0117] S8.2. Perform another difference calculation on the first-order difference sequence to obtain the second-order difference sequence, which reflects the change in the rate of change; similarly, the second-order difference can also be used as a separate feature or in combination with the original feature and the first-order difference feature;

[0118] S8.3. Design the network input layer to combine the original features, first-order difference features, and second-order difference features into a new feature vector;

[0119] S8.4. Train deep neural networks using labeled datasets.

[0120] Furthermore, the first-order and second-order differential parameters are used for dynamic feature extraction and the original features are input into the deep neural network calculation formula as follows:

[0121] S9.1. For a time series X = [x 1 , x 2 , …, x n ], first-order difference D 1 It can be calculated by the following formula

[0122] D 1 =[x 2 -x 1 , x 3 -x 2 , …, x n -x n-1 ]

[0123] S9.2, based on the first-order difference sequence D 1 , second order difference D 2 It can be further calculated as:

[0124] D 2 =[D 1 [2]-D 1 [1], D 1 [3]-D 1 [2], …, D 1 [n-1]-D 1 [n-2]]

[0125] S9.3, the original feature X, the first-order difference feature D 1 and the second-order difference feature D 2 Concatenate them to form an enhanced feature vector F. If X is a vector of length n, then the length of F will be 2n-1 or 2n-2;

[0126] S9.4. Use the constructed feature vector F as the input layer of the deep neural network.

[0127] S5, outputting the time variation information of the audio signal by using a sequence labeling method for the extracted feature sequence;

[0128] The purpose of sequence labeling is to automatically extract structured information from natural language text. It involves assigning a predefined category label to each element in the input sequence (usually a word or character in the text). This process is a typical sequence-to-sequence mapping problem, where the input is a sequence (such as a sentence) and the output is another sequence of the same length as the input sequence, where the label at each position represents the classification information of the corresponding input element.

[0129] Among them, outputting the time change information in the audio signal by the sequence labeling method involves the following steps:

[0130] S10.1. In the output layer of the neural network model, design the output dimensions; the dimensions of the output layer may be the same as the vocabulary size, with each dimension corresponding to a possible character or subword;

[0131] S10.2. Use the extracted feature sequence as the input of the model, prepare training targets for the labels corresponding to each frame of audio, use the cross entropy loss function for training, and use a large amount of labeled audio data to train the model so that the model can learn the mapping from audio features to sequence labels;

[0132] Among them, the cross entropy loss function is a method to measure the difference between the model's predicted probability distribution and the actual label distribution, involving the following steps:

[0133] S10.21. Make sure each sample has a corresponding label, usually 0 or 1, representing two different categories; the model will output a value between 0 and 1, indicating the probability of belonging to a certain category;

[0134] S10.22. Use the Sigma ID function to convert the model output into probability. The formula is:

[0135]

[0136] Where z is the output of the model without the activation function;

[0137] S10.23, the cross entropy loss calculation formula is:

[0138] L=-[y·log(p)+(1-y)·log(1-p)]

[0139] Where y is the true label (0 or 1) and p is the predicted probability;

[0140] S10.24. If it is batch data, the losses of all samples are usually summed or averaged to get the total loss;

[0141] S10.3. Given a new audio signal, repeat the preprocessing steps and input it into the trained model. The model will output one or a group of probability distributions for each frame of audio, indicating the most likely label or state at that time point. The obtained sequence label reflects the change information of the audio signal over time. This will be a series of characters or subwords that show the change in the content of the speech.

[0142] The above shows and describes the basic principles, main features and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited by the above embodiments. The above embodiments and descriptions are only preferred examples of the present invention and are not intended to limit the present invention. Without departing from the spirit and scope of the present invention, the present invention may have various changes and improvements, which fall within the scope of the present invention. The scope of protection of the present invention is defined by the attached claims and their equivalents.

Claims

1. A method for dynamic extraction of acoustic features based on deep neural network, characterized in that: The following steps are involved: S1, pre-processing the audio data and dividing the audio data into frames; S2, performing Fourier transform on the framed audio signal to convert it from a time domain signal to a frequency domain signal and obtain a spectrum diagram; S3, preprocessing the spectrum graph, and using the preprocessed spectrum graph as the input of the deep neural network; S4. Using first-order and second-order differential parameters in the deep neural network to realize dynamic feature extraction, and then combining the first-order and second-order differential parameters into a feature vector and inputting it into the deep neural network; S5. Output the time variation information of the audio signal by using the sequence labeling method using the extracted feature sequence.

2. The method for dynamic extraction of acoustic features based on a deep neural network according to claim 1, characterized in that: In S1, the audio data needs to be preprocessed, involving the following steps: S2.1, using an A / D converter to convert a continuous analog signal into a discrete digital signal; S2.2, perform pre-emphasis processing to increase high-frequency components.

3. The method for dynamic extraction of acoustic features based on a deep neural network according to claim 1, characterized in that: In S2, framing the audio data involves the following steps: S3.

1. Determine framing parameters; S3.2, loading audio data; S3.3, calculating the number of frames, and calculating the total number of frames according to the frame length and the total number of audio sampling points; S3.4, frame processing, traverse the audio signal, take out a frame of data each time, save each frame of data into a matrix or array, form a two-dimensional data structure, each row represents a frame of data; S3.5, perform overlapping processing. Except for the first and last frames, there will be overlapping parts between the frames.

4. The method for dynamic extraction of acoustic features based on a deep neural network according to claim 1, characterized in that: In S2, the framed audio signal is subjected to Fourier transform to convert it from the time domain to the frequency domain, which involves the following steps: S4.

1. Obtain the time domain signal to be analyzed from the audio; S4.2, converting the time domain signal into the frequency domain signal using the Fourier transform algorithm; S4.3, extracting the amplitude spectrum, i.e. the amplitude information of the spectrum, from the Fourier transform result; S4.4, convert the frequency axis to conventional units according to the sampling rate and Fourier transform length; S4.

5. Visualize the spectrum data as a graph.

5. The method for dynamic extraction of acoustic features based on a deep neural network according to claim 4, characterized in that: In S4.3, the calculation formula of the Fourier transform algorithm is: Fast Fourier Transform Where X(k) represents the complex value in the frequency domain, k represents the discrete frequency in the frequency domain, x(n) represents the complex value in the time domain, n represents the discrete time in the time domain, N represents the number of sampling points in the time domain, and W N is a complex root of length N and principal value 1.

6. The method for dynamic extraction of acoustic features based on a deep neural network according to claim 5, characterized in that: In S2, performing Fourier transform on the framed audio signal to obtain a spectrum diagram involves the following steps: S6.

1. For one-dimensional signals, use fast Fourier transform; for two-dimensional image signals, use two-dimensional fast Fourier transform; S6.

2. Adjust the center of the spectrum graph and move the DC component to the center; S6.

3. Calculate the amplitude spectrum and phase spectrum. The amplitude spectrum is the intensity part of the spectrum and can be obtained by calculating the absolute value of the complex fast Fourier transform result. The phase spectrum reflects the phase information of each frequency component of the signal and is obtained by calculating the complex angle. S6.

4. Determine the sampling frequency and signal length, calculate the value of the frequency axis; adjust the starting point and step size of the frequency axis according to whether the spectrum shift is performed; S6.

5. Draw a spectrum graph.

7. The method for dynamic extraction of acoustic features based on a deep neural network according to claim 1, characterized in that: In S3, the spectrum graph is preprocessed and used as the input of the deep neural network after the preprocessing is completed, involving the following steps: S7.1, preprocessing the spectrogram, and adding a window to each frame of the audio data after framing; S7.2, applying fast Fourier transform to each audio frame to obtain a spectrum; S7.3, perform the obtained spectrum diagram; S7.4, normalize the spectrum graph so that its data is distributed within a uniform range; S7.5, pairing the processed spectrogram with the classification label as a training set; S7.

6. Input the preprocessed spectrogram and classification labels into the deep neural network, use the training set for multiple iterations, calculate the predicted output through forward propagation, and then update the weights through backpropagation.

8. The method for dynamic extraction of acoustic features based on a deep neural network according to claim 1, characterized in that: In S4, first-order and second-order differential parameters are used in the deep neural network to realize dynamic feature extraction, and the first-order and second-order differential parameters are combined into a feature vector and input into the deep neural network, involving the following steps: S8.

1. performing differential calculation on the continuity of each audio sample sequence, using the calculated first-order differential sequence as a supplement to the original features, and inputting the calculated first-order differential sequence into the deep neural network together with the original features; S8.

2. Perform another difference calculation on the first-order difference sequence to obtain the second-order difference sequence; S8.

3. Design the network input layer to combine the original features, first-order difference features, and second-order difference features into a new feature vector; S8.

4. Train deep neural networks using labeled datasets.

9. The method for dynamic extraction of acoustic features based on a deep neural network according to claim 1, characterized in that: In S4, the first-order and second-order differential parameters are used for dynamic feature extraction and the original features are input into the deep neural network calculation formula: S9.

1. For a time series X = [x1, x2, ..., x n ], the first-order difference D1 can be calculated by the following formula D1=[x2-x1,x3-x2,…,x n -x n-1 ] S9.

2. Based on the first-order difference sequence D1, the second-order difference D2 can be further calculated as: D2=[D1[2]-D1[1], D1[3]-D1[2],…, D1[n-1]-D1[n-2]] S9.3, concatenate the original feature X, the first-order difference feature D1 and the second-order difference feature D2 to form an enhanced feature vector F, S9.

4. Use the constructed feature vector F as the input layer of the deep neural network.

10. The method for dynamic extraction of acoustic features based on a deep neural network according to claim 1, characterized in that: In S5, outputting the time variation information in the audio signal by a sequence labeling method involves the following steps: S10.

1. In the output layer of the neural network model, design the output dimension; S10.2, using the extracted feature sequence as the input of the model, preparing training targets for the labels corresponding to each frame of audio; S10.

3. Given a new audio signal, repeat the preprocessing steps and input it into the trained model. The model will output one or a group of probability distributions for each frame of audio, indicating the most likely label or state at that time point. The obtained sequence label reflects the change information of the audio signal over time.

Images