An audio early warning accurate identification method based on mixed features

Abstract

This invention discloses a method for accurate audio warning identification based on hybrid features. This method analyzes collected audio speech to determine the issuance time of the warning signal, thereby accurately evaluating the timeliness of the audio warning. First, a double noise reduction method using logmmse-spectral subtraction is employed to filter out noise information in the recorded speech. Next, endpoint detection based on short-time energy is used to mark the effective speech segments in the test speech. Then, MFCC features and waveform polynomial features are extracted from each frame of the effective speech segment. Subsequently, the two features are used as inputs to two channels of a convolutional neural network, and the outputs of the two channels are summed to obtain the hybrid features. Finally, the hybrid features are used as input to a softmax function, and the speech segment containing the target speech (audio warning signal) is determined by the maximum probability value. The starting position of this speech segment is the issuance time of the warning signal.

Description

Technical Field

[0001] This invention relates to a method for accurate identification of audio warnings based on hybrid features, belonging to the field of intelligent driving evaluation. Background Technology

[0002] Because the human auditory system is inherently more sensitive to low-frequency information and less sensitive to high-frequency information, MFCC features are the most commonly used features in traditional speech recognition. However, since MFCC features use a discrete cosine transform (DCT) step to reduce and abstract the data, the correlation between features is weakened. Deep learning algorithms, however, require correlation between features. Because Mel filters overlap, the energy values ​​they obtain are correlated. An nth-order polynomial is used to fit the energy spectrum of the Fourier transform of each frame of speech, and the resulting polynomial features are fused with the MFCC features to strengthen the correlation between features.

[0003] The purpose of this invention is to improve the selection of audio features and neural networks, providing an efficient and accurate method for evaluating audio warning signals. It utilizes a microphone sensor to acquire the basic data required for evaluation, processes the data with high precision, and analyzes the data to obtain the evaluation results. The evaluation primarily determines whether the system can issue audio warning messages in a timely and accurate manner. Summary of the Invention

[0004] This invention discloses a method for accurate audio warning identification based on hybrid features. This method analyzes collected audio speech to determine the issuance time of the warning signal, thereby accurately evaluating the timeliness of the audio warning. First, a dual noise reduction method using logmmse-spectral subtraction is employed to filter out noise information in the recorded speech. Next, endpoint detection based on short-time energy is used to identify valid speech segments in the test speech. Then, MFCC features and waveform polynomial features are extracted from each frame of the valid speech segment. Subsequently, these two features are used as inputs to two channels of a convolutional neural network, and the outputs of the two channels are summed to obtain the hybrid features. Finally, the hybrid features are used as input to a softmax function, and the speech segment containing the target speech (audio warning signal) is determined by the maximum probability value. The starting position of this speech segment is the issuance time of the warning signal.

[0005] The specific steps include:

[0006] Step 1: Perform dual noise reduction using logmmse-spectral subtraction.

[0007] The method of combining logmmse with spectral subtraction can be used to reduce noise for both short-term unstable noise and relatively regular stable noise.

[0008] LogMMSE is a statistical speech enhancement algorithm. After segmenting and windowing the audio, it first estimates noise parameters. Based on the characteristics of the acquired audio, it is assumed that the first n... noise The frame is a noisy signal. The first n... noise After the frame signal undergoes a Fourier transform, the real part is taken to obtain the amplitude spectrum s of each frame signal. The energy of the noise amplitude spectrum is calculated as shown in equation (1):

[0009]

[0010] Then, for each frame of speech x k Enhancement is performed. The amplitude spectrum s of each frame of speech is obtained by taking the real part of its Fourier transform. k The amplitude spectral energy, prior signal-to-noise ratio (SNR), and posterior signal-to-noise ratio (SNR) of the speech frame are calculated as shown in equations (2), (3), and (4), respectively.

[0011] p k (i)=[s k (i)] 2 (2)

[0012]

[0013]

[0014] Based on the above three parameters, the parameters for the decision-guided method are calculated as follows:

[0015] ξ k (i)=aγ pre (i)+(1-a)max(γ k (i),0) (5)

[0016] ξ k (i)=max(ξ k (i),ξ min (6)

[0017] Therefore, the enhancement coefficient of the speech frame is calculated:

[0018]

[0019]

[0020] Each sampling point s of the amplitude spectrum of the original speech k (i) Multiplying by the corresponding enhancement coefficient enh(i) and then performing an inverse Fourier transform yields the enhancement result for that frame of speech. In addition, the noise energy needs to be updated after each frame of speech has been calculated.

[0021] σ k (i)=vk (i)-ln(ξ k (i)) (9)

[0022]

[0023] If parameter d is less than the set threshold, then the noise energy is updated:

[0024] p noise (i)=μp noise (i)+(1-μ)p k (i) (11)

[0025] After logmmse denoising, spectral subtraction is applied to further reduce noise in the audio. Spectral subtraction can effectively remove relatively stable noise. Similarly, after segmenting and windowing the speech, noise parameters are estimated. The first n... noise After the frame signal undergoes a Fourier transform and its modulus is taken, the energy of the noise amplitude spectrum is calculated as follows:

[0026]

[0027]

[0028]

[0029]

[0030] Performing an inverse Fourier transform on the spectral subtraction-derived speech yields the denoised speech. Thus, the logmmse-spectral subtraction dual denoising is complete.

[0031] Step 2: Speech start-endpoint detection based on short-time energy

[0032] An endpoint detection method based on short-time energy in the time domain is adopted. Furthermore, to improve the accuracy of the speech start time results, the zero-crossing rate property, which usually interacts with short-time energy, is discarded. Short-time energy is defined as:

[0033]

[0034] In the formula, N represents the number of frames, and m represents the number of samples in each frame.

[0035] The threshold is set using the short-time energy of each frame of speech. The threshold setting method used in this experiment is as follows:

[0036]

[0037]

[0038] Speech endpoint detection is performed by iterating through the short-time energy of each frame of audio based on a set threshold. Specifically, the short-time energy is iterated, and points with energy greater than the threshold h are added as candidate speech points. Points with energy less than the threshold h are considered as the endpoints of the speech segment. For each newly added candidate starting point, its distance to the previous endpoint is compared. If the distance is less than x frames, the point is not considered as the starting point of the new speech segment, and it is merged with the previous speech segment to continue searching for the endpoint. After detection, the results of the first endpoint detection are expanded using a threshold l: for each detection result, expansion is performed from the left and right endpoints outwards, stopping when an energy less than the threshold l is encountered. The final result of speech endpoint detection is obtained.

[0039] Step 3: Extract speech mixing features

[0040] The steps for calculating MFCC are as follows:

[0041] (1) Pre-emphasize the sound signal using the following formula:

[0042] x'(n) = x(n) - μ·x(n-1) (19)

[0043] (2) Select appropriate frame size and frame shift to frame the speech signal. Assume that each frame of speech is represented as x after framing. k .

[0044] (3) After framing, a window is applied to each frame of signal, using a Hamming window:

[0045] x k (n)=x k (n)×W(n)(20)

[0046] a is set to 0.46. (21)

[0047] (4) Perform a discrete Fourier transform on each frame of speech:

[0048]

[0049] (5) Define a filter bank with M filters, using triangular filters; the filter definitions are as follows:

[0050]

[0051] f(m) is defined as:

[0052]

[0053] F mel (f) = 1125ln(1+f / 700)

[0054]

[0055] In the formula f l The lowest frequency defined for the filter, f h fs is the highest frequency defined for the filter, and fs is the sampling frequency;

[0056] Pass the energy spectrum through the aforementioned triangular filter bank and calculate the logarithmic energy of the output of each filter bank:

[0057]

[0058] (6) Substitute the above logarithmic energy into the discrete cosine transform to obtain the L-order Mel-Cepstral parameters, where L refers to the order of the MFCC coefficients:

[0059]

[0060] (7) Calculate the dynamic difference parameters as needed, using the following formula:

[0061]

[0062] After obtaining the first-order difference parameters, the second-order difference parameters can be obtained by substituting the first-order difference parameters into the equation. Θ can be either 1 or 2.

[0063] The first 13 coefficients of the Mel-Cepstral parameter and the first-order difference parameter are taken to obtain the MFCC feature vector of the speech frame.

[0064] The polynomial feature is the coefficients obtained by fitting an nth-order polynomial to the energy spectrum of each frame of speech after Fourier transform. Assume there is an nth-order polynomial:

[0065] p n =a n x n +a n-1 x n-1 +...+a1x+a0 (28)

[0066] After performing a Fourier transform on each frame of speech, the square of its modulus is calculated to obtain the energy spectrum. Energy is used as the function value, and the corresponding frequency is used as the independent variable against p. n By performing a fitting, the coefficients (a) are obtained. n ,a n-1 The polynomial features of the speech frame are represented by (a1, a0, ..., a1, a0).

[0067] Step 4: Obtain hybrid features based on dual-channel CNN

[0068] The MFCC and multinomial features obtained in step 3 are used as inputs to two channels of the CNN, respectively, and cross-correlation is performed with the convolution kernels of those channels. The outputs of the two channels are then summed to obtain the mixed features.

[0069] The input layer size is set to 224, and the convolutional layers use small 3×3 filters with a stride of 1. Zero-padding is applied to the input data to prevent the convolutional layers from altering the spatial dimensions of the input data. A 2×2 receptive field with a stride of 2 is used in the pooling layers. The size of the convolutional kernel defines its receptive field; a larger receptive field utilizes a larger neighborhood of pixels, extracting more information and resulting in better features. However, a larger receptive field also increases computational cost, leading to excessively long training times in deep networks.

[0070] Step 5: Use the softmax function to recognize voice warning signals.

[0071] The mixed feature vector obtained in step 4 is used as the input to the softmax function. Let the mixed feature be 'a', and the number of outputs be C, i.e., the inputs are a1, a2, a3, ..., aC. For each frame of speech, the probability that it belongs to class i is:

[0072]

[0073] The softmax function normalizes the input mixed feature vector, highlighting the value with the highest probability and suppressing other components with much lower probabilities, thus obtaining the probability distribution of each frame of speech. The segment with the highest probability value is the target speech segment, and the starting position of this speech segment is the time when the audio warning is issued.

[0074] The beneficial effects of this invention are as follows: This invention improves the selection of audio features, and at the same time utilizes the internal MFCC features and external multinomial features of audio combined with a convolutional neural network to identify warning audio, and can determine the time of issuance of the warning signal, thereby accurately evaluating the timeliness of audio warnings. Attached Figure Description

[0075] Figure 1 This is the overall flowchart for audio warning recognition;

[0076] Figure 2 This is an endpoint detection flowchart;

[0077] Figure 3 This is a flowchart of the hybrid feature extraction process;

[0078] Figure 4 This is a diagram of the CNN architecture. Detailed Implementation

[0079] Traditional neural networks used in image recognition methods have limitations. Unpacking an image into a vector results in the loss of spatial information; neighboring pixels in the same column may be far apart in the vector, making the patterns they form difficult for the model to recognize. Secondly, for large input images, the excessive number of parameters makes using fully connected layers prone to leading to an overly large model, inefficiency, and training difficulties. A large number of parameters can also cause overfitting.

[0080] Convolutional Neural Networks (CNNs) are an improvement on traditional neural networks. They have changed in the form and function of layers, and have adopted local connections and shared weights. On the one hand, this reduces the number of weights, making the network easier to optimize, and on the other hand, it reduces the risk of overfitting.

[0081] Because the human auditory system is inherently more sensitive to low-frequency information and less sensitive to high-frequency information, MFCC features are the most commonly used features in traditional speech recognition. However, since MFCC features use a discrete cosine transform (DCT) step to reduce and abstract the data, the correlation between features is weakened. Deep learning algorithms, however, require correlation between features. Because Mel filters overlap, the energy values ​​they obtain are correlated. An nth-order polynomial is used to fit the energy spectrum of the Fourier transform of each frame of speech, and the resulting polynomial features are fused with the MFCC features to strengthen the correlation between features.

[0082] The purpose of this invention is to improve the selection of audio features and neural networks, providing an efficient and accurate method for evaluating audio warning signals. It utilizes a microphone sensor to acquire the basic data required for evaluation, processes the data with high precision, and analyzes the data to obtain the evaluation results. The evaluation primarily determines whether the system can issue audio warning messages in a timely and accurate manner.

[0083] The collected audio data is analyzed to determine the issuance time of the warning signal, thereby accurately evaluating the timeliness of the audio warning. First, a dual noise reduction method (logmmse-spectral subtraction) is used to filter out noise information from the recorded speech. Next, endpoint detection based on short-time energy is used to identify the effective speech segments in the test speech. Then, MFCC features and waveform polynomial features are extracted from each frame of the effective speech segment. Subsequently, these two features are used as inputs to two channels of a convolutional neural network, and the outputs of the two channels are summed to obtain a mixed feature. Finally, the mixed feature is used as input to a softmax function, and the speech segment containing the target speech (audio warning signal) is determined by the maximum probability value. The starting position of this speech segment is the issuance time of the warning signal.

[0084] The specific steps include:

[0085] Step 1: Perform dual noise reduction using logmmse-spectral subtraction.

[0086] The method of combining logmmse with spectral subtraction can be used to reduce noise for both short-term unstable noise and relatively regular stable noise.

[0087] LogMMSE is a statistical speech enhancement algorithm. After segmenting and windowing the audio, it first estimates noise parameters. Based on the characteristics of the acquired audio, it is assumed that the first n... noise The frame is a noisy signal. The first n... noise After the frame signal undergoes a Fourier transform, the real part is taken to obtain the amplitude spectrum s of each frame signal. The energy of the noise amplitude spectrum is calculated as shown in equation (1):

[0088]

[0089] Then, for each frame of speech x k Enhancement is performed. The amplitude spectrum s of each frame of speech is obtained by taking the real part of its Fourier transform. k The amplitude spectral energy, prior signal-to-noise ratio (SNR), and posterior signal-to-noise ratio (SNR) of the speech frame are calculated as shown in equations (2), (3), and (4), respectively.

[0090] p k (i)=[s k (i)] 2 (2)

[0091]

[0092]

[0093] Based on the above three parameters, the parameters for the decision-guided method are calculated as follows:

[0094] ξ k (i)=aγ pre (i)+(1-a)max(γ k (i),0) (5)

[0095] ξ k (i)=max(ξ k (i),ξ min (6)

[0096] The enhancement coefficient of the speech frame is then calculated:

[0097]

[0098]

[0099] Each sampling point s of the amplitude spectrum of the original speech k(i) Multiplying by the corresponding enhancement coefficient enh(i) and then performing an inverse Fourier transform yields the enhancement result for that frame of speech. In addition, the noise energy needs to be updated after each frame of speech has been calculated.

[0100] σ k (i)=v k (i)-ln(ξ k (i)) (9)

[0101]

[0102] If parameter d is less than the set threshold, then the noise energy is updated:

[0103] p noise (i)=μp noise (i)+(1-μ)p k (i) (11)

[0104] After logmmse denoising, spectral subtraction is applied to further reduce noise in the audio. Spectral subtraction can effectively remove relatively stable noise. Similarly, after segmenting and windowing the speech, noise parameters are estimated. The first n... noise After the frame signal undergoes a Fourier transform and its modulus is taken, the energy of the noise amplitude spectrum is calculated as follows:

[0105]

[0106]

[0107]

[0108]

[0109] Performing an inverse Fourier transform on the spectral subtraction-derived speech yields the denoised speech. Thus, the logmmse-spectral subtraction dual denoising is complete.

[0110] Step 2: Speech start-endpoint detection based on short-time energy

[0111] An endpoint detection method based on short-time energy in the time domain is adopted. Furthermore, to improve the accuracy of the speech start time results, the zero-crossing rate property, which usually interacts with short-time energy, is discarded. Short-time energy is defined as:

[0112]

[0113] In the formula, N represents the number of frames, and m represents the number of samples in each frame.

[0114] The threshold is set using the short-time energy of each frame of speech. The threshold setting method used in this experiment is as follows:

[0115]

[0116]

[0117] Speech endpoint detection is performed by iterating through the short-time energy of each frame of audio based on a set threshold. Specifically, the short-time energy is iterated, and points with energy greater than the threshold h are added as candidate speech points. Points with energy less than the threshold h are considered as the endpoints of the speech segment. For each newly added candidate starting point, its distance to the previous endpoint is compared. If the distance is less than x frames, the point is not considered as the starting point of the new speech segment, and it is merged with the previous speech segment to continue searching for the endpoint. After detection, the results of the first endpoint detection are expanded using a threshold l: for each detection result, expansion is performed from the left and right endpoints outwards, stopping when an energy less than the threshold l is encountered. The final result of speech endpoint detection is obtained.

[0118] Step 3: Extract speech MFCC features and multinomial features

[0119] The steps for calculating MFCC are as follows:

[0120] (8) Pre-emphasize the sound signal using the following formula:

[0121] x'(n) = x(n) - μ·x(n-1) (19)

[0122] (9) Select appropriate frame size and frame shift to frame the speech signal. Assume that each frame of speech is represented as x after framing. k .

[0123] (10) After framing, a window is applied to each frame of signal, using a Hamming window:

[0124] x k (n)=x k (n)×W(n)(20)

[0125] a is set to 0.46. (21)

[0126] (11) Perform a discrete Fourier transform on each frame of speech:

[0127]

[0128] (12) Define a filter bank with M filters, using triangular filters; the filter definitions are as follows:

[0129]

[0130] f(m) is defined as:

[0131]

[0132] F mel (f) = 1125ln(1+f / 700)

[0133]

[0134] In the formula f l The lowest frequency defined for the filter, f h fs is the highest frequency defined for the filter, and fs is the sampling frequency;

[0135] Pass the energy spectrum through the aforementioned triangular filter bank and calculate the logarithmic energy of the output of each filter bank:

[0136]

[0137] (13) Substitute the above logarithmic energy into the discrete cosine transform to obtain the L-order Mel-Cepstral parameters, where L refers to the order of the MFCC coefficients:

[0138]

[0139] (14) Calculate the dynamic difference parameters as needed, using the following formula:

[0140]

[0141] After obtaining the first-order difference parameters, the second-order difference parameters can be obtained by substituting the first-order difference parameters into the equation. Θ can be either 1 or 2.

[0142] The first 13 coefficients of the Mel-Cepstral parameter and the first-order difference parameter are taken to obtain the MFCC feature vector of the speech frame.

[0143] The polynomial feature is the coefficients obtained by fitting an nth-order polynomial to the energy spectrum of each frame of speech after Fourier transform. Assume there is an nth-order polynomial:

[0144] p n =a n x n +a n-1 x n-1 +...+a1x+a0 (28)

[0145] After performing a Fourier transform on each frame of speech, the square of its modulus is calculated to obtain the energy spectrum. Energy is used as the function value, and the corresponding frequency is used as the independent variable against p. n By performing a fitting, the coefficients (a) are obtained. n ,a n-1 The polynomial features of the speech frame are represented by (a1, a0, ..., a1, a0).

[0146] Step 4: Obtain hybrid features based on dual-channel CNN

[0147] The MFCC and multinomial features obtained in step 3 are used as inputs to two channels of the CNN, respectively, and cross-correlation is performed with the convolution kernels of those channels. The outputs of the two channels are then summed to obtain the mixed features.

[0148] The input layer size is set to 224, and a small 3×3 filter is used for the convolutional layer with a stride of 1. Zero-padding is applied to the input data to prevent the convolutional layers from altering the spatial dimensions of the input data. A 2×2 receptive field with a stride of 2 is used in the pooling layer. The size of the convolutional kernel defines its receptive field; a larger receptive field utilizes a larger neighborhood of pixels, extracting more information and resulting in better features. However, a larger receptive field also increases computational cost, leading to excessively long training times in deep networks.

[0149] Step 5: Use the softmax function to recognize voice warning signals.

[0150] The mixed feature vector obtained in step 4 is used as the input to the softmax function. Let the mixed feature be 'a', and the number of outputs be C, i.e., the inputs are a1, a2, a3, ..., aC. For each frame of speech, the probability that it belongs to class i is:

[0151]

[0152] The softmax function normalizes the input mixed feature vector, highlighting the value with the highest probability and suppressing other components with much lower probabilities, thus obtaining the probability distribution of each frame of speech. The segment with the highest probability value is the target speech segment, and the starting position of this speech segment is the time when the audio warning is issued.

Claims

1. A method for accurate identification of audio warnings based on hybrid features, characterized in that, Includes the following steps: Step 1: Perform dual noise reduction using logmmse-spectral subtraction. A method combining logmmse and spectral subtraction is used to reduce noise for both short-term unstable noise and relatively regular stable noise. LogMMSE is a statistical speech enhancement algorithm. After segmenting and windowing the audio, it first estimates noise parameters. Based on the characteristics of the acquired audio, it assumes that the first n... noise The frame is a noisy signal, so the first n... noise After the frame signal undergoes a Fourier transform, the real part is taken to obtain the amplitude spectrum s of each frame signal. The energy of the noise amplitude spectrum is calculated as shown in equation (1): Then, for each frame of speech x k To enhance the signal, the amplitude spectrum s of each frame is obtained by taking the real part of its Fourier transform. k The amplitude spectral energy, prior signal-to-noise ratio (SNR), and posterior signal-to-noise ratio (SNR) of the speech frame are calculated as shown in equations (2), (3), and (4), respectively. p k (i)=[s k (i)] 2 (2) Based on the above three parameters, the parameters for the decision-guided method are calculated as follows: x k (i)=aγ pre (i)+(1-a)max(γ k (i),0) (5) x k (i)=max(ξ k (i),ξ min ) (6) Therefore, the enhancement coefficient of the speech frame is calculated: Each sampling point s of the amplitude spectrum of the original speech k (i) Multiply by the corresponding enhancement coefficient enh(i), and then perform an inverse Fourier transform to obtain the enhancement result of the speech frame. In addition, the noise energy needs to be updated after each speech frame is calculated: σ k (i)=v k (i)-ln(ξ k (i)) (9) If parameter d is less than the set threshold, then the noise energy is updated: p noise (i)=μp noise (i)+(1-μ)p k (i) (11) After logmmse denoising, spectral subtraction is applied to further reduce noise in the audio. Spectral subtraction can effectively remove relatively stable noise. Similarly, after segmenting and windowing the speech, noise parameters are estimated, and the first n... noise After the frame signal undergoes a Fourier transform and its modulus is taken, the energy of the noise amplitude spectrum is calculated as follows: Performing an inverse Fourier transform on the spectral subtraction speech yields the denoised speech, thus completing the logmmse-spectral subtraction dual denoising process. Step 2: Speech start-endpoint detection based on short-time energy An endpoint detection method based on short-time energy in the time domain is adopted. Furthermore, to improve the accuracy of the results at the speech start time, the zero-crossing rate attribute, which usually interacts with short-time energy, is discarded. Short-time energy is defined as: In the formula, N represents the number of frames, and m represents the number of samples in each frame; A threshold is set using the short-time energy of each frame of speech. The threshold setting method is as follows: Speech endpoint detection is performed by iterating through the short-time energy of each frame of audio according to the set threshold. Specifically, the short-time energy is iterated, and points with energy greater than the threshold h are added as the starting point of the speech and added as candidate speech. Points with energy less than the threshold h are added as the ending point of the speech segment. For newly added candidate starting points, the distance between them and the previous ending point is compared. If the distance between them is less than x frames, the point is not used as the starting point of the new speech segment. Instead, it is merged with the previous speech segment and the search for the ending point continues. After the detection is completed, the results of the first endpoint detection are expanded using a threshold l: for each detection result, the expansion is performed from the left and right endpoints outwards, stopping when the energy is less than the threshold l; thus obtaining the final result of the speech endpoint detection. Step 3: Extract MFCC-polynomial mixture features The steps for calculating MFCC are as follows: (1) Pre-emphasize the sound signal using the following formula: x'(n)=x(n)-μ·x(n-1) (19) (2) Select appropriate frame size and frame shift to frame the speech signal. Assume that each frame of speech is represented as x after framing. k ; (3) After framing, a window is applied to each frame of signal, using a Hamming window: x k (n)=x k (n)×W(n) (20) 0≤n≤N-1, a is 0.46(21) (4) Perform a discrete Fourier transform on each frame of speech: (5) Define a filter bank with M filters, using triangular filters; the filter definitions are as follows: f(m) is defined as: F mel (f)=1125ln(1+f / 700) In the formula f l The lowest frequency defined for the filter, f h fs is the highest frequency defined for the filter, and fs is the sampling frequency; Pass the energy spectrum through the aforementioned triangular filter bank and calculate the logarithmic energy of the output of each filter bank: (6) Substitute the above logarithmic energy into the discrete cosine transform to obtain the L-order Mel-Cepstral parameters, where L refers to the order of the MFCC coefficients: (7) Calculate the dynamic difference parameters as needed, using the following formula: After obtaining the first-order difference parameters, substitute the first-order difference parameters into the equation to obtain the second-order difference parameters, where Θ is 1 or 2. The first 13 coefficients of the Mel-Cepstral parameter and the first-order difference parameter are taken to obtain the MFCC feature vector of the speech frame. The polynomial feature is the coefficients obtained by fitting an nth-order polynomial to the energy spectrum of each frame of speech after Fourier transform. Assume there is an nth-order polynomial: p n =a n x n +a n-1 x n-1 +...+a1x+a0 (28) After performing a Fourier transform on each frame of speech, the square of its modulus is calculated to obtain the energy spectrum. Energy is used as the function value, and the corresponding frequency is used as the independent variable against p. n By performing a fitting, the coefficients (a) are obtained. n ,a n-1 ,...,a1,a0) are the polynomial features of the speech in this frame; Step 4: Obtain fused features based on dual-channel CNN The MFCC features and multinomial features obtained in step 3 are used as inputs to two channels of the CNN, respectively. They are cross-correlated with the convolution kernels of the channels, and the outputs of the two channels are added together to obtain the fused features. The input layer size is set to 224, and a small 3×3 filter is used for the convolutional layer with a stride of 1. Zero padding is applied to the input data so that the convolutional layer does not change the spatial dimension of the input data. A 2×2 receptive field with a stride of 2 is used in the pooling layer. The size of the convolutional kernel defines the receptive field of the kernel. The larger the receptive field, the larger the neighborhood of the pixels used, the more information can be extracted, and the better the extracted features. However, increasing the receptive field also means increasing the computational cost, which can lead to excessively long training time in deep network training. Step 5: Use the softmax function to recognize voice warning signals. The mixed feature vector obtained in step 4 is used as the input to the softmax function. Let the mixed feature be a, and the number of outputs be C, i.e., the inputs are a1, a2, a3, ..., aC. For each frame of speech, the probability that it belongs to class i is: The softmax function normalizes the input mixed feature vector, highlighting the value with the highest probability and suppressing other components that are much lower than the maximum value, thus obtaining the probability distribution of each frame of speech; the segment with the highest probability value is the target speech segment, and the starting position of this speech segment is the time when the audio warning is issued.

Images