An audio warning intelligent recognition method for intelligent driving evaluation

Abstract

The present invention discloses an audio warning intelligent recognition method for intelligent driving evaluation, which analyzes the collected audio warning speech to determine the issuance time of the audio warning, thereby conducting a fair evaluation of the timeliness of the audio warning. First, the noise information in the speech is filtered out by soft threshold wavelet denoising; then, the endpoint detection based on the double threshold method is used to mark the valid speech segment in the speech to be tested; then, the MFCC features are extracted for each frame of the valid speech segment; finally, the LSTM network is trained with the MFCC features. After the training is completed, the speech segment where the audio warning signal is located can be detected, and the starting position of the speech segment is the issuance time of the audio warning.

Description

Technical Field

[0001] The present invention relates to an audio warning intelligent recognition method for intelligent driving evaluation. The method analyzes collected audio warning voices to determine the issuance time of the audio warning, thereby conducting a fair evaluation of the timeliness of the audio warning, and belongs to the field of intelligent driving evaluation. Background Art

[0002] With the rapid development of vehicle intelligent technology, more and more guidance / warning functions can effectively assist drivers, thereby improving driving safety and traffic efficiency. These functions are gradually moving from engineering development to implementation. Testing and evaluating warning functions is a prerequisite for their promotion and application, and accurate identification of warning signals is the key to conducting warning function evaluation.

[0003] In the intelligent vehicle-road system evaluation scenario, the recognition of voice warning information mainly includes two aspects: one is the recognition of the warning voice; the other is the judgment of the warning time. At present, voice warning recognition can only rely on manual recognition or threshold calibration methods to determine the time when the alarm voice is issued.

[0004] At present, there is no complete evaluation system or evaluation equipment for the above application. The purpose of the method of the present invention is to provide a method for the above evaluation. The basic data required for the evaluation is obtained by using a microphone sensor, and the data is processed with high precision and analyzed to obtain the evaluation results. The evaluation mainly determines whether the system can issue an audio warning message in time. Summary of the invention

[0005] The present invention proposes an audio warning intelligent recognition method for intelligent driving evaluation. The audio collected by the microphone is subjected to soft threshold wavelet noise reduction to filter out the noise information in the recorded speech; then, endpoint detection based on the double threshold method is used to mark the valid speech segments in the speech to be tested; then, MFCC features are extracted for each frame of the valid speech segment; finally, the LSTM network is trained with MFCC features. After the training is completed, the speech segment where the target speech (audio warning signal) is located can be detected, and the evaluation index analysis of the speech recognition results is performed. The specific steps include:

[0006] Step 1: Threshold denoising based on wavelet transform

[0007] First, the original speech signal S(n) is decomposed by discrete wavelet. The N-order Daubechies wavelet system is selected as the wavelet basis function, abbreviated as dbN; the speech signal is decomposed by J-layer wavelet to obtain the wavelet coefficients c of the approximate component j (n) and the wavelet coefficients d of the detail components j (n), the specific expression is shown in formula (1):

[0008]

[0009] Where j represents the number of wavelet coefficient decomposition levels. j+1 (m) represents the approximate wavelet coefficient c j+1 The mth value of (n), d j+1 (m) represents the detail wavelet coefficient d j+1 (n) is the mth value of. l(n) and h(n) are the decomposed low-pass filter coefficient and high-pass filter coefficient corresponding to the wavelet basis function dbN. c0(n) in formula (1) is the original speech signal S(n). The approximate component c can be obtained by calculating formula (1): J (n) and detail components d1(n), d2(n), ..., d J (n).

[0010] Then, the decomposed wavelet coefficients are subjected to threshold denoising. In the wavelet domain, the modulus of the wavelet coefficients corresponding to the effective signal is often large; while the noise is random in the time domain, and the modulus of the wavelet coefficients corresponding to it is often small in the wavelet domain. Therefore, the approximate component c J (n) is retained, for the detail components d1(n), d2(n), ..., d J (n) Threshold denoising is performed. A soft threshold function is used to update the detail component coefficients. The soft threshold function expression is shown in formula (2):

[0011]

[0012] in is the updated detail component coefficient, d j (n) is the original detail component coefficient, and λ is the denoising threshold. The threshold λ is determined by the formula λ=σ(0.3936+0.1829×log2 N), where N is the length of the original signal S(n), and σ is given by The noise component is removed by the threshold denoising function to obtain the updated detail component coefficient

[0013] Finally, the denoised wavelet coefficients are reconstructed to obtain the denoised speech X(n). The wavelet coefficients after threshold denoising are reconstructed, and the wavelet reconstruction expression is shown in formula (3):

[0014]

[0015] in and are the reconstructed low-pass filter coefficients and high-pass filter coefficients of the wavelet basis function dbN, respectively, and are the mirror filters of l(n) and h(n). j+1 (n) is the approximate component, from c J(n)Start iteration; is the detail component coefficient after denoising. This is the denoised speech signal X(n), and wavelet denoising is now completed.

[0016] Step 2: Speech endpoint detection based on double thresholds

[0017] First, the denoised speech X(n) is divided into frames. With t seconds as one frame (t is generally 20ms), the speech is divided into frames to obtain x i (n). Assume that a speech can be obtained by m frames. If the sampling frequency of speech X(n) is f, the number of sampling points of a frame of speech is K = t × f. Framing can be regarded as windowing the speech, and the window function used is a rectangular window.

[0018] Now we define two speech features for each frame of speech: short-time energy and short-time zero-crossing rate, and then use these two speech features to perform endpoint detection.

[0019] The short-time energy of a frame of speech signal is defined as shown in formula (4):

[0020]

[0021] The short-time zero-crossing rate of a frame of speech signal is defined as shown in formula (5):

[0022]

[0023] in

[0024] Next, define two short-term energy thresholds E high With E low , and a short-time zero-crossing rate threshold Z th .

[0025] E high E is a relatively high short-time energy threshold, which can be used to distinguish the voiced part of the speech. high Take half of the average short-time energy of all frames, and its value is shown in formula (6). high The frame is selected and determined as speech segment a. This part of speech is mainly voiced sound with large short-term energy.

[0026]

[0027] E low is a lower short-term energy threshold, which can be used to select speech with lower energy and expand the speech segment. low Take the short-term energy average of the first 10 frames and E high of The sum of the short-time energy greater than E low The speech segment of is added to speech segment a to obtain speech segment b.

[0028]

[0029] Z th is the threshold of the short-time zero-crossing rate, which is three times the sum of the short-time zero-crossing rates of the previous 10 frames, as shown in formula (8). th The unvoiced part of is added to speech segment b to obtain speech segment c.

[0030]

[0031] The part with fewer frames in the speech segment c is removed, and the remaining selected segment is the speech segment, which can complete the endpoint detection of the speech segment and mark the starting point of the valid speech segment in the speech to be tested.

[0032] Step 3: Extract speech MFCC features

[0033] MFCC features are extracted from the valid speech segments obtained after endpoint detection. Assuming that a certain segment of speech is s(n), the extraction process is as follows.

[0034] (1) Pre-emphasis. Pre-emphasize the speech segment to enhance the high-frequency part and make the signal spectrum flat. The calculation formula is shown in formula (9):

[0035] s′(n)=s(n)-μs(n-1) (30)

[0036] Among them, μ can be 0.9~1.0, generally 0.97.

[0037] (2) Framing: Framing uses the framing results obtained during endpoint detection.

[0038] (3) Windowing. Windowing is performed on each speech frame to improve the continuity of the left and right ends of the frame. The calculation formula is shown in formula (10):

[0039] g(n)=s′(n)×w(n) (31)

[0040] Among them, w(n) can generally be taken as a Hamming window, as shown in formula (11), where K is the number of sampling points in a frame.

[0041]

[0042] (4) Discrete Fourier Transform: Perform DFT on each frame of speech signal, and the calculation formula is as shown in formula (12).

[0043]

[0044] (5) Triangular filter bank. Define a set of Mel triangular filters. Generally, the number of filters is T = 26. The filter definition is as shown in formula (13):

[0045]

[0046] The definition of f(m) is as follows:

[0047]

[0048] f in formula (14) l With f h According to the speech sampling frequency f s Define the minimum and maximum frequencies, f mel is the Mel frequency. The floor(x) function represents rounding down, and K is the sampling point length of a frame of speech.

[0049] (6) Calculate the logarithmic energy of the filter output. Pass G(k) obtained by DFT through the Mel triangle filter bank and obtain its logarithmic energy. The calculation formula is as follows:

[0050]

[0051] (7) Discrete cosine transform. The logarithmic energy is transformed through discrete cosine transform, and the calculation formula is as follows:

[0052]

[0053] Wherein, T is the number of Mel triangle filters, L indicates that the obtained v(n) is the L-order MFCC coefficient, and L can generally be 13.

[0054] (8) Dynamic differential parameters. The static characteristics of speech can be expressed by v(n), while the dynamic characteristics can be expressed by its differential spectrum, and the calculation formula is shown in formula (17).

[0055]

[0056] Here, Ω can be 1 or 2. That is the first-order difference parameter, which is generally 13 coefficients.

[0057] The standard Mel cepstrum parameter v(n) and the first-order difference parameter Merge into a vector λ, where λ is the MFCC feature of the speech segment s(n), which is 26 dimensions.

[0058] Step 4: RNN-based speech recognition

[0059] The MFCC features obtained in step 3 are labeled and divided into two categories: those containing the target speech and those excluding the target speech. They are randomly assigned to the training set and the validation set in a ratio of 4:1 and trained using a recurrent neural network.

[0060] RNN adopts a single-layer LSTM structure. The input is the 26-dimensional MFCC speech features obtained in step 3. The dimension of the hidden unit of the LSTM network is set to 40. Finally, it passes through the fully connected layer and is classified and output by the softmax function.

[0061] The LSTM model uses the cell state C t To record long-term memory, through the hidden layer state h t To record short-term memory, there are three "gates": forget gate f t , input gate i t AND output gate o t .

[0062] Forget Gate t decides which information to discard from the cell state, by h t-1 With x t Output, the calculation expression is as shown in formula (18). Where σ represents the sigmoid function, W f is the weight matrix, b f is the bias term.

[0063] f t =σ(W f ·[h t-1 ,x t ]+b f ) (39)

[0064] Input Gate i t Long-term memory candidate Determines what information is stored in the cell state. The specific expression is shown in formula (19). i With W C is the weight matrix, b i With b C is the bias term.

[0065]

[0066] Update cell status C t , the calculation expression is as shown in formula (20).

[0067]

[0068] Based on cell state C t AND output gate o t Output h t , the calculation expression is as shown in formula (21). Where Wo is the weight matrix, b o is the bias term

[0069]

[0070] After setting up the LSTM network structure, the weight matrix is ​​initialized with orthogonal initialization, the bias term is initialized with zero, and the BPTT update strategy is used for training until the network parameters converge.

[0071] The trained neural network model is used to identify the MFCC features of the speech to be tested, and determine which segment of speech contains the target speech. The starting position of the speech segment is the time when the audio warning is issued. BRIEF DESCRIPTION OF THE DRAWINGS

[0072] Figure 1 It is the overall flow chart of audio warning recognition;

[0073] Figure 2 It is the block diagram of wavelet decomposition system;

[0074] Figure 3 It is the block diagram of wavelet reconstruction system;

[0075] Figure 4 is the endpoint detection flow chart;

[0076] Figure 5 It is the MFCC feature extraction flow chart;

[0077] Figure 6 It is the LSTM loop model structure;

[0078] Figure 7 This is the structure diagram of the LSTM network built. DETAILED DESCRIPTION

[0079] With the rapid development of vehicle intelligent technology, more and more guidance / warning functions can effectively assist drivers, thereby improving driving safety and traffic efficiency. These functions are gradually moving from engineering development to implementation. Testing and evaluating warning functions is a prerequisite for their promotion and application, and accurate identification of warning signals is the key to conducting warning function evaluation.

[0080] In the intelligent vehicle-road system evaluation scenario, the recognition of voice warning information mainly includes two aspects: one is the recognition of the warning voice; the other is the judgment of the warning time. At present, the recognition of voice warning information can only rely on manual recognition or threshold calibration methods to determine the time when the alarm voice is issued.

[0081] At present, there is no complete evaluation system or evaluation equipment for the above application. The purpose of the method of the present invention is to provide a method for the above evaluation. The basic voice data required for the evaluation is obtained by using a microphone sensor, and the data collected by the automobile warning system interface is combined with the data collected by the automobile warning system interface. The evaluation results are obtained by analysis based on high-precision data processing. The evaluation mainly determines whether the system can issue an audio warning message in time. The specific steps include:

[0082] For the audio collected by the microphone, the noise information in the recorded speech is filtered out through soft threshold wavelet noise reduction; then, the endpoint detection based on the double threshold method is used to mark the valid speech segment in the speech to be tested; then, the MFCC features are extracted for each frame of the valid speech segment; finally, the LSTM network is trained with the MFCC features. After the training is completed, the speech segment where the target speech (audio warning signal) is located can be detected, and the evaluation index analysis of the speech recognition results is performed. The overall flow chart is shown in Figure 1 The specific steps include:

[0083] Step 1: Threshold denoising based on wavelet transform

[0084] First, the original speech signal S(n) is decomposed by discrete wavelet. The N-order Daubechies wavelet system is selected as the wavelet basis function, abbreviated as dbN; the speech signal is decomposed by J-layer wavelet to obtain the wavelet coefficients c of the approximate component j (n) and the wavelet coefficients d of the detail components j (n), the specific expression is shown in formula (1):

[0085]

[0086] Where j represents the number of wavelet coefficient decomposition levels. j+1 (m) represents the approximate wavelet coefficient c j+1 The mth value of (n), d j+1 (m) represents the detail wavelet coefficient d j+1 (n) is the mth value of. l(n) and h(n) are the decomposed low-pass filter coefficient and high-pass filter coefficient corresponding to the wavelet basis function dbN. c0(n) in formula (1) is the original speech signal S(n). The approximate component c can be obtained by calculating formula (1): J (n) and detail components d1(n), d2(n), ..., d J (n). The system block diagram of wavelet decomposition is as follows Figure 2 As shown, L and H represent a low-pass filter and a high-pass filter respectively.

[0087] Then, the decomposed wavelet coefficients are subjected to threshold denoising. In the wavelet domain, the modulus of the wavelet coefficients corresponding to the effective signal is often large; while the noise is random in the time domain, and the modulus of the wavelet coefficients corresponding to it is often small in the wavelet domain. Therefore, the approximate component c J (n) is retained, for the detail components d1(n), d2(n), ..., d J (n) Threshold denoising is performed. A soft threshold function is used to update the detail component coefficients. The soft threshold function expression is shown in formula (2):

[0088]

[0089] in is the updated detail component coefficient, d j (n) is the original detail component coefficient, and λ is the denoising threshold. The threshold λ is determined by the formula λ=σ(0.3936+0.1829×log2 N), where N is the length of the original signal S(n), and σ is given by The noise component is removed by the threshold denoising function to obtain the updated detail component coefficient

[0090] Finally, the denoised wavelet coefficients are reconstructed to obtain the denoised speech X(n). The wavelet coefficients after threshold denoising are reconstructed, and the wavelet reconstruction expression is shown in formula (3):

[0091]

[0092] in and are the reconstructed low-pass filter coefficients and high-pass filter coefficients of the wavelet basis function dbN, respectively, and are the mirror filters of l(n) and h(n). j+1 (n) is the approximate component, from c J (n)Start iteration; is the detail component coefficient after denoising. The system block diagram of wavelet reconstruction is as follows Figure 3 As shown. After layers of iterations, we get This is the denoised speech signal X(n), and wavelet denoising is now completed.

[0093] Step 2: Speech endpoint detection based on double thresholds

[0094] First, the denoised speech X(n) is divided into frames. With t seconds as one frame (t is generally 20ms), the speech is divided into frames to obtain x i (n). Assume that a speech can be obtained by m frames. If the sampling frequency of speech X(n) is f, the number of sampling points of a frame of speech is K = t × f. Framing can be regarded as windowing the speech, and the window function used is a rectangular window.

[0095] Now we define two speech features for each frame of speech: short-time energy and short-time zero-crossing rate, and then use these two speech features to perform endpoint detection.

[0096] The short-time energy of a frame of speech signal is defined as shown in formula (4):

[0097]

[0098] The short-time zero-crossing rate of a frame of speech signal is defined as shown in formula (5):

[0099]

[0100] in

[0101] Next, define two short-term energy thresholds E high With E low , and a short-time zero-crossing rate threshold Z th .

[0102] E high E is a relatively high short-time energy threshold, which can be used to distinguish the voiced part of the speech. high Take half of the average short-time energy of all frames, and its value is shown in formula (6). high The frame is selected and determined as speech segment a. This part of speech is mainly voiced sound with large short-term energy.

[0103]

[0104] E low is a lower short-term energy threshold, which can be used to select speech with lower energy and expand the speech segment. low Take the short-term energy average of the first 10 frames and E high of The sum of the short-time energy greater than E low The speech segment of is added to speech segment a to obtain speech segment b.

[0105]

[0106] Z th is the threshold of the short-time zero-crossing rate, which is three times the sum of the short-time zero-crossing rates of the previous 10 frames, as shown in formula (8). th The unvoiced part of is added to speech segment b to obtain speech segment c.

[0107]

[0108] The part with fewer frames in the speech segment c is removed, and the remaining selected segment is the speech segment, which can complete the endpoint detection of the speech segment and mark the starting point of the valid speech segment in the speech to be tested.

[0109] Step 3: Extract speech MFCC features

[0110] MFCC features are extracted from the valid speech segments obtained after endpoint detection. Assuming that a certain speech segment is s(n), the extraction process is as follows.

[0111] (1) Pre-emphasis. Pre-emphasize the speech segment to enhance the high-frequency part and make the signal spectrum flat. The calculation formula is shown in formula (9):

[0112] s′(n)=s(n)-μs(n-1) (51)

[0113] Among them, μ can be 0.9~1.0, generally 0.97.

[0114] (2) Framing: Framing uses the framing results obtained during endpoint detection.

[0115] (3) Windowing. Windowing is performed on each speech frame to improve the continuity of the left and right ends of the frame. The calculation formula is shown in formula (10):

[0116] g(n)=s′(n)×w(n) (52)

[0117] Among them, w(n) can generally be taken as a Hamming window, as shown in formula (11), where K is the number of sampling points in a frame.

[0118]

[0119] (4) Discrete Fourier Transform: Perform DFT on each frame of speech signal, and the calculation formula is as shown in formula (12).

[0120]

[0121] (5) Triangular filter bank. Define a set of Mel triangular filters. Generally, the number of filters is T = 26. The filter definition is as shown in formula (13):

[0122]

[0123] The definition of f(m) is as follows:

[0124]

[0125] f in formula (14) l With f h According to the speech sampling frequency f sDefine the minimum and maximum frequencies, f mel is the Mel frequency. The floor(x) function represents rounding down, and K is the sampling point length of a frame of speech.

[0126] (6) Calculate the logarithmic energy of the filter output. Pass G(k) obtained by DFT through the Mel triangle filter bank and obtain its logarithmic energy. The calculation formula is as follows:

[0127]

[0128] (7) Discrete cosine transform. The logarithmic energy is transformed through discrete cosine transform, and the calculation formula is as follows:

[0129]

[0130] Wherein, T is the number of Mel triangle filters, L indicates that the obtained v(n) is the L-order MFCC coefficient, and L can generally be 13.

[0131] (8) Dynamic differential parameters. The static characteristics of speech can be expressed by v(n), while the dynamic characteristics can be expressed by its differential spectrum, and the calculation formula is shown in formula (17).

[0132]

[0133] Here, Ω can be 1 or 2. That is the first-order difference parameter, which is generally 13 coefficients.

[0134] The standard Mel cepstrum parameter v(n) and the first-order difference parameter Merge into a vector λ, where λ is the MFCC feature of the speech segment s(n), which is 26 dimensions.

[0135] Step 4: RNN-based speech recognition

[0136] The MFCC features obtained in step 3 are labeled and divided into two categories: those containing the target speech and those excluding the target speech. They are randomly assigned to the training set and the validation set in a ratio of 4:1 and trained using a recurrent neural network.

[0137] RNN adopts a single-layer LSTM structure. The input is the 26-dimensional MFCC speech features obtained in step 3. The dimension of the hidden unit of the LSTM network is set to 40. Finally, it passes through the fully connected layer and is classified and output by the softmax function, such as Figure 7 shown.

[0138] The LSTM model uses the cell state C t To record long-term memory, through the hidden layer state h t To record short-term memory, there are three "gates": forget gate ft , input gate i t AND output gate o t .

[0139] Forget Gate t decides which information to discard from the cell state, by h t-1 With x t Output, the calculation expression is as shown in formula (18). Where σ represents the sigmoid function, W f is the weight matrix, b f is the bias term.

[0140] f t =σ(W f ·[h t-1 ,x t ]+b f ) (60)

[0141] Input Gate i t Long-term memory candidate Determines what information is stored in the cell state. The specific expression is shown in formula (19). i With W C is the weight matrix, b i With b C is the bias term.

[0142]

[0143] Update cell status C t , the calculation expression is as shown in formula (20).

[0144]

[0145] Based on cell state C t AND output gate o t Output h t , the calculation expression is as shown in formula (21). Where W o is the weight matrix, b o is the bias term

[0146]

[0147] After setting up the LSTM network structure, the weight matrix is ​​initialized with orthogonal initialization, the bias term is initialized with zero, and the BPTT update strategy is used for training until the network parameters converge.

[0148] The trained neural network model is used to identify the MFCC features of the speech to be tested, and determine which segment of speech contains the target speech. The starting position of the speech segment is the time when the audio warning is issued.

Claims

1. An audio warning intelligent recognition method for intelligent driving evaluation, characterized by: The following steps are involved: Step 1: Threshold denoising based on wavelet transform First, the original speech signal S(n) is decomposed by discrete wavelet; the N-order Daubechies wavelet system is selected as the wavelet basis function, abbreviated as dbN; the speech signal is decomposed by J-layer wavelet to obtain the wavelet coefficient c of the approximate component j (n) and the wavelet coefficients d of the detail components j (n), the specific expression is shown in formula (1): Where j represents the number of wavelet coefficient decomposition layers; c j+1 (m) represents the approximate wavelet coefficient c j+1 The mth value of (n), d j+1 (m) represents the detail wavelet coefficient d j+1 (n); l(n) and h(n) are the decomposed low-pass filter coefficient and high-pass filter coefficient corresponding to the wavelet basis function dbN respectively; c0(n) in formula (1) is the original speech signal S(n); the approximate component c is obtained by calculation of formula (1): J (n) and detail components d1(n), d2(n), ..., d J (n); Then, the decomposed wavelet coefficients are subjected to threshold denoising. In the wavelet domain, the modulus of the wavelet coefficients corresponding to the effective signal is often large. However, the noise is random in the time domain, and the modulus of the wavelet coefficients corresponding to it is often small in the wavelet domain. Therefore, the approximate component c J (n) is retained, for the detail components d1(n), d2(n), ..., d J (n) Perform threshold denoising; use a soft threshold function to update the detail component coefficients. The soft threshold function expression is shown in formula (2): in is the updated detail component coefficient, d j (n) is the original detail component coefficient, λ is the denoising threshold; the threshold λ is determined by the formula λ=σ(0.3936+0.1829×log2 N), N is the length of the original signal S(n), and σ is given by Determine; remove the noise component through the threshold denoising function to obtain the updated detail component coefficient Finally, the denoised wavelet coefficients are reconstructed to obtain the denoised speech X(n); the wavelet coefficients after threshold denoising are reconstructed, and the wavelet reconstruction expression is shown in formula (3): in and are the reconstructed low-pass filter coefficients and high-pass filter coefficients of the wavelet basis function dbN, which are the mirror filters of l(n) and h(n); c j+1 (n) is the approximate component, from c J (n)Start iteration; is the detail component coefficient after denoising; obtained after layers of iterations This is the denoised speech signal X(n), and wavelet denoising is now complete. Step 2: Speech endpoint detection based on double thresholds First, the denoised speech X(n) is divided into frames; Take t seconds as a frame, t is 20ms, divide the speech into frames, and get x i (n); Assume that a speech has m frames; If the sampling frequency of speech X(n) is f, then the number of sampling points of a frame of speech is K = t × f; Framing is regarded as windowing the speech, and the window function used is a rectangular window; Now define two speech features for each frame of speech: short-time energy and short-time zero-crossing rate, and then use these two speech features to perform endpoint detection; The short-time energy of a frame of speech signal is defined as shown in formula (4): The short-time zero-crossing rate of a frame of speech signal is defined as shown in formula (5): in Next, define two short-term energy thresholds E high With E low , and a short-time zero-crossing rate threshold Z th ; E high is a higher short-time energy threshold, which is used to distinguish the voiced part of the speech; E high Take half of the average short-time energy of all frames, whose value is shown in formula (6); high The frame is selected and determined as speech segment a. This part of speech is mainly voiced sound with large short-term energy. E low is a lower short-term energy threshold, which is used to select the speech with lower energy and expand the speech segment; E low Take the short-term energy average of the first 10 frames and E high of The sum of the short-time energy greater than E low The speech segment of is added to the speech segment a to obtain the speech segment b; Z th is the threshold of the short-time zero-crossing rate, which is three times the sum of the short-time zero-crossing rates of the previous 10 frames, as shown in formula (8); th The unvoiced part of is added to speech segment b to obtain speech segment c; The part with fewer frames in the speech segment c is removed, and the remaining selected segment is the speech segment, that is, the endpoint detection of the speech segment is completed, and the starting point of the valid speech segment is marked in the speech to be tested; Step 3: Extract speech MFCC features Extract MFCC features from the valid speech segments obtained after endpoint detection. Assuming that a certain speech segment is s(n), the extraction process is as follows; (1) Pre-emphasis: Pre-emphasize the speech segment to enhance the high-frequency part and make the signal spectrum flat. The calculation formula is shown in formula (9): s′(n)=s(n) -μs(n -1) (9) Where μ is 0.9 to 1.0; (2) Framing: Framing is performed using the framing result obtained during endpoint detection. (3) Windowing: Windowing is performed on each speech frame to improve the continuity of the left and right ends of the frame. The calculation formula is shown in formula (10): g(n)=s′(n)×w(n) (10) Among them, w(n) takes the Hamming window, as shown in formula (11), where K is the number of sampling points in a frame; (4) Discrete Fourier Transform: Perform DFT on each frame of speech signal. The calculation formula is as follows: (5) Triangular filter bank: A set of Mel triangular filters is defined. The number of filters is T = 26. The filter definition is as shown in formula (13): The definition of f(m) is as follows: f in formula (14) l With f h According to the speech sampling frequency f s Define the minimum and maximum frequencies, f mel is the Mel frequency; the floor(x) function represents rounding down, and K is the sampling point length of a frame of speech; (6) Calculate the logarithmic energy of the filter output. Pass G(k) obtained by DFT through the Mel triangle filter bank to obtain its logarithmic energy. The calculation formula is as follows: (7) Discrete cosine transform: Logarithmic energy is transformed through discrete cosine transform, and the calculation formula is as shown in formula (16): Where T is the number of Mel triangle filters, L means that the obtained v(n) is the L-order MFCC coefficient, which is 13; (8) Dynamic differential parameters: The static characteristics of speech are represented by v(n), while the dynamic characteristics are represented by its differential spectrum. The calculation formula is as shown in formula (17): Where Ω is 1 or 2; That is the first-order difference parameter, which is also 13 coefficients; The standard Mel cepstrum parameter v(n) and the first-order difference parameter Merge into vector λ, λ is the MFCC feature of speech segment s(n), which is 26-dimensional; Step 4: RNN-based speech recognition The MFCC features obtained in step 3 are labeled and divided into two categories: those containing the target speech and those not containing the target speech. They are randomly assigned to the training set and the validation set in a ratio of 4:1 and trained using a recurrent neural network. RNN adopts a single-layer LSTM structure; the input is the 26-dimensional MFCC speech features obtained in step 3, the dimension of the hidden unit of the LSTM network is set to 40, and finally passes through the fully connected layer and is classified and output by the softmax function; The LSTM model uses the cell state C t To record long-term memory, through the hidden layer state h t To record short-term memory, there are three "gates": forget gate f t , input gate i t AND output gate o t ; Forget Gate t The decision of what information to discard from the cell state is determined by h t-1 With x t Output, the calculation expression is as shown in formula (18); where σ represents the sigmoid function, W f is the weight matrix, b f is the bias term; f t =σ(W f ·[h t-1 ,x t ]+b f ) (18) Input Gate i t Long-term memory candidate Determines what information is stored in the cell state. The specific expression is as shown in formula (19); W i With W C is the weight matrix, b i With b C is the bias term; Update cell status C t , the calculation expression is as follows: Based on cell state C t AND output gate o t Output h t , the calculation expression is as shown in formula (21); where W o is the weight matrix, b o is the bias term After setting up the LSTM network structure, the weight matrix is ​​initialized with orthogonal initialization, the bias term is initialized with zero, and the BPTT update strategy is used for training until the network parameters converge. The trained neural network model is used to identify the MFCC features of the speech to be tested, and determine which segment of speech contains the target speech. The starting position of the speech segment is the time when the audio warning is issued.

Images