Complex speech recognition method, system and terminal for power grid dispatching based on BRNN

Abstract

The present invention discloses a complex speech recognition method, system and terminal for power grid dispatching based on BRNN, which relates to the field of speech recognition technology. The key points of the technical solution are: obtaining original speech information, and obtaining speech data after preprocessing the original speech information; extracting key feature parameters in the speech data to obtain key feature sequences; constructing an initial acoustic model, and obtaining a final acoustic model after training the initial acoustic model with a data set; inputting the key feature sequence into the final acoustic model to obtain text after the original speech information is recognized; the initial acoustic model is an end-to-end BRNN-CTC model constructed by combining a BRNN neural network and a CTC target loss function, and the BRNN neural network is constructed with initial consonants and finals as basic units. The present invention constructs a BRNN neural network with initial consonants and finals as basic units, and its BRNN-CTC model has a higher recognition accuracy rate in complex environments.

Description

Technical Field

[0001] The present invention relates to the field of speech recognition technology, and more specifically, to a complex speech recognition method, system and terminal for power grid dispatching based on BRNN. Background Art

[0002] Automatic speech recognition (ASR) is a technology that can convert speech signals into text. In the past, the GMM-HMM-based acoustic model was considered to be the main framework popular in the ASR field. GMM is used to recognize frames into states and act on the mapping of speech input to HMM states. HMM is used to combine states into phonemes, and then combine phonemes into words to obtain the changes in speech signals in time series. With the gradual popularity of deep learning, DNN has been introduced into the acoustic modeling of ASR. Unlike the GMM-HMM-based acoustic model, DNN replaces GMM to model the probability of observed states, which can better utilize context information to achieve nonlinear feature transformation and improve the accuracy of speech recognition. However, the DNN-HMM-based framework still has the problem of mandatory alignment of training data, which makes optimization difficult. In addition, HMM is a generative model, and its conditional independence assumption is inconsistent with reality. In this case, RNN and LSTM can further improve the accuracy of speech recognition for sequence labeling tasks through their powerful sequence output capabilities.

[0003] As one of the power production and operation organizations, the dispatching class of the power dispatching control center has low efficiency and high cost in the existing manual dispatching method, which cannot meet the increasing workload. To this end, the existing technology uses speech recognition technology in power dispatching operations, converting voice information into text signals, and then converting them into commands that can be recognized by the equipment. However, the power grid business covers a wide range, and not only the operating environment is complex and diverse, but also affected by the operator's word count and language ability, resulting in the relative accuracy of the above-mentioned existing speech recognition technology.

[0004] Therefore, how to study and design a BRNN-based power grid dispatching complex speech recognition method, system and terminal that can overcome the above-mentioned defects. Summary of the invention

[0005] In order to solve the deficiencies in the prior art, the purpose of the present invention is to provide a BRNN-based power grid dispatching complex speech recognition method, system and terminal, a BRNN neural network is constructed with initials and finals as basic units, and an end-to-end BRNN-CTC model is constructed in combination with a CTC target loss function. The BRNN network structure can better utilize context information and has a higher recognition accuracy in complex environments.

[0006] The above technical objectives of the present invention are achieved through the following technical solutions:

[0007] In the first aspect, a complex speech recognition method for power grid dispatching based on BRNN is provided, comprising the following steps:

[0008] Acquire original voice information, and obtain voice data after preprocessing the original voice information;

[0009] Extract key feature parameters from the speech data to obtain a key feature sequence;

[0010] Construct an initial acoustic model, and train the initial acoustic model with the data set to obtain the final acoustic model;

[0011] Input the key feature sequence into the final acoustic model to obtain the text after the original speech information is recognized;

[0012] Its characteristic is that the initial acoustic model is an end-to-end BRNN-CTC model constructed by combining a BRNN neural network and a CTC target loss function, and the BRNN neural network is constructed with initials and finals as basic units.

[0013] Furthermore, the preprocessing of the original voice information includes:

[0014] Pre-emphasis: The original voice information is high-pass filtered to enhance the high-frequency signal and weaken the low-frequency signal to obtain the voice signal;

[0015] Framing: overlapping sampling method is used for framing. The length of a frame is 10Ms-30Ms, and the ratio of frame shift to frame length is between 0 and 0.5.

[0016] Windowing: using the Hamming window function to transform the elements of each frame after framing with the elements corresponding to the window sequence;

[0017] Endpoint detection, effective speech detection based on short-time energy and short-time average zero-crossing rate.

[0018] Furthermore, the initial acoustic model is specifically:

[0019] Assume that the training data is S = {(x 1 ,z 1 ),(x 2 ,z 2 ),...(x N ,z N )}, where the number of samples is N, the sample is x, x=(x 1 ,x 2 ,x 3 ,...,x T ), x i ∈Rm , indicating that a sample is divided into T frames, x i Represents the characteristic parameters of the i-th frame, the label is z, z = (z 1 ,z 2 ,z 3 ,...z U ), indicating the correct phoneme corresponding to sample x;

[0020] After the features are processed by two RNNs, the posterior probability y of the phoneme is obtained by softmax calculation. represents the probability that the phoneme is k at time t, and the sum of all phoneme probabilities in one frame is 1, that is,

[0021] Furthermore, the data set adopts a data set of multiple samples, and the multiple samples are composed of voice information of male, female, Mandarin, dialect, noisy environment, non-noisy environment, single person, multiple people, and different age groups.

[0022] Furthermore, the identification method also includes:

[0023] The word error rate is calculated by comparing the recognized text with the standard result obtained by feedback;

[0024] The proportion of samples of the corresponding class in the data set is adjusted in a positively correlated manner according to the size of the word error rate.

[0025] Furthermore, the calculation formula of the word error rate is specifically:

[0026]

[0027] Among them, P represents the word error rate; N represents the total number of characters in the standard result; S represents the number of characters that need to be replaced; D represents the number of characters that need to be deleted; and I represents the number of characters that need to be inserted.

[0028] Furthermore, the proportion of samples in the data set is synchronously adjusted according to the word error rates of multiple types of samples.

[0029] Secondly, a complex speech recognition system for power grid dispatching based on BRNN is provided, including:

[0030] A preprocessing module is used to obtain original voice information and obtain voice data after preprocessing the original voice information;

[0031] A feature extraction module is used to extract key feature parameters from speech data and obtain a key feature sequence;

[0032] The model building module is used to build an initial acoustic model and obtain the final acoustic model after training the initial acoustic model with the data set;

[0033] The text recognition module is used to input the key feature sequence into the final acoustic model to obtain the text after the original speech information is recognized;

[0034] The initial acoustic model in the model construction module is an end-to-end BRNN-CTC model constructed by combining the BRNN neural network and the CTC target loss function. The BRNN neural network is constructed with initials and finals as basic units.

[0035] In a third aspect, a computer terminal is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the BRNN-based complex speech recognition method for power grid dispatching as described in any one of the first aspects is implemented.

[0036] In a fourth aspect, a computer-readable medium is provided, on which a computer program is stored, and the computer program is executed by a processor to implement the BRNN-based complex speech recognition method for power grid dispatching as described in any one of the first aspects.

[0037] Compared with the prior art, the present invention has the following beneficial effects:

[0038] 1. The complex speech recognition method for power grid dispatching based on BRNN proposed in the present invention constructs a BRNN neural network with initials and finals as basic units, and combines an end-to-end BRNN-CTC model constructed by a CTC target loss function. The BRNN network structure can better utilize context information and has a higher recognition accuracy in complex environments;

[0039] 2. The present invention dynamically adjusts the ratio of the data set used for training in the BRNN-CTC model according to the word error rates of samples of different types, so that the long-term recognition process can always maintain a high accuracy rate. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] The drawings described herein are used to provide a further understanding of the embodiments of the present invention, constitute a part of this application, and do not constitute a limitation of the embodiments of the present invention. In the drawings:

[0041] Figure 1 is a flow chart in an embodiment of the present invention;

[0042] Figure 2 is an end-to-end framework diagram based on CTC in an embodiment of the present invention;

[0043] Figure 3 is a system block diagram of an embodiment of the present invention. DETAILED DESCRIPTION

[0044] In order to make the objectives, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with embodiments and drawings. The exemplary embodiments of the present invention and their description are only used to explain the present invention and are not intended to limit the present invention.

[0045] Example 1: A complex speech recognition method for power grid dispatching based on BRNN, such as Figure 1 As shown, the following steps are included:

[0046] S1: obtaining original voice information, and preprocessing the original voice information to obtain voice data;

[0047] S2: extract key feature parameters from the speech data to obtain a key feature sequence;

[0048] S3: construct an initial acoustic model, and obtain the final acoustic model after training the initial acoustic model with the data set; the initial acoustic model is an end-to-end BRNN-CTC model constructed by combining the BRNN neural network and the CTC target loss function, and the BRNN neural network is constructed with initials and finals as basic units;

[0049] S4: Input the key feature sequence into the final acoustic model to obtain the text after the original voice information is recognized; convert the text into Linux instructions, transmit the Linux instructions to the designated dispatch desk, and then the designated dispatch desk workstation executes the Linux command to open the screen.

[0050] In this embodiment, the preprocessing of the original voice information includes pre-emphasis, framing, windowing and endpoint detection. Pre-emphasis, the original voice information is subjected to high-pass filtering to enhance the high-frequency signal and weaken the low-frequency signal to obtain the voice signal. Framing, the overlapping sampling method is used for framing, the frame length is 10Ms-30Ms, and the ratio of frame shift to frame length is between 0 and 0.5. Windowing, the Hamming window function is used to transform and calculate the elements of each frame after framing with the elements corresponding to the window sequence. Endpoint detection, effective voice detection is performed based on short-time energy and short-time average zero-crossing rate.

[0051] like Figure 2 As shown, the initial acoustic model is specifically: Let the training data be S = {(x 1 ,z 1 ),(x 2 ,z 2 ),...(x N ,z N )}, where the number of samples is N, the sample is x, x=(x 1 ,x 2 ,x 3 ,...,x T ), x i ∈Rm , indicating that a sample is divided into T frames, x i Represents the characteristic parameters of the i-th frame, the label is z, z = (z 1 ,z 2 ,z 3 ,...z U ), which indicates the correct phoneme corresponding to sample x; after the feature is processed by two RNNs, the posterior probability y of the phoneme is obtained by softmax calculation. represents the probability that the phoneme is k at time t, and the sum of all phoneme probabilities in one frame is 1, that is,

[0052] In this embodiment, the data set is randomly divided in a ratio of 7:3 to obtain a training set and a test set, and then the training set is divided in a certain ratio to obtain a validation set and a training set. The validation set is used to adjust parameters, and the test set is used to verify the effect of the model.

[0053] The dataset uses a multi-class sample dataset, which consists of voice information of male, female, Mandarin, dialect, noisy environment, non-noisy environment, single person, multiple people, and different age groups.

[0054] The recognition method also includes: calculating the word error rate by character comparison based on the recognized text and the standard result obtained by feedback; and adjusting the proportion of corresponding class samples in the data set in a positive correlation manner according to the size of the word error rate.

[0055] The calculation formula of word error rate is as follows:

[0056]

[0057] Among them, P represents the word error rate; N represents the total number of characters in the standard result; S represents the number of characters that need to be replaced; D represents the number of characters that need to be deleted; and I represents the number of characters that need to be inserted.

[0058] As an optional implementation, the proportion of samples in the data set is synchronously adjusted according to the word error rates of multiple types of samples.

[0059] Experimental verification: The 13652 voices provided in this embodiment were processed to obtain 14 hours and 50 minutes of valid data. The voice quality was uneven. The collected data had problems such as mixed gender, dialects, accents, etc. The constructed voice used Mandarin. Each voice was about 10 seconds long and there were 36115 non-repeated words. The training set and test set were divided into training set and test set in a ratio of 7:3, and then the data set was feature extracted.

[0060] The training set is input into the BRNN-CTC model for training. While ensuring the consistency of the number of hidden layers, the number of neurons in each hidden layer is doubled because BRNN is bidirectional.

[0061] The training results are shown in Table 1, and the average accuracy comparison results are shown in Table 2. The effect of speech recognition using deep neural network is better than that of shallow network based on GMM-HMM, and the BRNN network structure can better utilize context information and obtain higher recognition accuracy.

[0062] Table 1 Training results based on BRNN-CTC

[0063]

[0064] Table 2 Comparison of average accuracy

[0065]

[0066] From the above, we can see that speech recognition in complex environments based on BRNN is indeed better than traditional model recognition.

[0067] Example 2: A complex speech recognition system for power grid dispatching based on BRNN, such as Figure 3 As shown, it includes a preprocessing module, a feature extraction module, a model building module and a text recognition module.

[0068] Among them, the preprocessing module is used to obtain the original voice information, and obtain the voice data after preprocessing the original voice information. The feature extraction module is used to extract the key feature parameters in the voice data and obtain the key feature sequence. The model construction module is used to construct the initial acoustic model, and obtain the final acoustic model after training the initial acoustic model with the data set; the initial acoustic model in the model construction module is an end-to-end BRNN-CTC model constructed by combining the BRNN neural network and the CTC target loss function, and the BRNN neural network is constructed with initials and finals as basic units. The text recognition module is used to input the key feature sequence into the final acoustic model to obtain the text after the original voice information is recognized.

[0069] Working principle: The present invention constructs a BRNN neural network with initials and finals as basic units, and combines an end-to-end BRNN-CTC model constructed by a CTC target loss function. The BRNN network structure can better utilize context information and has a higher recognition accuracy in complex environments. In addition, the ratio of the data set used for training in the BRNN-CTC model is dynamically adjusted according to the word error rate of samples of different types, which can enable the long-term recognition process to always maintain a high accuracy rate.

[0070] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0071] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0072] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0073] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0074] The above specific implementation methods further illustrate the objectives, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above are only specific implementation methods of the present invention and are not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A complex speech recognition method for power grid dispatching based on BRNN, comprising the following steps: Acquire original voice information, and obtain voice data after preprocessing the original voice information; Extract key feature parameters from the speech data to obtain a key feature sequence; Construct an initial acoustic model, and train the initial acoustic model with the data set to obtain the final acoustic model; Input the key feature sequence into the final acoustic model to obtain the text after the original speech information is recognized; The method is characterized in that the initial acoustic model is an end-to-end BRNN-CTC model constructed by combining a BRNN neural network and a CTC target loss function, and the BRNN neural network is constructed with initials and finals as basic units; The preprocessing of the original voice information includes: Pre-emphasis: The original voice information is high-pass filtered to enhance the high-frequency signal and weaken the low-frequency signal to obtain the voice signal; Framing: overlapping sampling method is used for framing. The length of a frame is 10Ms-30Ms, and the ratio of frame shift to frame length is between 0 and 0.

5. Windowing: using the Hamming window function to transform the elements of each frame after framing with the elements corresponding to the window sequence; Endpoint detection, effective speech detection based on short-time energy and short-time average zero-crossing rate; The initial acoustic model is specifically: Assume that the training data is S = {(x1,z1),(x2,z2),...(x N ,z N )}, where the number of samples is N, the sample is x, x=(x1,x2,x3,...,x T ), x i ∈R m , indicating that a sample is divided into T frames, x i Represents the feature parameters of the i-th frame, the label is z, z=(z1,z2,z3,...z U ), indicating the correct phoneme corresponding to sample x; After the features are processed by two RNNs, the posterior probability y of the phoneme is obtained by softmax calculation. represents the probability that the phoneme is k at time t, and the sum of all the phoneme probabilities in one frame is 1, that is, 2. The complex speech recognition method for power grid dispatching based on BRNN according to claim 1 is characterized in that: The data set uses a data set of multiple samples, and the multiple samples are composed of voice information of male, female, Mandarin, dialect, noisy environment, non-noisy environment, single person, multiple people, and different age groups.

3. The complex speech recognition method for power grid dispatching based on BRNN according to claim 2 is characterized in that: The identification method also includes: The word error rate is calculated by comparing the recognized text with the standard result obtained by feedback; The proportion of samples of the corresponding class in the data set is adjusted in a positively correlated manner according to the size of the word error rate.

4. The complex speech recognition method for power grid dispatching based on BRNN according to claim 3 is characterized in that: The calculation formula of the word error rate is specifically: Among them, P represents the word error rate; N represents the total number of characters in the standard result; S represents the number of characters that need to be replaced; D represents the number of characters that need to be deleted; and I represents the number of characters that need to be inserted.

5. The complex speech recognition method for power grid dispatching based on BRNN according to claim 1 is characterized in that: The proportion of samples in the data set is synchronously adjusted according to the word error rates of multiple types of samples.

6. A complex speech recognition system for power grid dispatching based on BRNN, characterized by: include: A preprocessing module is used to obtain original voice information and obtain voice data after preprocessing the original voice information; A feature extraction module is used to extract key feature parameters from speech data and obtain a key feature sequence; The model building module is used to build an initial acoustic model and obtain the final acoustic model after training the initial acoustic model with the data set; The text recognition module is used to input the key feature sequence into the final acoustic model to obtain the text after the original speech information is recognized; The method is characterized in that the initial acoustic model in the model construction module is an end-to-end BRNN-CTC model constructed by combining a BRNN neural network and a CTC target loss function, and the BRNN neural network is constructed with initials and finals as basic units; The preprocessing of the original voice information includes: Pre-emphasis: The original voice information is high-pass filtered to enhance the high-frequency signal and weaken the low-frequency signal to obtain the voice signal; Framing: overlapping sampling method is used for framing. The length of a frame is 10Ms-30Ms, and the ratio of frame shift to frame length is between 0 and 0.

5. Windowing: using the Hamming window function to transform the elements of each frame after framing with the elements corresponding to the window sequence; Endpoint detection, effective speech detection based on short-time energy and short-time average zero-crossing rate; The initial acoustic model is specifically: Assume that the training data is S = {(x1,z1),(x2,z2),...(x N ,z N )}, where the number of samples is N, the sample is x, x=(x1,x2,x3,...,x T ), x i ∈R m , indicating that a sample is divided into T frames, x i Represents the feature parameters of the i-th frame, the label is z, z=(z1,z2,z3,...z U ), indicating the correct phoneme corresponding to sample x; After the features are processed by two RNNs, the posterior probability y of the phoneme is obtained by softmax calculation. represents the probability that the phoneme is k at time t, and the sum of all the phoneme probabilities in one frame is 1, that is, 7. A computer terminal comprising a memory, a processor and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the BRNN-based complex speech recognition method for power grid dispatching is implemented as described in any one of claims 1-5.

8. A computer readable medium having a computer program stored thereon, characterized in that: The computer program is executed by a processor to implement the BRNN-based complex speech recognition method for power grid dispatching as described in any one of claims 1 to 5.

Images