An artificial intelligence distribution network dispatching system based on speech recognition

Abstract

The invention discloses an artificial intelligence distribution network dispatching system based on speech recognition, comprising a first speech recognition module and a second speech recognition module, wherein: the first speech recognition module, by adopting a Bi-LSTM model to capture the dependency relationship of the speech data of the speech acquisition module, extracts the features within and between speech frames, and outputs the fuzzy Mandarin dispatching text and the fuzzy dialect dispatching text corresponding to the dispatcher; the second speech recognition module, by adopting a BERT model to perform deep processing on the meanings of the fuzzy Mandarin dispatching text and the fuzzy dialect dispatching text and adopting a multi-layer perceptron to perform iterative prediction calculation on the interactive processing results of the two texts, satisfies the semantic consistency of the two texts, and generates a final power grid dispatching text. The invention solves the problems of high error rate of repeated recognition, slow recognition speed and low semantic recognition accuracy of speech recognition of grassroots power grid dispatchers.

Description

Technical Field

[0001] The present invention belongs to the field of power grid dispatching based on speech recognition, and specifically refers to an artificial intelligence distribution network dispatching system based on speech recognition. Background Art

[0002] At present, artificial intelligence technology based on voice recognition has not been widely used in grassroots local power grid dispatching business. Many dispatchers use shorthand on paper with a pen, and the recorded content may deviate from the actual situation. Accurate and efficient dispatching records are related to the accuracy of dispatching processing.

[0003] In local grassroots units of the power grid, communication is often based on local dialects, supplemented by Mandarin. However, in the actual implementation of power grid dispatching, the speech recognition results are generated into text output for power grid dispatching to ensure the reliability of power grid dispatching. However, before the speech recognition results are generated into text, the dialect needs to be converted into Mandarin. The current power dispatching speech recognition system that converts dialects into Mandarin has problems such as high repeated recognition error rate, slow recognition speed, and low semantic recognition accuracy, which brings poor results to power grid dispatching. Summary of the invention

[0004] In order to solve the problems of high error rate of repeated recognition, slow recognition speed, low semantic recognition accuracy, etc. in the above-mentioned prior art, the present invention proposes an artificial intelligence distribution network scheduling system based on speech recognition to improve the above-mentioned problems.

[0005] The specific application is as follows:

[0006] An artificial intelligence distribution network dispatching system based on speech recognition includes a speech management system module, a speech acquisition module, a data storage module, a speech preprocessing module, a first speech recognition module, a second speech recognition module and a speech recognition optimization module;

[0007] The voice management system module is used to manage the scheduling of services, data and communications of each module of the system;

[0008] The voice collection module is used to collect the dispatcher's voice data;

[0009] The data storage module is used to store the data collected by the voice collection module;

[0010] The speech preprocessing module is used to perform noise reduction and endpoint detection on the speech data of the speech acquisition module;

[0011] The first speech recognition module captures the dependency of the speech data of the speech acquisition module by adopting a Bi-LSTM model, extracts features within and between speech frames, and outputs fuzzy Mandarin dispatch text and fuzzy dialect dispatch text corresponding to the dispatcher;

[0012] The second speech recognition module uses the BERT model to interactively process the meanings of the fuzzy Mandarin dispatch text and the fuzzy dialect dispatch text and uses a multi-layer perceptron to iteratively predict and calculate the interactive processing results of the two texts to meet the semantic consistency of the two texts and generate the final power grid dispatch text;

[0013] The speech recognition optimization module is used to evaluate and re-optimize the recognition results.

[0014] Further, the data storage module includes a distributed storage unit, a data backup unit and a data recovery unit;

[0015] The distributed storage unit is used to perform distributed storage of the database on multiple servers to ensure data security and reliability;

[0016] The data backup unit is used to regularly back up data to prevent data loss;

[0017] The data recovery unit is used to quickly recover data when the data is lost or damaged;

[0018] The voice collection module captures the dispatching voice of the power grid dispatcher through a microphone and telephone recording, and the dispatching voice is generated into an audio file, which is stored in the data storage module;

[0019] The speech preprocessing module, the preprocessing process is:

[0020] S1: filtering the audio file with a volume lower than a certain volume by setting a threshold to remove background noise in the speech;

[0021] S2: Using a hidden Markov model to identify the start and end positions of valid speech in the audio file.

[0022] Furthermore, the first speech recognition module processing process is:

[0023] S3: Speech signal preprocessing: the input audio file is subjected to speech preprocessing operation using a speech preprocessing module, and the long speech signal is divided into shorter frames or segments, each frame lasting 20 milliseconds;

[0024] S4: Feature extraction: Through the internal loop connection and gating mechanism, deep learning and memory are performed on each frame of speech signal to extract intra-frame features, including at least frequency distribution and energy level; by analyzing the conversion and relationship between frames, inter-frame features of speech signals are extracted, including at least the transition of phonemes, speech coherence and changes in intonation;

[0025] S5: Bidirectional LSTM layer construction: Initialize the Bi-LSTM model, including setting the hidden layer size, number of layers, and learning rate. The Bi-LSTM model contains two directional LSTM layers, one for processing the forward sequence and the other for processing the reverse sequence.

[0026] S6: Construct sequence: Organize the extracted feature frames into a sequence, which will be used as the input of the Bi-LSTM model. Pad and truncate the sequence to make its length match the input requirement of the Bi-LSTM model.

[0027] S7: Feature learning: The sequence data is input into the Bi-LSTM model, and the model learns the temporal dependencies in the sequence through LSTM layers in both forward and backward directions;

[0028] S8: Feature fusion: The outputs of the forward and backward LSTM layers are fused, and the bidirectional features at each moment are obtained by concatenation and weighted averaging.

[0029] S9: Context information extraction: Utilize the output of the Bi-LSTM module to extract context information from the speech signal. The context information is trained using a Bi-LSTM-based CTC acoustic model to generate two dispatch texts based on fuzzy Mandarin dispatch and fuzzy dialect dispatch.

[0030] Further, the second speech recognition module includes an input unit, a feature extraction unit, a feature interaction unit and a speech verification unit;

[0031] The input unit uses the BERT model to convert the two dispatch texts into a word-level embedding matrix, calculates the word similarity of the two dispatch texts as a matching signal, sets a similarity threshold, and if it is greater than the threshold, determines that both dispatch texts are Mandarin texts and the dispatcher inputs in Mandarin, outputs a recognized speech text, and the output recognized speech text is used for power grid dispatching operations;

[0032] If it is less than the threshold, the dispatcher is judged to be a dialect input, and then they are fused into the embedding matrix respectively. The matrix of the fuzzy Mandarin text is represented by A, and the matrix of the fuzzy dialect dispatch text is represented by B. The calculation is:

[0033]

[0034] Among them, ||A||2 and ||B||2 represent the bi-norm of matrices A and B, and W contains the word similarity information of A and B, which are then integrated into matrix A and matrix B to obtain the matrix of fused word similarity. The fusion calculation process is:

[0035] A=AW (Formula 2);

[0036] B=BW (Formula 3);

[0037] For part of speech, the part of speech sequence of the text is represented by a vector and input into the Bi-LSTM model to learn the feature representation of the text sentence structure. Given a text sequence of length n [w1, w2, ...w n ], mapping the part-of-speech tag of the word in the text to the vector space, for the word w i Each part of speech has a unique index representation, which is represented by the part-of-speech vector sequence pos[w1, w2, ...w n ] is input to the Bi-LSTM model to learn sentence structure features from two directions, forward and backward:

[0038] X A =BiLSTM(pos A [w1,w2,...w n ]) (Formula 4);

[0039] X B =BiLSTM(pos B [w1,w2,...w n ]) (Formula 5);

[0040] The feature extraction unit (62) performs feature extraction through a Transformer encoder, and the extraction process is as follows:

[0041] N1: The matrix dimension of the text obtained after the text is processed by the input unit is S×E, where S is the maximum sequence length and E is the embedding dimension;

[0042] N2: Calculate the matrix Q(query), K(key), and V(value) through the attention mechanism, where W Q , W K , W V is the weight matrix,

[0043] Q=W Q A (Formula 6);

[0044] K=W K A (Formula VII);

[0045] V=W V A (Formula 8);

[0046] N3: After obtaining the matrices Q, K, and V, perform self-attention calculations, where d k is the dimension of K,

[0047]

[0048] N4: Through multi-head attention, that is, m different linear transformations are used to project Q, K, and V. Finally, all the attention results are concatenated to obtain M, which is passed into a linear layer to obtain the output M of the multi-head attention. attention , where m is the number of attention heads,

[0049]

[0050] M=Concat(head i ,…,head m );

[0051] M attention =Linear(M);

[0052] N5: Perform residual connection and normalization on the result of step N4 and use it as the input of the fully connected layer:

[0053] z output =LayerNorm(A+M attention );

[0054] N6: After the result of step N5 is sent to the fully connected layer, a residual connection and layer normalization are performed. The dimension of the output matrix is ​​consistent with A.

[0055] Y output =LayerNorm(A+F(z output ));

[0056] The feature interaction unit (63) is used to represent the matrix of the fuzzy Mandarin text after the feature extraction by the feature extraction unit as A S×E , the corresponding part-of-speech embedding matrix is ​​(X A ) S×E , the matrix of fuzzy dialect text is represented as B S×E , the corresponding part-of-speech embedding moment is (X B ) S×E ;

[0057] Calculate the interactive attention matrix of two text messages as (S attion ) S×S ,

[0058] S attion =AB T ,

[0059] Yes (S attion ) S×S Each row of is averaged and pooled, and then fused to A to get the interactive A. At this time, A contains the attention weight information of B on all the words in A, which is calculated as:

[0060] A=AveragePooling(S attion ) row ×A;

[0061] Yes (S attion ) S×S Each column of A is average pooled and then fused to B to get the interactive B. At this time, B contains the attention weight information of A to all the words in B, which is calculated as:

[0062] B=AveragePooling((S attion ) row ) T ×B;

[0063] Calculate the part-of-speech embedding matrix interaction attention matrix (P attention ) S×S :

[0064] P attention =X A X B T ;

[0065] Yes (P attention ) S×S Each row of is averaged and pooled, and then fused to X A Get the X after interaction A , at this time X A Contains X B X A All the part-of-speech attention weight information in is calculated as:

[0066] X A =AveragePooling(P attention ) row ×X A ;

[0067] Yes (P attention ) S×S Each row of is averaged and pooled, and then fused to X B Get the X after interaction B , at this time X B Contains X A X B All the part-of-speech attention weight information in is calculated as:

[0068] X B =(AveragePooling((P attebtion ) col )) T ×X B ;

[0069] The results are then summed and normalized. After N interactions, the result contains more interaction information and context information, where N is the number of Transformer encoders.

[0070] The voice verification unit (64) is used to generate the final power dispatch text, and the process is as follows:

[0071] The two text matrices of the fuzzy Mandarin text and the fuzzy dialect text after the interaction of the feature interaction unit are represented as N and M. The corresponding vectors obtained after mean pooling are n and m respectively. The corresponding part-of-speech matrix after interaction is represented as (X N ) and (X M ), the corresponding vectors x1 and x2 are obtained after mean pooling, and the vectors are concatenated and input into the multi-layer perceptron to obtain the matching result, which is calculated as:

[0072] Y=H(Concat(n,m,x1,x2,n×m,|nm|))(Formula 10)

[0073] Among them, n×m represents the bitwise multiplication of vector n and vector m, focusing on the semantically identical parts of the two texts, |n–m| represents the absolute value of the bitwise subtraction of vector n and vector m, focusing on the semantically different parts of the two texts, H is a multi-layer feedforward neural network, and a matching result threshold Q is preset. If Y is greater than Q, the interactive text corresponding to the fuzzy Mandarin text and the interactive text corresponding to the fuzzy dialect text are output. If Y is less than Q, the feature interaction unit is iteratively processed, and the iterative processing process is:

[0074] The number of Transformer encoders N is incremented by 1 until the data input formula 10 for iterative processing by the feature interaction unit makes Y greater than Q, and then the iterative operation is exited to generate the final power dispatch text.

[0075] Further, the speech recognition optimization module includes a scheduling log display unit, a word error rate unit, a sentence error rate unit and a speech recognition optimization unit;

[0076] The dispatch log display unit is used to query the dispatch log and display the query dispatch log on a visual interface to check whether the dispatch operation of the dispatcher is abnormal;

[0077] The word error rate unit is used to count the word error rate in the output recognized text;

[0078] The sentence error rate unit is used to count the sentence error rates in the recognized text;

[0079] The speech recognition optimization unit is used to check whether there is a recognition difference between the semantics of the Mandarin text and the dialect text, and if there is a difference, calibrate the scheduling text;

[0080] The calibration process of the scheduling text is:

[0081] L1: Establish two machine learning text models. The first model corresponds to Mandarin, and the second model corresponds to dialects. Establish two word data tables. The first word table is the Mandarin corresponding table, and the second word table is the dialect corresponding table.

[0082] L2: Collect the professional vocabulary set ai = {a1, a2...an} of the power grid, and select the machine learning model of the SVM vector machine;

[0083] L3: Determine whether the speech recognition of the power grid professional word A1 to Mandarin or dialect words is wrong;

[0084] L4: If there is a Mandarin recognition error, check whether the first word list has a record of the word. If not, record the power grid professional Mandarin word A1 in the first word list, and add the correct Mandarin word corresponding item to the first word list to form A(A1, B). If yes, replace the correct item corresponding to the word; if there is a dialect recognition error, check whether the second word list has a record of the word. If not, record the power grid professional dialect word A1 in the second word list, and add the correct dialect word corresponding item to the second word list to form A(A1, B). If yes, replace the correct item corresponding to the word; each row of data in the word list is represented as A(A1, B), where A1 is an erroneous semantic word and B is the corresponding correct semantic word;

[0085] L5: Execute the task of generating text through speech recognition. If wrong words appear in the dialect text or Mandarin text, turn to step L4. Finally, all word semantics are recognized correctly and the correct scheduling text is generated. The first word list and the second word list are stored in the data storage module.

[0086] The beneficial effects of the artificial intelligence distribution network dispatching system based on speech recognition of the present invention are as follows:

[0087] The present invention uses a Bi-LSTM model to capture the dependency of the voice data of the voice acquisition module, extracts the features within and between voice frames, outputs the fuzzy Mandarin dispatch text and the fuzzy dialect dispatch text corresponding to the dispatcher, uses a BERT model to interactively process the meanings of the fuzzy Mandarin dispatch text and the fuzzy dialect dispatch text, and uses a multi-layer perceptron to iteratively predict and calculate the interactive processing results of the two texts, so as to meet the semantic consistency of the two texts and generate the final power grid dispatch text; according to the dialect characteristics of grassroots power grid dispatchers, the Bi-LSTM model can be used to improve the recognition accuracy by learning the context information of the voice signal, which is particularly suitable for processing dialects with unique characteristics. A speech recognition processing method for data with complex language characteristics generates two fuzzy power dispatching texts. Further, for the two fuzzy texts, the BERT model is used to fine-tune the two fuzzy texts generated by Bi-LSTM. The BERT model has been pre-trained on a large amount of text and can understand the deep semantic information of the text. Through interactive fine-tuning of the text, BERT can adapt to the specific dialect context in the field of power dispatching, improve the accuracy of semantic recognition, ensure that the speech recognition speed of the dialect of grassroots power grid dispatchers is fast, and the semantic recognition accuracy is high. Through machine learning, the incorrect word list of semantic recognition is saved, the error rate of speech recognition word repetition is reduced, and the speed of speech recognition generating dispatching text is improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0088] Figure 1 This is a system architecture diagram of an artificial intelligence distribution network dispatching system based on speech recognition in the present invention;

[0089] Figure 2 This is a second speech recognition module diagram of an artificial intelligence distribution network scheduling system based on speech recognition according to the present invention;

[0090] Figure 3 This is a data storage module diagram of an artificial intelligence distribution network scheduling system based on speech recognition in the present invention;

[0091] Figure 4 This is a speech recognition optimization module diagram of an artificial intelligence distribution network dispatching system based on speech recognition in the present invention.

[0092] Among them, 1-voice management system module, 2-voice collection module, 3-data storage module, 31-distributed storage unit, 32-data backup unit, 33-data recovery unit, 4-voice preprocessing module, 5-first voice recognition module, 6-second voice recognition module, 61-input unit, 62-feature extraction unit, 63-feature interaction unit, 64-voice verification unit, 7-voice recognition optimization module, 71-scheduling log display unit, 72-word error rate unit, 73-sentence error rate unit, 74-voice recognition optimization unit. DETAILED DESCRIPTION

[0093] The technical solution of the present invention will be described clearly and completely below in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0094] Example 1

[0095] First, the technical terms involved in the embodiments of the present application are explained.

[0096] (1) Bi-LSTM model

[0097] Bi-LSTM (Bidirectional Long Short-Term Memory) is a special type of recurrent neural network (RNN) that processes sequence data by introducing two LSTM layers, one in forward order and the other in reverse order. This structure enables Bi-LSTM to capture both forward-looking and retrospective information in time series, thereby better understanding and predicting time series data. Bi-LSTM has a wide range of applications in natural language processing (NLP), speech recognition, time series analysis and other fields.

[0098] (2) BERT model

[0099] BERT (Bidirectional Encoder Representations from Transformers) is a pre-trained deep learning model for natural language processing (NLP) tasks. It was proposed by Google in 2018 and quickly became a foundational model in the field of NLP, which had a profound impact on subsequent research and applications. The following are several key features of the BERT model;

[0100] Bidirectional Context

[0101] BERT is unique in its bidirectional training strategy, which considers the contextual information on both the left and right sides of a word at the same time during training. This is different from traditional unidirectional language models, which can only consider the context from left to right or from right to left. BERT's bidirectional contextual representation allows the representation of each word to contain information about the entire input sequence.

[0102] Transformer Architecture

[0103] BER is based on the Transformer architecture, a model that relies on the Self-Attention Mechanism, which allows the model to dynamically focus on different parts of the sequence when processing the sequence. The self-attention mechanism enables BERT to effectively handle long-distance dependency problems;

[0104] (3) CTC Acoustic Model

[0105] The CTC (Connectionist Temporal Classification) acoustic model is an end-to-end model for speech recognition tasks. It allows the model to learn the corresponding text representation directly from the audio signal without the complex process of separating the traditional acoustic model and the language model. CTC solves the length misalignment problem between the input sequence (audio feature) and the output sequence (text or phoneme) by introducing an additional "blank" symbol;

[0106] (4) Transformer Encoder

[0107] Transformer encoder is a model in deep learning, which is particularly important in the field of natural language processing (NLP). The Transformer model was first proposed by Vaswani et al. in their 2017 paper "Attention Is All You Need". It is completely based on the attention mechanism and abandons the recurrent neural network (RNN) structure widely used in previous models.

[0108] Key features of the Transformer encoder:

[0109] Self-Attention: allows the model to consider other elements in the sequence while processing each element of the sequence, which enables the model to capture long-range dependencies within the sequence;

[0110] Multi-Head Attention: Extend the self-attention mechanism to multiple heads, where each head learns a different representation of the sequence, and then merges these representations to obtain richer information;

[0111] Positional Encoding: Since the Transformer model itself does not have the ability to capture the order of the sequence, positional encoding is introduced to provide the position information of the words in the sequence;

[0112] Residual Connections: The output of each sublayer (self-attention layer and feed-forward neural network layer) is added to its input and then layer normalization is performed, which helps to avoid the vanishing gradient problem in deep networks;

[0113] Layer Normalization: Normalization is applied on the output of each sub-layer, which helps stabilize the training process;

[0114] Figure 1 This is a system architecture diagram of an artificial intelligence distribution network scheduling system based on speech recognition in the present invention.

[0115] The system includes a speech management system module 1, a speech acquisition module 2, a data storage module 3, a speech preprocessing module 4, a first speech recognition module 5, a second speech recognition module 6 and a speech recognition optimization module 7;

[0116] The voice management system module 1 is used to manage the scheduling of services, data and communications of each module of the system;

[0117] The voice collection module 2 is used to collect the dispatcher's voice data;

[0118] The data storage module 3 is used to store the data collected by the voice collection module 2;

[0119] The speech preprocessing module 4 is used to perform noise reduction and endpoint detection on the speech data of the speech acquisition module 2;

[0120] The first speech recognition module 5 captures the dependency of the speech data of the speech acquisition module 2 by adopting the Bi-LSTM model, extracts the features within and between speech frames, and outputs the fuzzy Mandarin dispatch text and the fuzzy dialect dispatch text corresponding to the dispatcher;

[0121] The second speech recognition module 6 uses the BERT model to interactively process the meanings of the fuzzy Mandarin dispatch text and the fuzzy dialect dispatch text and uses a multi-layer perceptron to iteratively predict and calculate the interactive processing results of the two texts to meet the semantic consistency of the two texts and generate the final power grid dispatch text;

[0122] The speech recognition optimization module 7 is used to evaluate and re-optimize the recognition results.

[0123] This scheme adopts the Bi-LSTM model to generate two fuzzy power dispatching texts based on the dialect characteristics of grassroots power grid dispatchers, improves and fine-tunes the BERT model speech recognition algorithm to achieve semantic consistency between the two texts, and solves the problems of high error rate of repeated recognition, slow recognition speed, and low semantic recognition accuracy of speech recognition of grassroots power grid dispatchers;

[0124] Example 2, this example is based on the above example, please refer to Figure 1 , the speech preprocessing module 4, the preprocessing process is:

[0125] S1: filtering the audio file with a volume lower than a certain volume by setting a threshold to remove background noise in the speech;

[0126] S2: Using a hidden Markov model to identify the start and end positions of valid speech in the audio file.

[0127] Example 3, this example is based on the above example, please refer to Figure 1 , using the Bi-LSTM (bidirectional long short-term memory network) module to extract speech features is a key step in the speech recognition system. It can effectively capture the temporal dependencies and long-term features in speech signals. The following is a detailed process of extracting speech features using the Bi-LSTM module:

[0128] Speech signal preprocessing:

[0129] The input dispatcher voice signal is pre-processed by noise reduction, sound enhancement and other operations to improve the quality of the voice signal.

[0130] Perform sound segmentation to segment the long speech signal into shorter frames or segments. Each frame usually lasts 20-30 milliseconds. In this embodiment, 20 milliseconds is selected as the segmentation period.

[0131] Feature extraction:

[0132] Feature extraction algorithms, such as Mel-frequency cepstral coefficients (MFCC) and filter bank output (FBANK), are applied to each frame of speech signal to convert the time domain signal into frequency domain features.

[0133] These features can capture important acoustic properties of speech signals, such as the dispatcher's pitch, rhythm, and timbre.

[0134] Build sequence:

[0135] The extracted feature frames are organized into a sequence, which will serve as the input of the Bi-LSTM model.

[0136] As needed, the sequences can be padded or truncated so that their length matches the input requirements of the Bi-LSTM model.

[0137] Bidirectional LSTM layer construction:

[0138] Initialize the Bi-LSTM model, including setting the hidden layer size to 128, the number of layers to 3, and the learning rate to .

[0139] The Bi-LSTM model contains two directional LSTM layers, one for processing the forward sequence and the other for processing the reverse sequence.

[0140] Feature Learning:

[0141] The sequence data is input into the Bi-LSTM model, and the model learns the temporal dependencies in the sequence through LSTM layers in both forward and backward directions.

[0142] In the forward LSTM layer, information is passed from the beginning to the end of the sequence; in the backward LSTM layer, information is passed from the end to the beginning of the sequence.

[0143] Feature extraction within speech frames

[0144] Intra-frame features: The feature vector of each frame contains the local acoustic information of the frame, such as frequency distribution, energy level, etc.

[0145] Bi-LSTM can deeply learn and memorize these local features through internal recurrent connections and gating mechanisms, thereby extracting richer intra-frame features.

[0146] Speech frame feature extraction

[0147] Inter-frame features: Bi-LSTM can extract inter-frame features of speech signals, such as phoneme transitions, speech coherence, and changes in intonation, by analyzing the transitions and relationships between frames.

[0148] These inter-frame features are very important for understanding the overall structure and contextual information of speech.

[0149] Feature Fusion:

[0150] The outputs of the forward and backward LSTM layers are fused, usually by concatenation and weighted averaging, to obtain a bidirectional feature representation at each moment.

[0151] These feature representations contain the past and future information of each frame of speech and can more comprehensively reflect the characteristics of speech signals.

[0152] Contextual Information Extraction:

[0153] The output of the Bi-LSTM module is used to extract long-term dependencies and contextual information in speech signals.

[0154] This information is important for understanding speech content, distinguishing different speakers, and recognizing commands in speech.

[0155] Feature output:

[0156] The features extracted by the Bi-LSTM module are output for subsequent speech recognition tasks, such as acoustic model training and speech-to-text conversion.

[0157] These features can be used alone or in combination with other features (such as those extracted by the BERT module) to improve the accuracy of speech recognition.

[0158] Through the above process, the Bi-LSTM module can effectively extract rich temporal features and contextual information from the speech signal, providing strong support for the power grid dispatching speech recognition system.

[0159] Example 4, this example is based on the above example, please refer to Figure 2 , the second speech recognition module 6 includes an input unit 61, a feature extraction unit 62, a feature interaction unit 63 and a speech verification unit 64;

[0160] The input unit 61 uses the BERT model to convert the two dispatch texts into a word-level embedding matrix, calculates the word similarity of the two dispatch texts as a matching signal, sets a similarity threshold, and if it is greater than the threshold, determines that both dispatch texts are Mandarin texts and the dispatcher inputs in Mandarin, outputs a recognized speech text, and the output recognized speech text is used for power grid dispatching operations;

[0161] If it is less than the threshold, the dispatcher is judged to be a dialect input, and then they are fused into the embedding matrix respectively. The matrix of the BERT model for the fuzzy Mandarin text is represented by A, and the matrix of the BERT model for the fuzzy dialect dispatch text is represented by B. The calculation is as shown in formula 1.

[0162]

[0163] Among them, ||A||2 and ||B||2 represent the bi-norm of matrices A and B, and W contains the word similarity information of A and B, which are then integrated into matrix A and matrix B to obtain the matrix of fused word similarity. The fusion calculation process is as follows:

[0164] A=AW (Formula 2),

[0165] B=BW (Formula 3),

[0166] For part of speech, the part of speech sequence of the text is represented by a vector and input into the Bi-LSTM model to learn the feature representation of the text sentence structure. Given a text sequence of length n [w1, w2, ...w n], mapping the part-of-speech tag of the word in the text to the vector space, for the word w i Each part of speech has a unique index representation, which is represented by the part-of-speech vector sequence pos[w1, w2, ...w n ] is input into the Bi-LSTM model to learn sentence structure features from two directions, namely forward and backward. The formula is as follows:

[0167] X A =BiLSTM(pos A [w1,w2,...w n ])(Formula 4),

[0168] X B =BiLSTM(pos B [w1,w2,...w n ])(Formula 5);

[0169] The feature extraction unit 62 performs feature extraction through a Transformer encoder, and the extraction process is as follows:

[0170] N1: The matrix dimension of the text obtained after the text is processed by the input unit is S×E, where S is the maximum sequence length and E is the embedding dimension. In this embodiment, S=16 and E=384 are selected as parameter inputs;

[0171] N2: Calculate the matrix Q(query), K(key), and V(value) through the attention mechanism, where W Q , W K , W V is the weight matrix,

[0172] Q=W Q A (Formula 6),

[0173] K=W K A (Formula 7),

[0174] V=W V A (Formula 8);

[0175] N3: After obtaining the matrices Q, K, and V, perform self-attention calculations, where d k is the dimension of K,

[0176]

[0177] N4: Through multi-head attention, that is, m different linear transformations are used to project Q, K, and V. Finally, all the attention results are concatenated to obtain M, which is passed into a linear layer to obtain the output M of the multi-head attention. attention , where m is the number of attention heads,

[0178]

[0179] M=Concat(head i ,…,head m ),

[0180] M attention =Linear(M);

[0181] N5: Perform residual connection and normalization on the result of step N4 and use it as the input of the fully connected layer:

[0182] z output =LayerNorm(A+M attention );

[0183] N6: After the result of step N5 is sent to the fully connected layer, a residual connection and layer normalization are performed. The dimension of the output matrix is ​​consistent with A.

[0184] Y output =LayerNorm(A+F(z output ));

[0185] The feature interaction unit 63, through the feature extraction unit after the fuzzy Mandarin text matrix is ​​represented as A S×E , the corresponding part-of-speech embedding matrix is ​​(X A ) S×E , the matrix of fuzzy dialect text is represented as B S×E , the corresponding part-of-speech embedding moment is (X B ) S×E ;

[0186] Calculate the interactive attention matrix of two text messages as (S attion ) S×S ,

[0187] S attion =AB T ,

[0188] Yes (S attion ) S×S Each row of is averaged and pooled, and then fused to A to get the interactive A. At this time, A contains the attention weight information of B on all the words in A, which is calculated as follows:

[0189] A=AveragePooling(S attion ) row ×A,

[0190] Yes (S attion ) S×SEach column of A is averaged and pooled, and then fused to B to obtain the interactive B. At this time, B contains the attention weight information of A on all the words in B, which is calculated as follows:

[0191] B=AveragePooling((S attion ) row ) T ×B,

[0192] Calculate the part-of-speech embedding matrix interaction attention matrix (P attention ) S×S :

[0193] P attention =X A X B T ,

[0194] Yes (P attention ) S×S Each row of is averaged and pooled, and then fused to X A Get the X after interaction A , at this time X A Contains X B X A All the part-of-speech attention weight information in is calculated as follows:

[0195] X A =AveragePooling(P attention ) row ×X A , for (P attention ) S×S Each row of is averaged and pooled, and then fused to X B Get the X after interaction B , at this time X B Contains X A X B All the part-of-speech attention weight information in is calculated as follows:

[0196] X B =(AveragePooling((P attention ) col )) T ×X B ,

[0197] The results are then summed and normalized. After N interactions, the result contains more interaction information and context information, where N is the number of Transformer encoders.

[0198] The voice verification unit 64 performs the verification process as follows:

[0199] The two text matrices after the fuzzy Mandarin text and the fuzzy dialect text are interacted by the feature interaction unit 63 are represented as N and M. After mean pooling, the corresponding vectors obtained are n and m. The corresponding part-of-speech matrix after interaction is represented as (X N ) and (X M ), the corresponding vectors x1 and x2 are obtained after mean pooling, and the vectors are concatenated and input into the multi-layer perceptron to obtain the matching result, which is calculated as:

[0200] Y=H(Concat(n,m,x1,x2,n×m,|nm|)) (Formula 10)

[0201] Among them, n×m represents the bitwise multiplication of vector n and vector m, focusing on the semantically identical parts of the two texts, |n–m| represents the absolute value of the bitwise subtraction of vector n and vector m, focusing on the semantically different parts of the two texts, H is a multi-layer feedforward neural network, and a matching result threshold Q is preset. If Y is greater than Q, the interactive text corresponding to the fuzzy Mandarin text and the interactive text corresponding to the fuzzy dialect text are output. If Y is less than Q, the feature interaction unit 63 is iteratively processed, and the iterative processing process is:

[0202] The number of Transformer encoders N is incremented by 1 until the data for iterative processing by the feature interaction unit 63 is input into formula 10, so that Y is greater than Q, and the iterative operation is exited.

[0203] Specifically, the complexity of the model is usually closely related to the number of layers of the Transformer encoder. The more layers there are, the longer the training time is. Therefore, it is urgent to find a model with fewer layers, faster training speed and higher accuracy. In this embodiment, the number of layers of the Transformer encoder is set to 1, 2, 3, 4, 5, and 6 respectively. According to the test results, the number of encoder layers is finally set to 3. The performance of the feature extraction unit is related to the number of attention heads, but too many heads may cause the model to overfit. In this embodiment, the number of attention heads is set to 4, 6, 8, and 12. According to the test results, the number of encoder attention heads is finally set to 8. When training the model, attention should be paid to the convergence of the model. If the model converges, the training should be stopped, otherwise the model will overfit and fail to achieve the desired effect. The Transformer encoder is used as an excellent feature extractor, and multiple interactions of Mandarin dispatch text, dialect dispatch text information and sentence structure information are used to learn richer feature expressions, which has a good performance on the text matching effect.

[0204] Through the above process, the advantages of Bi-LSTM and BERT can be effectively combined to extract richer and more accurate speech features, thereby improving the overall performance of the power grid dispatching speech recognition system.

[0205] Example 5, this example is based on the above example, please refer to Figure 4 , the speech recognition optimization module 7 includes a scheduling log display unit 71, a word error rate unit 72, a sentence error rate unit 73 and a speech recognition optimization unit 74;

[0206] The dispatch log display unit 71 is used to query the dispatch log and display the query dispatch log on a visual interface to check whether the dispatch operation of the dispatcher is abnormal;

[0207] The word error rate unit 72 is used to count the word error rate in the recognized text;

[0208] The sentence error rate unit 73 is used to count and output the sentence error rate in the recognized text;

[0209] The speech recognition optimization unit 74 is used to check whether there is a recognition difference between the semantics of the Mandarin text and the dialect text, and if there is a difference, calibrate the scheduling text;

[0210] The calibration process of the scheduling text is:

[0211] L1: Establish two machine learning text models. The first model corresponds to Mandarin, and the second model corresponds to dialects. Establish two word data tables. The first word table is the Mandarin corresponding table, and the second word table is the dialect corresponding table.

[0212] L2: Collect the professional vocabulary set ai = {a1, a2...an} of the power grid, and select the machine learning model of the SVM vector machine;

[0213] L3: Determine whether the speech recognition of the power grid professional word a1 to the Mandarin or dialect word is wrong;

[0214] L4: If there is a Mandarin recognition error, check whether the first word list has a record of the word. If not, record the power grid professional Mandarin word a1 in the first word list, and add the correct Mandarin word corresponding item to the first word list to form A(a1, B). If yes, replace the correct item corresponding to the word; if there is a dialect recognition error, check whether the second word list has a record of the word. If not, record the power grid professional dialect word a1 in the second word list, and add the correct dialect word corresponding item to the second word list to form A(a1, B). If yes, replace the correct item corresponding to the word; each row of data in the word list is represented as A(a1, B), where a1 is an erroneous semantic word and B is the corresponding correct semantic word;

[0215] L5: Execute the task of generating text through speech recognition. If there are wrong words in the dialect text or the Mandarin text, turn to step L4. Finally, all the semantics of the words are correctly recognized and the correct scheduling text is generated. The first word list and the second word list are stored in the data storage module 3.

[0216] In the above operation, this solution introduces a machine learning model to record and save error-prone text words, reduce the problem of repeated errors in speech recognition words, and improve speech recognition speed;

[0217] The power grid dispatching speech recognition system of the present invention can be deployed on a server of a power grid dispatching center, and the speech recognition module can process the voice instructions of the dispatcher in real time and generate a dispatching text. The dispatching text can be queried through a visual interface to check whether there are any problems with the dispatching operation. The system can automatically execute the corresponding dispatching operation according to the recognition result, thereby improving the intelligent level of power grid dispatching.

[0218] The beneficial effects of the artificial intelligence distribution network dispatching system based on speech recognition of the present invention are as follows:

[0219] The present invention uses a Bi-LSTM model to capture the dependency of the voice data of the voice acquisition module, extracts the features within and between voice frames, outputs the fuzzy Mandarin dispatch text and the fuzzy dialect dispatch text corresponding to the dispatcher, uses a BERT model to interactively process the meanings of the fuzzy Mandarin dispatch text and the fuzzy dialect dispatch text, and uses a multi-layer perceptron to iteratively predict and calculate the interactive processing results of the two texts, so as to meet the semantic consistency of the two texts and generate the final power grid dispatch text; according to the dialect characteristics of grassroots power grid dispatchers, the Bi-LSTM model can be used to improve the recognition accuracy by learning the context information of the voice signal, which is particularly suitable for processing dialects with unique characteristics. A speech recognition processing method for data with complex language characteristics generates two fuzzy power dispatching texts. Further, for the two fuzzy texts, the BERT model is used to fine-tune the two fuzzy texts generated by Bi-LSTM. The BERT model has been pre-trained on a large amount of text and can understand the deep semantic information of the text. Through interactive fine-tuning of the text, BERT can adapt to the specific dialect context in the field of power dispatching, improve the accuracy of semantic recognition, ensure that the speech recognition speed of the dialect of grassroots power grid dispatchers is fast, and the semantic recognition accuracy is high. Through machine learning, the incorrect word list of semantic recognition is saved, the error rate of speech recognition word repetition is reduced, and the speed of speech recognition generating dispatching text is improved.

[0220] The present invention and its embodiments are described above, which is not restrictive. What is shown in the accompanying drawings is only one of the embodiments of the present invention, and the actual content is not limited to this. In short, if ordinary technicians in this field are inspired by it and do not deviate from the purpose of the invention, they can creatively design structural methods and embodiments similar to the technical scheme, which should all fall within the scope of protection of the present invention.

Claims

1. An artificial intelligence distribution network dispatching system based on speech recognition, characterized in that: It comprises a voice management system module (1), a voice collection module (2), a data storage module (3), a voice preprocessing module (4), a first voice recognition module (5), a second voice recognition module (6) and a voice recognition optimization module (7); The voice management system module (1) is used to manage the scheduling of services, data and communications of each module of the system; The voice collection module (2) is used to collect the dispatcher's voice data; The data storage module (3) is used to store the data collected by the voice collection module (2); The speech preprocessing module (4) is used to perform noise reduction and endpoint detection on the speech data of the speech acquisition module (2); The first speech recognition module (5) captures the dependency relationship of the speech data of the speech acquisition module (2) by using a Bi-LSTM model, extracts features within and between speech frames, and outputs fuzzy Mandarin dispatch text and fuzzy dialect dispatch text corresponding to the dispatcher; The second speech recognition module (6) uses a BERT model to interactively process the meanings of the fuzzy Mandarin dispatch text and the fuzzy dialect dispatch text and uses a multi-layer perceptron to iteratively predict and calculate the interactive processing results of the two texts, so as to meet the semantic consistency of the two texts and generate a final power grid dispatch text; The speech recognition optimization module (7) is used to evaluate and optimize the process according to the recognition results; The second speech recognition module (6) comprises an input unit (61), a feature extraction unit (62), a feature interaction unit (63) and a speech verification unit (64); The input unit (61) uses a BERT model to convert the fuzzy Mandarin dispatch text and the fuzzy dialect dispatch text into a word-level embedding matrix, calculates the word similarity between the fuzzy Mandarin dispatch text and the fuzzy dialect dispatch text as a matching signal, sets a similarity threshold, and if the similarity is greater than the similarity threshold, determines that the fuzzy Mandarin dispatch text and the fuzzy dialect dispatch text are both Mandarin texts, and the dispatcher input is Mandarin, outputs a recognized speech text, and the output recognized speech text is used for power grid dispatching operations; If it is less than the similarity threshold, it is determined that the dispatcher input is a dialect, and then they are merged into the embedding matrix respectively. The matrix of the fuzzy Mandarin dispatch text transformed by the BERT model is represented by A, and the matrix of the fuzzy dialect dispatch text transformed by the BERT model is represented by B. Calculate: ， in, and Represents the bi-norm of matrices A and B, W contains the word similarity information of A and B, and then merged into matrix A and matrix B to obtain the matrix of fused word similarity. The fusion calculation process is: A = AW (Formula 2), B = BW (Formula 3), For part of speech, the part of speech sequence of the text is represented by a vector and input into the Bi-LSTM model to learn the feature representation of the text sentence structure. Given a text sequence of length n [ , ,... ], mapping the part-of-speech tags of words in the text to the vector space, for words Each part of speech has a unique index representation, which is represented by the part of speech vector sequence pos[ , ,... ] is input to the Bi-LSTM model to learn sentence structure features from two directions, forward and backward: ； ； The feature extraction unit (62) performs feature extraction through a Transformer encoder, and the extraction process is as follows: N1: The matrix dimension of the text obtained after the text is processed by the input unit is S×E, where S is the maximum sequence length and E is the embedding dimension; N2: Calculate the matrices Q (query), K (key), and V (value) through the attention mechanism, where is the weight matrix, ； ； ； N3: After obtaining the matrices Q, K, and V, self-attention calculation is performed, where is the dimension of K, ， N4: Through multi-head attention, that is, m different linear transformations are used to project Q, K, and V. Finally, all the attention results are concatenated to obtain M, which is passed into a linear layer to obtain the output of the multi-head attention. , where m is the number of attention heads, ； ； ； N5: Perform residual connection and normalization on the result of step N4 and use it as the input of the fully connected layer: ； N6: After the result of step N5 is sent to the fully connected layer, a residual connection and layer normalization are performed. The dimension of the output matrix is ​​consistent with A. ； The feature interaction unit (63) is used to express the matrix of the fuzzy Mandarin dispatch text after the feature extraction by the feature extraction unit as follows: , the corresponding part-of-speech embedding matrix is , the matrix representation of the fuzzy dialect dispatch text is , the corresponding part-of-speech embedding moment is ; Calculate the interactive attention matrix of two text messages as , ， right Each row of is averaged and pooled, and then fused to A to get the interactive A. At this time, A contains the attention weight information of B on all the words in A, which is calculated as: ， right Each column of A is average pooled and then fused to B to get the interactive B. At this time, B contains the attention weight information of A to all the words in B, which is calculated as: ， Calculate the part-of-speech embedding matrix corresponding to the two texts and the interactive attention matrix : ， right Each row of is averaged and pooled, and then fused into After the interaction ,at this time Included right All the part-of-speech attention weight information in is calculated as: ， right Each row of is averaged and pooled, and then fused into After the interaction , at this time Included right All the part-of-speech attention weight information in is calculated as: , The results are then summed and normalized. After N interactions, the result contains more interaction information and context information, where N is the number of Transformer encoders. The speech verification unit (64) is used to generate the final power grid dispatch text, and the process is as follows: The two text matrices after the fuzzy Mandarin dispatch text and the fuzzy dialect dispatch text interact with each other through the feature interaction unit (63) are represented as A and B. After mean pooling, the corresponding vectors a and b are obtained respectively. The corresponding part-of-speech matrix after interaction is represented as ( )and( ), respectively, after mean pooling, the corresponding vectors are and , after vector concatenation, input into the multi-layer perceptron to obtain the matching result, and calculate: ， Wherein, a×b represents the bitwise multiplication of vector a and vector b, focusing on the semantically identical parts of the two texts, |a–b| represents the absolute value of the bitwise subtraction of vector a and vector b, focusing on the semantically different parts of the two texts, H is a multi-layer feedforward neural network, and a matching result threshold Q is preset. If Y is greater than Q, the interactive text corresponding to the fuzzy Mandarin dispatch text and the interactive text corresponding to the fuzzy dialect dispatch text are output. If Y is less than Q, the feature interaction unit (63) is iteratively processed, and the iterative processing process is as follows: The number of Transformer encoders N is incremented by 1 until the data input formula 10 of the feature interaction unit (63) is iteratively processed, so that Y is greater than Q, and the iterative operation is exited to generate the final power grid dispatch text.

2. According to claim 1, an artificial intelligence distribution network dispatching system based on speech recognition is characterized in that: The data storage module (3) comprises a distributed storage unit (31), a data backup unit (32) and a data recovery unit (33); The distributed storage unit (31) is used to perform distributed storage of the database on multiple servers to ensure data security and reliability; The data backup unit (32) is used to regularly back up data to prevent data loss; The data recovery unit (33) is used to quickly recover data when data is lost or damaged; The voice collection module (2) captures the dispatching voice of the power grid dispatcher through a microphone and telephone recording, and the dispatching voice is generated into an audio file, and the audio file is stored in the data storage module (3); The speech preprocessing module (4) comprises the following preprocessing steps: S1: filtering the audio file with a volume lower than a certain volume by setting a threshold to remove background noise in the speech; S2: Using a hidden Markov model to identify the start and end positions of valid speech in the audio file.

3. The artificial intelligence distribution network dispatching system based on speech recognition according to claim 2 is characterized in that: The processing process of the first speech recognition module (5) is as follows: S3: Speech signal preprocessing: the input audio file is subjected to speech preprocessing operation using the speech preprocessing module (4), so as to divide the long speech signal into shorter frames or segments, each frame lasting 20 milliseconds; S4: Feature extraction: Through the internal loop connection and gating mechanism, deep learning and memory are performed on each frame of speech signal to extract intra-frame features, including at least frequency distribution and energy level; by analyzing the conversion and relationship between frames, inter-frame features of speech signals are extracted, including at least the transition of phonemes, speech coherence and changes in intonation; S5: Bidirectional LSTM layer construction: Initialize the Bi-LSTM model, including setting the hidden layer size, number of layers, and learning rate. The Bi-LSTM model contains two directional LSTM layers, one for processing the forward sequence and the other for processing the reverse sequence. S6: Construct sequence: Organize the extracted feature frames into a sequence, which will be used as the input of the Bi-LSTM model. Pad and truncate the sequence to make its length match the input requirement of the Bi-LSTM model. S7: Feature learning: The sequence data is input into the Bi-LSTM model, and the model learns the temporal dependencies in the sequence through LSTM layers in both forward and backward directions; S8: Feature fusion: The outputs of the forward and backward LSTM layers are fused, and the bidirectional features at each moment are obtained by concatenation and weighted averaging. S9: Context information extraction: Utilize the output of the Bi-LSTM module to extract context information from the speech signal. The context information is trained using a Bi-LSTM-based CTC acoustic model to generate fuzzy Mandarin dispatch text and fuzzy dialect dispatch text.

4. The artificial intelligence distribution network dispatching system based on speech recognition according to claim 1 is characterized in that: The speech recognition optimization module (7) comprises a scheduling log display unit (71), a word error rate unit (72), a sentence error rate unit (73) and a speech recognition optimization unit (74); The dispatch log display unit (71) is used to query the dispatch log and display the query dispatch log on a visual interface to check whether the dispatch operation of the dispatcher is abnormal; The word error rate unit (72) is used to count the word error rate appearing in the output recognition text; The sentence error rate unit (73) is used to count the sentence error rates appearing in the output recognition text; The speech recognition optimization unit (74) is used to check whether there is a recognition difference between the semantics of the Mandarin dispatch text and the dialect dispatch text, and if there is a difference, calibrate the dispatch text; The calibration process of the scheduling text is: L1: Establish two machine learning text models. The first model corresponds to Mandarin, and the second model corresponds to dialects. Establish two word data tables. The first word table is the Mandarin corresponding table, and the second word table is the dialect corresponding table. L2: Collect the professional vocabulary set ai={a1, a2...an} of the power grid and select the machine learning model of the SVM vector machine; L3: Determine whether the Mandarin or dialect words of the power grid professional words A1 in speech recognition are wrong; L4: If there is a Mandarin recognition error, check whether the first word list has the word record. If not, record the power grid professional Mandarin word A1 in the first word list, and add the correct Mandarin word corresponding item to the first word list to form A(A1, B). If there is, replace the correct item corresponding to the word; If the dialect recognition is wrong, check whether the second word list has the word record, if not, record the power grid professional dialect word A1 in the second word list, add the correct dialect word corresponding item to the second word list to form A(A1, B), if yes, replace the correct item corresponding to the word; Each row of data in the word table is represented as A(A1, B), where A1 is the incorrect semantic word and B is the corresponding correct semantic word; L5: Execute the task of generating text through speech recognition. If an incorrect word appears in the dialect text or the Mandarin text, turn to step L4. Finally, all the semantics of the words are recognized correctly and a correct scheduling text is generated. The first word list and the second word list are stored in the data storage module (3).

Images