Power grid dispatching voice information checking method and system

Abstract

The invention discloses a power grid dispatching voice information checking method and system, the method is suitable for the power grid dispatching voice information checking system, and the system comprises a dispatching voice recognition module and an RAG checking module. According to the method, a Mel filter bank, a Conformer encoder, a CTC decoder and a Bi Transformer recognizer in a scheduling voice recognition module are used for recognizing and converting scheduling voice into a scheduling text, and a retrieval unit, a scheduling information knowledge base and a checking unit in an RAG checking module are used for performing knowledge retrieval and checking on the scheduling text; the scheduling voice is accurately and efficiently recognized and checked to generate the scheduling instruction text, the scheduling voice of a dispatcher is recognized and checked under the actual condition, and the problem that the scheduling instruction of the power system cannot be accurately and efficiently checked due to the fact that the scheduling voice instruction cannot be accurately and efficiently recognized in the prior art is solved. And safe and stable operation of the power grid cannot be guaranteed.

Description

Technical Field

[0001] The present invention relates to the field of voice information verification, and in particular to a method and system for verifying voice information of power grid dispatching. Background Art

[0002] Power system dispatch is the core link to ensure the safe and stable operation of the power grid. With the continuous expansion of the scale and increase of complexity of the power grid, the accuracy and timeliness of dispatch information have become particularly important. At present, some power companies have begun to try to introduce rule-based automatic verification systems. Such systems usually use preset rule bases and template matching methods to check the format and basic content of dispatch information.

[0003] However, this verification method requires dispatch information data in a standard format for verification, and cannot verify dispatch information data in a non-standard format, such as the dispatcher's voice instructions. In reality, power system dispatching by voice is a conventional method. Therefore, the verification system of the prior art cannot accurately and efficiently recognize the dispatch voice instructions, resulting in the inability to accurately and efficiently verify the power system dispatch instructions, which may lead to the inability to ensure the safe and stable operation of the power grid. Summary of the invention

[0004] The present invention provides a method and system for verifying power grid dispatching voice information, which can solve the problem that the prior art cannot accurately and efficiently identify dispatching voice instructions, resulting in the inability to accurately and efficiently verify power system dispatching instructions, which may in turn lead to the inability to ensure safe and stable operation of the power grid.

[0005] In order to solve the above technical problems, an embodiment of the present invention provides a power grid dispatching voice information verification method, which is applicable to a power grid dispatching voice information verification system, wherein the power grid dispatching voice information verification system includes a dispatching voice recognition module and a RAG verification module; the dispatching voice recognition module includes a Mel filter bank, a Conformer encoder, a CTC decoder and a BiTransformer recognizer; the RAG verification module includes a retrieval unit, a dispatching information knowledge base and a verification unit; the RAG verification module includes a retrieval unit, a dispatching information knowledge base and a verification unit; the method includes:

[0006] Obtain the power grid dispatching voice to be verified;

[0007] The Fbank feature of the power grid dispatching speech to be checked is extracted by using the Mel filter bank to obtain the Fbank feature of the power grid dispatching speech;

[0008] The Fbank features of the power grid dispatching speech are extracted through the Conformer encoder to obtain the coding features of the power grid dispatching speech;

[0009] The coding features of the power grid dispatching speech are decoded by the CTC decoder to obtain the N-best recognition result sequence;

[0010] According to the coding features of the power grid dispatching speech and the N-best recognition result sequence, the dispatching information is recognized by the BiTransformer recognizer to obtain the dispatching recognition text of the power grid dispatching speech;

[0011] According to the dispatch recognition text of the power grid dispatch voice, knowledge retrieval is performed through the retrieval unit in the RAG verification module and the dispatch information knowledge base to obtain the dispatch information knowledge text with the highest similarity to the dispatch recognition text;

[0012] According to the scheduling identification text and the scheduling information knowledge text, the scheduling information is checked by the checking unit in the RAG checking module to obtain the scheduling information checking result.

[0013] Further, the Fbank feature extraction is performed on the power grid dispatching speech to be checked by the Mel filter bank to obtain the Fbank feature of the power grid dispatching speech, including:

[0014] Convert the power grid dispatching voice to be verified into an audio format and divide the audio frames into several processed audio frames;

[0015] The Mel filter bank is used to extract Fbank features and perform cepstral mean variance normalization on each processed audio frame to obtain Fbank features of several audio frames.

[0016] Furthermore, the Conformer encoder includes a plurality of Conformer modules; each Conformer module includes a multi-head autonomous intention layer, a convolutional layer, a first feedforward layer and a second feedforward layer;

[0017] The method of extracting coding features of the Fbank features of the power grid dispatching speech by using the Conformer encoder to obtain coding features of the power grid dispatching speech includes:

[0018] The first feedforward layer and the multi-head autonomous intention layer in the Conformer module perform nonlinear transformation and global feature extraction on the Fbank features of each audio frame in turn to obtain the global features of each audio frame;

[0019] The convolutional layer and the second feed-forward layer in the Conformer module perform local feature extraction and nonlinear transformation on the global features of each audio frame in turn to obtain the encoding features of each audio frame.

[0020] Further, the CTC decoder includes a recognition word probability matrix generation module, a search algorithm module, a hot word recognition score calculation module and an N-best recognition result sequence generation module;

[0021] The coding features of the power grid dispatching speech are decoded by the CTC decoder to obtain an N-best recognition result sequence, including:

[0022] The recognition word probability matrix generation module calculates the recognition word probability matrix of the coding features of each audio frame to obtain the recognition word probability matrix of the power grid dispatching speech to be verified;

[0023] According to the recognition word probability matrix of the power grid dispatching speech to be verified, the recognition path search is performed on each audio frame through the search algorithm module to obtain several N-best candidate recognition result sequences;

[0024] The word recognition score calculation module calculates the hot word recognition score for each N-best candidate recognition result sequence to obtain the hot word recognition scores of several N-best candidate recognition result sequences;

[0025] According to the recognition word probability matrix and the hot word recognition scores of several N-best candidate recognition result sequences, the comprehensive recognition score is calculated by the preset comprehensive recognition score calculation formula in the N-best recognition result sequence generation module, and the N-best recognition result sequence is determined based on the comprehensive recognition score.

[0026] Furthermore, the preset comprehensive recognition score calculation formula is as follows:

[0027] y * =argmar(logP(y|x)+λP c (y));

[0028] Wherein, y* represents the comprehensive recognition score; P(y|x) represents the recognition word probability matrix; λ represents the hot word recognition score weight coefficient; P c (y) represents the hot word recognition score of the N-best candidate recognition result sequence.

[0029] Further, the BiTransformer recognizer includes a plurality of L2R ​​transformer modules, a plurality of R2L transformer modules and a scheduling recognition text generation module;

[0030] The method of performing dispatch information recognition by a BiTransformer recognizer according to the coding features of the power grid dispatch speech and the N-best recognition result sequence to obtain the dispatch recognition text of the power grid dispatch speech includes:

[0031] According to the coding features of the power grid dispatching speech and the N-best recognition result sequence, the L2R transformer module and the R2L transformer module in the BiTransformer recognizer are used to perform left-to-right decoding processing calculations and right-to-left decoding processing calculations, respectively, to obtain the vocabulary prediction confidence of left-to-right decoding processing and right-to-left decoding processing;

[0032] According to the vocabulary prediction confidence of the left-to-right decoding process and the right-to-left decoding process, the final recognition confidence score is calculated by the preset scheduling recognition text score calculation formula in the scheduling recognition text generation module, and the final scheduling recognition text is determined based on the final recognition confidence score.

[0033] Furthermore, the preset scheduling recognition text score calculation formula is:

[0034] score=L2R_score*(1-reverse_weight)+R2L_score*reverse_weight+ctc_scores*ctc_weight;

[0035] Among them, the score represents the final recognition confidence score; the L2R_score represents the vocabulary prediction confidence of the left-to-right decoding process; the R2L_score represents the vocabulary prediction confidence of the right-to-left decoding process; the reverse_weight represents the preset balance weight of the decoding process from left to right and from right to left; the ctc_scores represents the preset decoding score of the CTC decoder; the ctc_weight represents the preset decoding weight of the CTC decoder.

[0036] Further, the scheduling information is checked by the checking unit in the RAG checking module according to the scheduling identification text and the scheduling information knowledge text to obtain the scheduling information checking result, including:

[0037] When the scheduling information verification result is wrong, the language model of the verification unit in the RAG verification module is used to correct the scheduling strategy of the scheduling recognition text according to the scheduling information knowledge text to obtain the corrected scheduling strategy, and the scheduling information verification result and the corrected scheduling strategy are fed back to the scheduling center;

[0038] When the dispatch information verification result is correct, the dispatch information verification result and the dispatch identification text are fed back to the dispatch center.

[0039] Based on the above method embodiment, the present invention provides a corresponding system embodiment;

[0040] An embodiment of the present invention provides a power grid dispatching voice information verification system, comprising: a power grid dispatching voice acquisition module, a dispatching voice recognition module and a RAG verification module; the dispatching voice recognition module comprises a Mel filter bank, a Conformer encoder, a CTC decoder and a BiTransformer recognizer; the RAG verification module comprises a retrieval unit, a dispatching information knowledge base and a verification unit; the RAG verification module comprises a retrieval unit, a dispatching information knowledge base and a verification unit;

[0041] The power grid dispatching voice acquisition module is used to acquire the power grid dispatching voice to be verified;

[0042] The Mel filter bank is used to perform Fbank feature extraction on the power grid dispatching speech to be checked to obtain the Fbank features of the power grid dispatching speech;

[0043] The Conformer encoder is used to extract coding features from the Fbank features of the power grid dispatching speech to obtain coding features of the power grid dispatching speech;

[0044] The CTC decoder is used to decode the coding features of the power grid dispatching speech to obtain an N-best recognition result sequence;

[0045] The BiTransformer recognizer is used to recognize the dispatch information according to the coding features of the power grid dispatch speech and the N-best recognition result sequence, and obtain the dispatch recognition text of the power grid dispatch speech;

[0046] The retrieval unit and the dispatch information knowledge base are used to perform knowledge retrieval based on the dispatch recognition text of the power grid dispatch voice to obtain the dispatch information knowledge text with the highest similarity to the dispatch recognition text;

[0047] The verification unit is used to verify the scheduling information according to the scheduling identification text and the scheduling information knowledge text to obtain the scheduling information verification result.

[0048] Further, the CTC decoder includes a recognition word probability matrix generation module, a search algorithm module, a hot word recognition score calculation module and an N-best recognition result sequence generation module;

[0049] The recognition word probability matrix generation module is used to calculate the recognition word probability matrix for the coding features of each audio frame to obtain the recognition word probability matrix of the power grid dispatching speech to be verified;

[0050] The search algorithm module is used to search for recognition paths for each audio frame according to the recognition word probability matrix of the power grid dispatching speech to be verified, and obtain a plurality of N-best candidate recognition result sequences;

[0051] The hot word recognition score calculation module is used to calculate the hot word recognition score of each N-best candidate recognition result sequence to obtain the hot word recognition scores of several N-best candidate recognition result sequences;

[0052] The N-best recognition result sequence generation module is used to calculate the comprehensive recognition score according to the recognition word probability matrix and the hot word recognition scores of several N-best candidate recognition result sequences, and determine the N-best recognition result sequence according to the comprehensive recognition score.

[0053] Compared with the prior art, the embodiments of the present invention have the following beneficial effects:

[0054] The present invention performs Fbank feature extraction on the dispatching speech through a Mel filter group, then uses a Conformer encoder to extract coding features from the Fbank features, then uses a CTC decoder to perform Fbank feature extraction and coding feature extraction decoding processing on the coding features to obtain an N-best recognition result sequence, and finally uses a BiTransformer recognizer to perform dispatching information recognition according to the coding features and the N-best recognition result sequence to obtain a dispatching text recognition result of the power grid dispatching speech, and according to the dispatching recognition text, searches are performed from a dispatching information knowledge base through a retrieval unit in a RAG verification module to obtain a dispatching information knowledge text with the highest similarity to the dispatching recognition text, and uses the verification unit in the RAG verification module to verify and judge the dispatching recognition text according to the dispatching information knowledge text to obtain a dispatching information verification result. That is, the present invention converts the dispatch instructions from voice format to text format through the Mel filter group, Conformer encoder, CTC decoder and BiTransformer recognizer in the dispatch speech recognition module, and performs dispatch knowledge retrieval on the dispatch instructions in the dispatch text, and finally verifies the dispatch instructions in the dispatch text according to the retrieved dispatch knowledge, thereby achieving accurate and efficient recognition of the dispatch instructions in the dispatch voice and generating the dispatch text, and performing retrieval and verification processing on the dispatch text through the RAG technology, thereby achieving recognition and verification of the dispatch voice of the dispatcher under actual circumstances, solving the problem that the prior art cannot accurately and efficiently recognize the dispatch voice instructions, resulting in the inability to accurately and efficiently verify the dispatch instructions of the power system, which may further lead to the inability to ensure the safe and stable operation of the power grid. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] Figure 1 : A flowchart of a method for verifying voice information of power grid dispatching provided by an embodiment of the present invention;

[0056] Figure 2 : A system structure diagram of a power grid dispatching voice information verification system provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0057] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0058] In the description of the present invention, it should be understood that the terms “first” and “second” are only used for descriptive purposes and cannot be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features.

[0059] Embodiment 1:

[0060] Reference Figure 1 , is a flowchart of a method for verifying voice information of power grid dispatching provided by an embodiment of the present invention, the method is applicable to a power grid dispatching voice information verification system, the system includes a dispatching voice recognition module and a RAG verification module; the dispatching voice recognition module includes a Mel filter bank, a Conformer encoder, a CTC decoder and a BiTransformer recognizer; the RAG verification module includes a retrieval unit, a dispatching information knowledge base and a verification unit; the RAG verification module includes a retrieval unit, a dispatching information knowledge base and a verification unit; and the method includes at least the following steps:

[0061] Step S1: Acquire the power grid dispatching voice to be verified;

[0062] In this embodiment, the power grid dispatching voice to be verified can be obtained from the dispatching records of the power system dispatching center.

[0063] Step S2: performing Fbank feature extraction on the power grid dispatching speech to be checked through a Mel filter bank to obtain the Fbank features of the power grid dispatching speech;

[0064] In this embodiment, the Fbank feature extraction of the power grid dispatching speech to be checked by the Mel filter bank to obtain the Fbank feature of the power grid dispatching speech includes:

[0065] Convert the power grid dispatching voice to be verified into an audio format and divide the audio frames into several processed audio frames;

[0066] Exemplarily, the power grid dispatching voice audio to be verified is converted into a mono audio format with a sampling rate of 16000 Hz, and the audio after the audio format conversion is divided into audio frames according to a 25 ms frame length and a 10 ms frame shift to obtain a plurality of processed audio frames;

[0067] The Fbank feature extraction and cepstral mean variance normalization processing are performed on each processed audio frame through the Mel filter group to obtain Fbank features of several audio frames; wherein the number of Mel filters of the Mel filter group can be set according to actual conditions; illustratively, the number of Mel filters can be set to 80.

[0068] Step S3: extracting coding features of the Fbank features of the power grid dispatching speech through a Conformer encoder to obtain coding features of the power grid dispatching speech;

[0069] In this embodiment, the Conformer encoder includes a plurality of Conformer modules; each Conformer module includes a multi-head autonomous intention layer, a convolutional layer, a first feedforward layer and a second feedforward layer;

[0070] Exemplarily, the Conformer encoder may be provided with 12 Conformer modules; the first feedforward layer and the second feedforward layer may be provided with 2048 linear units; the first feedforward layer and the second feedforward layer further include a nonlinear activation function; the number of the Conformer modules may be provided according to actual conditions; the number of the linear units of the first feedforward layer and the second feedforward layer may be provided according to actual conditions;

[0071] The method of extracting coding features of the Fbank features of the power grid dispatching speech by using the Conformer encoder to obtain coding features of the power grid dispatching speech includes:

[0072] The first feedforward layer and the multi-head autonomous intention layer in the Conformer module perform nonlinear transformation and global feature extraction on the Fbank features of each audio frame in turn to obtain the global features of each audio frame;

[0073] The convolutional layer and the second feed-forward layer in the Conformer module perform local feature extraction and nonlinear transformation on the global features of each audio frame in turn to obtain the encoding features of each audio frame.

[0074] In this embodiment, the Conformer module also includes a normalization layer; the multi-head autonomous intention layer, convolution layer, first feedforward layer and second feedforward layer are respectively connected to the normalization layer; the normalization layer is used to normalize the outputs of the multi-head autonomous intention layer, convolution layer, first feedforward layer and second feedforward layer respectively.

[0075] Step S4: decoding the coding features of the power grid dispatching speech through the CTC decoder to obtain an N-best recognition result sequence;

[0076] In this embodiment, the CTC decoder includes a recognition word probability matrix generation module, a search algorithm module, a hot word recognition score calculation module, and an N-best recognition result sequence generation module;

[0077] The coding features of the power grid dispatching speech are decoded by the CTC decoder to obtain an N-best recognition result sequence, including:

[0078] The recognition word probability matrix generation module calculates the recognition word probability matrix of the coding features of each audio frame to obtain the recognition word probability matrix of the power grid dispatching speech to be verified;

[0079] In this embodiment, according to the coding features of each audio frame, the recognition probability of each character in the dictionary corresponding to each audio is calculated to form a recognition word probability matrix. Each column of the matrix represents a time frame, and each row corresponds to a character in the dictionary, including blank characters;

[0080] According to the recognition word probability matrix of the power grid dispatching speech to be verified, the recognition path search is performed on each audio frame through the search algorithm module to obtain several N-best candidate recognition result sequences;

[0081] The word recognition score calculation module calculates the hot word recognition score for each N-best candidate recognition result sequence to obtain the hot word recognition scores of several N-best candidate recognition result sequences;

[0082] Exemplarily, when a preset power grid dispatch hot word is included in a certain N-best candidate recognition result sequence, a hot word recognition score is assigned to the N-best candidate recognition result sequence, and finally the total hot word recognition score of the N-best candidate recognition result sequence is counted;

[0083] According to the recognition word probability matrix and the hot word recognition scores of several N-best candidate recognition result sequences, a comprehensive recognition score is calculated by a preset comprehensive recognition score calculation formula in the N-best recognition result sequence generation module, and according to the comprehensive recognition score, an N-best recognition result sequence is determined;

[0084] In this embodiment, the preset comprehensive recognition score calculation formula is:

[0085] y * =argmax(logP(y|x)+λP c (y));

[0086] Wherein, y* represents the comprehensive recognition score; P(y|x) represents the recognition word probability matrix; λ represents the hot word recognition score weight coefficient; P c(y) represents the hot word recognition score of the N-best candidate recognition result sequence.

[0087] Step S5: According to the coding features of the power grid dispatching speech and the N-best recognition result sequence, the dispatching information is recognized by the BiTransformer recognizer to obtain the dispatching recognition text of the power grid dispatching speech;

[0088] In this embodiment, the BiTransformer recognizer includes a number of L2R ​​transformer modules, a number of R2L transformer modules and a scheduling recognition text generation module;

[0089] In this embodiment, the number and parameters of the plurality of L2R ​​transformer modules and the plurality of R2L transformer modules can be set according to actual conditions; illustratively, the number of the plurality of L2R ​​transformer modules and the plurality of R2L transformer modules are both set to 3, and the number of attention heads in each transformer module can be set to 8, the number of linear units can be set to 2048, and the dropout_rate can be set to 0.1;

[0090] The method of performing dispatch information recognition by a BiTransformer recognizer according to the coding features of the power grid dispatch speech and the N-best recognition result sequence to obtain the dispatch recognition text of the power grid dispatch speech includes:

[0091] According to the coding features of the power grid dispatching speech and the N-best recognition result sequence, the L2R transformer module and the R2L transformer module in the BiTransformer recognizer are used to perform left-to-right decoding processing calculations and right-to-left decoding processing calculations, respectively, to obtain the vocabulary prediction confidence of left-to-right decoding processing and right-to-left decoding processing;

[0092] In this embodiment, the coding features of the power grid dispatching speech and the N-best recognition result sequence are respectively input into the L2R transformer module and the R2L transformer module in the BiTransformer recognizer. When the L2R transformer module performs left-to-right decoding processing calculation on the coding features of the power grid dispatching speech and the N-best recognition result sequence, the R2L transformer module performs right-to-left decoding processing calculation on the coding features of the power grid dispatching speech and the N-best recognition result sequence, and finally obtains the vocabulary prediction confidence of the left-to-right decoding processing and the vocabulary prediction confidence of the right-to-left decoding processing;

[0093] According to the vocabulary prediction confidence of the left-to-right decoding process and the right-to-left decoding process, the final recognition confidence score is calculated by the preset scheduling recognition text score calculation formula in the scheduling recognition text generation module, and the final scheduling recognition text is determined based on the final recognition confidence score.

[0094] In this embodiment, the preset scheduling recognition text score calculation formula is:

[0095] score=L2R_score*(1-reverse_weight)+R2L_score*reverse_weight+ctc_scores*ctc_weight;

[0096] Among them, the score represents the final recognition confidence score; the L2R_score represents the vocabulary prediction confidence of the left-to-right decoding process; the R2L_score represents the vocabulary prediction confidence of the right-to-left decoding process; the reverse_weight represents the preset balance weight of the decoding process from left to right and from right to left; the ctc_scores represents the preset decoding score of the CTC decoder; the ctc_weight represents the preset decoding weight of the CTC decoder.

[0097] Step S6: Based on the dispatch recognition text of the power grid dispatch voice, knowledge retrieval is performed through the retrieval unit in the RAG verification module and the dispatch information knowledge base to obtain the dispatch information knowledge text with the highest similarity to the dispatch recognition text;

[0098] In this embodiment, the rules and regulations, operating procedures, equipment manuals and historical dispatch records of power grid dispatch are obtained, and the rules and regulations, operating procedures, equipment manuals and historical dispatch records of power grid dispatch can be text-encoded and text-vectorized using BERT technology, and the processed power grid dispatch knowledge text is stored in a database, thereby constructing a dispatch information knowledge base.

[0099] In this embodiment, the dispatch recognition text according to the power grid dispatch voice is retrieved through the retrieval unit in the RAG verification module and the dispatch information knowledge base to obtain the dispatch information knowledge text with the highest similarity to the dispatch recognition text, including:

[0100] The dispatch recognition text of the power grid dispatch speech can be vectorized by using BERT or GPT technology to generate a dispatch recognition text semantic vector;

[0101] According to the semantic vector of the dispatch recognition text, a similarity search is performed from the dispatch information knowledge base, and then the dispatch information knowledge text with the highest similarity to the dispatch recognition text is obtained.

[0102] Step S7: Based on the scheduling identification text and the scheduling information knowledge text, the scheduling information is verified by the verification unit in the RAG verification module to obtain the scheduling information verification result.

[0103] In this embodiment, the scheduling information is checked by the checking unit in the RAG checking module according to the scheduling identification text and the scheduling information knowledge text to obtain the scheduling information checking result, including:

[0104] When the scheduling information verification result is wrong, the language model of the verification unit in the RAG verification module is used to correct the scheduling strategy of the scheduling recognition text according to the scheduling information knowledge text to obtain the corrected scheduling strategy, and the scheduling information verification result and the corrected scheduling strategy are fed back to the scheduling center;

[0105] When the dispatch information verification result is correct, the dispatch information verification result and the dispatch identification text are fed back to the dispatch center.

[0106] Embodiment 2:

[0107] Reference Figure 2 , is a system structure diagram of a power grid dispatching voice information verification system provided by an embodiment of the present invention, the system at least includes: a power grid dispatching voice acquisition module, a dispatching voice recognition module and a RAG verification module; the dispatching voice recognition module includes a Mel filter bank, a Conformer encoder, a CTC decoder and a BiTransformer recognizer; the RAG verification module includes a retrieval unit, a dispatching information knowledge base and a verification unit; the RAG verification module includes a retrieval unit, a dispatching information knowledge base and a verification unit;

[0108] The power grid dispatching voice acquisition module is used to acquire the power grid dispatching voice to be verified;

[0109] The Mel filter bank is used to perform Fbank feature extraction on the power grid dispatching speech to be checked to obtain the Fbank features of the power grid dispatching speech;

[0110] The Conformer encoder is used to extract coding features from the Fbank features of the power grid dispatching speech to obtain coding features of the power grid dispatching speech;

[0111] The CTC decoder is used to decode the coding features of the power grid dispatching speech to obtain an N-best recognition result sequence;

[0112] The BiTransformer recognizer is used to recognize the dispatch information according to the coding features of the power grid dispatch speech and the N-best recognition result sequence, and obtain the dispatch recognition text of the power grid dispatch speech;

[0113] The retrieval unit and the dispatch information knowledge base are used to perform knowledge retrieval based on the dispatch recognition text of the power grid dispatch voice to obtain the dispatch information knowledge text with the highest similarity to the dispatch recognition text;

[0114] The verification unit is used to verify the scheduling information according to the scheduling identification text and the scheduling information knowledge text to obtain the scheduling information verification result.

[0115] In this embodiment, the CTC decoder includes a recognition word probability matrix generation module, a search algorithm module, a hot word recognition score calculation module, and an N-best recognition result sequence generation module;

[0116] The recognition word probability matrix generation module is used to calculate the recognition word probability matrix for the coding features of each audio frame to obtain the recognition word probability matrix of the power grid dispatching speech to be verified;

[0117] The search algorithm module is used to search for recognition paths for each audio frame according to the recognition word probability matrix of the power grid dispatching speech to be verified, and obtain a plurality of N-best candidate recognition result sequences;

[0118] The hot word recognition score calculation module is used to calculate the hot word recognition score of each N-best candidate recognition result sequence to obtain the hot word recognition scores of several N-best candidate recognition result sequences;

[0119] The N-best recognition result sequence generation module is used to calculate the comprehensive recognition score according to the recognition word probability matrix and the hot word recognition scores of several N-best candidate recognition result sequences, and determine the N-best recognition result sequence according to the comprehensive recognition score.

[0120] The specific embodiments described above further illustrate the purpose, technical solutions and beneficial effects of the present invention. It should be understood that the above description is only a specific embodiment of the present invention and is not intended to limit the scope of protection of the present invention. It is particularly pointed out that for those skilled in the art, any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention should be included in the scope of protection of the present invention.

Claims

1. A method for verifying voice information of power grid dispatching, characterized in that: Applicable to the power grid dispatching voice information verification system, the power grid dispatching voice information verification system includes a dispatching voice recognition module and a RAG verification module; the dispatching voice recognition module includes a Mel filter bank, a Conformer encoder, a CTC decoder and a BiTransformer recognizer; the RAG verification module includes a retrieval unit, a dispatching information knowledge base and a verification unit; The RAG verification module includes a retrieval unit, a scheduling information knowledge base and a verification unit; The power grid dispatching voice information verification method comprises: Obtain the power grid dispatching voice to be verified; The Fbank feature of the power grid dispatching speech to be checked is extracted by using the Mel filter bank to obtain the Fbank feature of the power grid dispatching speech; The Fbank features of the power grid dispatching speech are extracted through the Conformer encoder to obtain the coding features of the power grid dispatching speech; The coding features of the power grid dispatching speech are decoded by the CTC decoder to obtain the N-best recognition result sequence; According to the coding features of the power grid dispatching speech and the N-best recognition result sequence, the dispatching information is recognized by the BiTransformer recognizer to obtain the dispatching recognition text of the power grid dispatching speech; According to the dispatch recognition text of the power grid dispatch voice, knowledge retrieval is performed through the retrieval unit in the RAG verification module and the dispatch information knowledge base to obtain the dispatch information knowledge text with the highest similarity to the dispatch recognition text; According to the scheduling identification text and the scheduling information knowledge text, the scheduling information is checked by the checking unit in the RAG checking module to obtain the scheduling information checking result.

2. A method for verifying power grid dispatching voice information according to claim 1, characterized in that: The Fbank feature extraction of the power grid dispatching speech to be checked is performed by the Mel filter bank to obtain the Fbank feature of the power grid dispatching speech, including: Convert the power grid dispatching voice to be verified into an audio format and divide the audio frames into several processed audio frames; The Mel filter bank is used to extract Fbank features and perform cepstral mean variance normalization on each processed audio frame to obtain Fbank features of several audio frames.

3. A method for verifying power grid dispatching voice information according to claim 2, characterized in that: The Conformer encoder includes a plurality of Conformer modules; each of the Conformer modules includes a multi-head autonomous intention layer, a convolutional layer, a first feedforward layer and a second feedforward layer; The method of extracting coding features of the Fbank features of the power grid dispatching speech by using the Conformer encoder to obtain coding features of the power grid dispatching speech includes: The first feedforward layer and the multi-head autonomous intention layer in the Conformer module perform nonlinear transformation and global feature extraction on the Fbank features of each audio frame in turn to obtain the global features of each audio frame; The convolutional layer and the second feed-forward layer in the Conformer module perform local feature extraction and nonlinear transformation on the global features of each audio frame in turn to obtain the encoding features of each audio frame.

4. A method for verifying power grid dispatching voice information according to claim 3, characterized in that: The CTC decoder includes a recognition word probability matrix generation module, a search algorithm module, a hot word recognition score calculation module, and an N-best recognition result sequence generation module; The coding features of the power grid dispatching speech are decoded by the CTC decoder to obtain an N-best recognition result sequence, including: The recognition word probability matrix generation module calculates the recognition word probability matrix of the coding features of each audio frame to obtain the recognition word probability matrix of the power grid dispatching speech to be verified; According to the recognition word probability matrix of the power grid dispatching speech to be verified, the recognition path search is performed on each audio frame through the search algorithm module to obtain several N-best candidate recognition result sequences; The word recognition score calculation module calculates the hot word recognition score for each N-best candidate recognition result sequence to obtain the hot word recognition scores of several N-best candidate recognition result sequences; According to the recognition word probability matrix and the hot word recognition scores of several N-best candidate recognition result sequences, the comprehensive recognition score is calculated by the preset comprehensive recognition score calculation formula in the N-best recognition result sequence generation module, and the N-best recognition result sequence is determined based on the comprehensive recognition score.

5. A method for verifying power grid dispatching voice information according to claim 4, characterized in that: The preset comprehensive recognition score calculation formula is as follows: and * =argmax(logP(y|x)+λP c (and)); Wherein, y* represents the comprehensive recognition score; P(y|x) represents the recognition word probability matrix; λ represents the hot word recognition score weight coefficient; P c (y) represents the hot word recognition score of the N-best candidate recognition result sequence.

6. A method for verifying power grid dispatching voice information according to claim 5, characterized in that: The BiTransformer recognizer includes a plurality of L2R ​​transformer modules, a plurality of R2L transformer modules and a scheduling recognition text generation module; The method of performing dispatch information recognition by a BiTransformer recognizer according to the coding features of the power grid dispatch speech and the N-best recognition result sequence to obtain the dispatch recognition text of the power grid dispatch speech includes: According to the coding features of the power grid dispatching speech and the N-best recognition result sequence, the L2R transformer module and the R2L transformer module in the BiTransformer recognizer are used to perform left-to-right decoding processing calculations and right-to-left decoding processing calculations, respectively, to obtain the vocabulary prediction confidence of left-to-right decoding processing and right-to-left decoding processing; According to the vocabulary prediction confidence of the left-to-right decoding process and the right-to-left decoding process, the final recognition confidence score is calculated by the preset scheduling recognition text score calculation formula in the scheduling recognition text generation module, and the final scheduling recognition text is determined based on the final recognition confidence score.

7. A method for verifying power grid dispatching voice information according to claim 6, characterized in that: The preset scheduling recognition text score calculation formula is: score=L2R_score*(1-reverse_weight)+R2L_score*reverse_weight+ctc_scores*ctc_weight; Among them, the score represents the final recognition confidence score; the L2R_score represents the vocabulary prediction confidence of the left-to-right decoding process; the R2L_score represents the vocabulary prediction confidence of the right-to-left decoding process; the reverse_weight represents the preset balance weight of the decoding process from left to right and from right to left; the ctc_scores represents the preset decoding score of the CTC decoder; the ctc_weight represents the preset decoding weight of the CTC decoder.

8. A method for verifying power grid dispatching voice information according to claim 7, characterized in that: According to the scheduling identification text and the scheduling information knowledge text, the scheduling information is checked by the checking unit in the RAG checking module to obtain the scheduling information checking result, including: When the scheduling information verification result is wrong, the language model of the verification unit in the RAG verification module is used to correct the scheduling strategy of the scheduling recognition text according to the scheduling information knowledge text to obtain the corrected scheduling strategy, and the scheduling information verification result and the corrected scheduling strategy are fed back to the scheduling center; When the dispatch information verification result is correct, the dispatch information verification result and the dispatch identification text are fed back to the dispatch center.

9. A power grid dispatching voice information verification system, characterized in that: include: Power grid dispatching voice acquisition module, dispatching voice recognition module and RAG verification module; The dispatch speech recognition module includes a Mel filter bank, a Conformer encoder, a CTC decoder and a BiTransformer recognizer; the RAG verification module includes a retrieval unit, a dispatch information knowledge base and a verification unit; the RAG verification module includes a retrieval unit, a dispatch information knowledge base and a verification unit; The power grid dispatching voice acquisition module is used to acquire the power grid dispatching voice to be verified; The Mel filter bank is used to perform Fbank feature extraction on the power grid dispatching speech to be checked to obtain the Fbank features of the power grid dispatching speech; The Conformer encoder is used to extract coding features from the Fbank features of the power grid dispatching speech to obtain coding features of the power grid dispatching speech; The CTC decoder is used to decode the coding features of the power grid dispatching speech to obtain an N-best recognition result sequence; The BiTransformer recognizer is used to recognize the dispatch information according to the coding features of the power grid dispatch speech and the N-best recognition result sequence, and obtain the dispatch recognition text of the power grid dispatch speech; The retrieval unit and the dispatch information knowledge base are used to perform knowledge retrieval based on the dispatch recognition text of the power grid dispatch voice to obtain the dispatch information knowledge text with the highest similarity to the dispatch recognition text; The verification unit is used to verify the scheduling information according to the scheduling identification text and the scheduling information knowledge text to obtain the scheduling information verification result.

10. A power grid dispatching voice information verification system according to claim 9, characterized in that: The CTC decoder includes a recognition word probability matrix generation module, a search algorithm module, a hot word recognition score calculation module, and an N-best recognition result sequence generation module; The recognition word probability matrix generation module is used to calculate the recognition word probability matrix for the coding features of each audio frame to obtain the recognition word probability matrix of the power grid dispatching speech to be verified; The search algorithm module is used to search for recognition paths for each audio frame according to the recognition word probability matrix of the power grid dispatching speech to be verified, and obtain a plurality of N-best candidate recognition result sequences; The hot word recognition score calculation module is used to calculate the hot word recognition score of each N-best candidate recognition result sequence to obtain the hot word recognition scores of several N-best candidate recognition result sequences; The N-best recognition result sequence generation module is used to calculate the comprehensive recognition score according to the recognition word probability matrix and the hot word recognition scores of several N-best candidate recognition result sequences, and determine the N-best recognition result sequence according to the comprehensive recognition score.

Images