An automatic speech recognition method and system for power grid communication scheduling

Abstract

The application provides an automatic speech recognition method and system for power grid communication scheduling, and relates to the technical field of speech recognition. The method comprises the following steps: obtaining to-be-recognized speech data for power grid communication scheduling and performing segmentation processing to obtain a plurality of to-be-recognized speech paragraphs and inputting the to-be-recognized speech paragraphs into a first speech recognition model to generate a first speech recognition result; calculating a complexity feature factor of each to-be-recognized speech paragraph, and dividing the plurality of to-be-recognized speech paragraphs into first difficulty paragraphs and second difficulty paragraphs; extracting a plurality of groups of sub-paragraph speech data from the first difficulty paragraphs, inputting the plurality of groups of sub-paragraph speech data and the to-be-recognized speech paragraphs of the second difficulty paragraphs into a second speech recognition model to generate a second speech recognition result, and performing feature fusion on the first speech recognition result and the second speech recognition result to obtain a target speech recognition result of the to-be-recognized speech data. The application realizes the improvement of the recognition efficiency of power grid communication scheduling speech.

Description

Technical Field

[0001] This invention relates to the field of speech recognition technology, and in particular to an automatic speech recognition method and system for power grid communication dispatch. Background Technology

[0002] In modern power system operation, communication dispatch plays a crucial role, ensuring the stable and efficient operation of the power grid. Voice applications in the power sector are complex, involving diverse scenarios, noisy working environments, information density, and high security requirements. Telephone voice communication, as an important dispatch and command communication method, heavily relies on the power communication dispatch voice system for issuing instructions and reporting on-site situations. Due to the special nature of power grid data, which requires extremely high security and reliability, internal information communication cannot be conducted via the internet but only through the power intranet. Dispatch instructions are also issued via dispatch telephones, and dispatch recordings are stored in intranet devices. This audio-based data storage format greatly hinders information retrieval, making recording retrieval time-consuming and labor-intensive. In this context, voice recognition technology is particularly important. Converting relevant voice data into text data through voice recognition technology can significantly shorten data retrieval time and improve information retrieval efficiency.

[0003] While some commonly used speech recognition technologies have achieved good results in general speech processing, in the field of power grid communication and dispatching, due to the large number of specialized terms involved, which appear less frequently in general corpora, the recognition accuracy of some general models for these specialized terms still needs improvement. While some models that can be used to recognize specialized terms can specifically identify certain terms, they require more computational resources and take longer. Summary of the Invention

[0004] To address at least one of the problems existing in the aforementioned background technology, this invention proposes an automatic speech recognition method and system for power grid communication dispatch. It adopts an innovative dual-model strategy, integrating a general model and a professional model, thereby improving the efficiency and accuracy of traditional speech recognition systems in professional fields. It demonstrates significant advantages in processing speech data containing professional terminology, effectively improving the recognition efficiency of power grid communication dispatch speech.

[0005] The first aspect of this invention provides an automatic speech recognition method for power grid communication dispatch, comprising:

[0006] Acquire the voice data to be identified for power grid communication scheduling, and segment the voice data to be identified to obtain multiple voice segments to be identified;

[0007] Multiple speech segments to be recognized are input into the first speech recognition model to generate the first speech recognition result. The first speech recognition result includes the recognized text of each speech segment to be recognized, as well as the confidence score and timestamp corresponding to each word in each recognized text.

[0008] The complexity feature factor of each speech segment to be identified is calculated based on the first recognition result. Multiple speech segments to be identified are classified based on a preset complexity threshold, including marking speech segments to be identified with a complexity feature factor less than the preset complexity threshold as first difficulty segments and marking speech segments to be identified with a complexity feature factor not less than the preset complexity threshold as second difficulty segments.

[0009] Multiple sets of sub-segment speech data are extracted from the first difficulty segment. The multiple sets of sub-segment speech data and the speech segment to be recognized from the second difficulty segment are input into the second speech recognition model to generate the second speech recognition result. Each set of sub-segment speech data corresponds to a word in the first difficulty segment whose confidence score is less than the first preset confidence threshold. The first speech recognition result and the second speech recognition result are fused to obtain the target speech recognition result of the speech data to be recognized.

[0010] Preferably, the complexity feature factor of each speech segment to be recognized is calculated based on the first recognition result, including:

[0011] The first complexity parameter is obtained by calculating the proportion of words with confidence scores less than the first preset confidence threshold in each speech segment to be recognized based on the first recognition result.

[0012] Words with confidence scores lower than the second preset confidence threshold in each speech segment to be identified are selected. Multiple target words are determined from all selected words based on the power grid communication scheduling vocabulary library. The proportion of target words in each speech segment to be identified is calculated to obtain the second complexity parameter.

[0013] The complexity feature factor of each speech segment to be recognized is calculated based on the following formula:

[0014]

[0015] In the formula, C represents the complexity feature factor of the speech segment to be recognized, M1 represents the first complexity parameter of the speech segment to be recognized, M2 represents the second complexity parameter of the speech segment to be recognized, and μ, These are the first weight parameter and the second weight parameter, respectively.

[0016] Preferably, the second speech recognition model further includes:

[0017] A second speech recognition model was constructed based on the Transformer model architecture, and the second speech recognition model was trained using a training dataset, which included multiple sets of power grid communication scheduling speech data and the transcribed text of each set of power grid communication scheduling speech data.

[0018] During the training process of the second speech recognition model, multiple sets of power grid communication scheduling speech data are converted into Mel spectrograms and used as input to the second speech recognition model. The transcribed text of each set of power grid communication scheduling speech data is used as the training target of the second speech recognition model. The cross-entropy loss function is used to calculate the difference between the predicted value and the true value of the second speech recognition model.

[0019] Preferably, multiple target words are determined from all selected words based on the power grid communication dispatch vocabulary library, including:

[0020] The edit distance between each selected word and multiple professional words in the power grid communication and dispatch terminology library is calculated. For any selected word, if there is at least one professional word in the power grid communication and dispatch terminology library with an edit distance greater than the preset similarity threshold, the word is marked as a target word. All selected words are traversed to obtain multiple target words.

[0021] Preferably, multiple sets of sub-segment speech data are extracted from the first difficulty segment, and the multiple sets of sub-segment speech data and the speech segment to be recognized from the second difficulty segment are input into the second speech recognition model to generate a second speech recognition result, including:

[0022] Based on the timestamp of each word in the first speech recognition result, extract the sub-segment speech data corresponding to each word in the first difficulty segment whose confidence score is less than the first preset confidence threshold. Input the multiple sets of sub-segment speech data and the speech segment to be recognized in the second difficulty segment into the second speech recognition model to generate the second speech recognition result, including the first recognition result of the multiple sets of sub-segment speech data and the second recognition result of the speech segment to be recognized in the second difficulty segment.

[0023] The first recognition result includes the recognized text of each group of sub-segment speech data and the confidence score corresponding to the recognized text. The second recognition result includes the recognized text of each speech segment to be recognized in the multiple speech segments to be recognized in the second difficulty segment, as well as the confidence score and timestamp corresponding to each word in each recognized text.

[0024] Preferably, the first speech recognition result and the second speech recognition result are fused to obtain the target speech recognition result of the speech data to be recognized, including:

[0025] For multiple sets of sub-segment speech data in the first recognition result, compare the confidence scores of the recognized text in the first recognition result with the confidence scores of the recognized text in the first speech recognition result, and select the speech text with the higher confidence score as the target recognition text of the sub-segment speech data.

[0026] For multiple speech segments to be recognized in the second difficulty level, multiple sets of associated recognition texts that are related in the first speech recognition result and the second recognition result are determined based on multiple timestamps. The target recognition text of multiple speech segments to be recognized in the second difficulty level is determined based on the recognition text with higher confidence score in each set of associated recognition texts.

[0027] The multiple recognition texts corresponding to the speech data other than the multiple groups of sub-segments in the first recognition result are marked as target recognition texts, thus obtaining the target speech recognition result of the speech data to be recognized.

[0028] Preferably, multiple speech segments to be recognized are sampled at a first sampling frequency and fed into a first speech recognition model, and multiple sets of sub-segment speech data and speech segments to be recognized in a second difficulty segment are sampled at a second sampling frequency and fed into a second speech recognition model.

[0029] A second aspect of the present invention provides an automatic speech recognition system for power grid communication dispatching, the system being used to implement the above-described automatic speech recognition method for power grid communication dispatching, comprising:

[0030] The voice data preprocessing module is used to acquire the voice data to be recognized for power grid communication scheduling, and to segment the voice data to be recognized to obtain multiple voice segments to be recognized.

[0031] The first speech processing module is used to input multiple speech segments to be recognized into the first speech recognition model and generate the first speech recognition result. The first speech recognition result includes the recognized text of each speech segment to be recognized, as well as the confidence score and timestamp corresponding to each word in each recognized text.

[0032] The speech segment classification module is used to calculate the complexity feature factor of each speech segment to be identified based on the first recognition result, and classify multiple speech segments to be identified based on a preset complexity threshold, including marking speech segments to be identified with a complexity feature factor less than the preset complexity threshold as first difficulty segments, and marking speech segments to be identified with a complexity feature factor not less than the preset complexity threshold as second difficulty segments.

[0033] The second speech processing module extracts multiple sets of sub-segment speech data from the first difficulty segment, inputs the multiple sets of sub-segment speech data and the speech segment to be recognized from the second difficulty segment into the second speech recognition model, and generates the second speech recognition result.

[0034] The recognition result fusion module is used to fuse the features of the first speech recognition result and the second speech recognition result to obtain the target speech recognition result of the speech data to be recognized.

[0035] Preferably, the second speech recognition model used by the second speech processing module further includes:

[0036] A second speech recognition model was constructed based on the Transformer model architecture, and the second speech recognition model was trained using a training dataset, which included multiple sets of power grid communication scheduling speech data and the transcribed text of each set of power grid communication scheduling speech data.

[0037] Preferably, the speech segment classification module calculates a complexity feature factor for each speech segment to be identified based on the first recognition result, including:

[0038] The first complexity parameter is obtained by calculating the proportion of words with confidence scores less than the first preset confidence threshold in each speech segment to be recognized based on the first recognition result.

[0039] Words with confidence scores lower than the second preset confidence threshold in each speech segment to be identified are selected. Multiple target words are determined from all selected words based on the power grid communication scheduling vocabulary library. The proportion of target words in each speech segment to be identified is calculated to obtain the second complexity parameter.

[0040] The complexity feature factor of each speech segment to be recognized is calculated based on the following formula:

[0041]

[0042] In the formula, C represents the complexity feature factor of the speech segment to be recognized, M1 represents the first complexity parameter of the speech segment to be recognized, M2 represents the second complexity parameter of the speech segment to be recognized, and μ, These are the first weight parameter and the second weight parameter, respectively.

[0043] The present invention has the following beneficial effects:

[0044] This invention employs a lightweight, general-purpose speech recognition model to perform preliminary recognition processing on the speech data to be recognized, obtaining text output and related metadata. Further analysis of multiple words within the speech data using the metadata is then conducted to calculate complexity feature factors characterizing the recognition difficulty of the speech segment. A specialized speech recognition model is then used to meticulously process speech data with high recognition difficulty. Finally, the recognition results of the two models are fused to obtain the target speech recognition result. This invention utilizes an innovative dual-model strategy, integrating a general-purpose model and a specialized model, improving the efficiency and accuracy of traditional speech recognition systems in professional applications. It demonstrates significant advantages in processing speech data containing technical terminology and can effectively improve the recognition efficiency of power grid communication dispatch speech. Attached Figure Description

[0045] Figure 1 This is a flowchart illustrating an automatic speech recognition method for power grid communication scheduling in an embodiment of the present invention.

[0046] Figure 2 This is a schematic diagram of an automatic voice recognition system for power grid communication dispatch according to an embodiment of the present invention. Detailed Implementation

[0047] To enable those skilled in the art to better understand the technical solutions of this invention, the technical solutions of the embodiments of this invention will be clearly and completely described below with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are only for explaining the invention and are not intended to limit the invention.

[0048] like Figure 1 As shown in the figure, an automatic speech recognition method for power grid communication dispatch provided by an embodiment of the present invention includes the following steps:

[0049] S10. Obtain the voice data to be identified for power grid communication scheduling, and perform segmentation processing on the voice data to be identified to obtain multiple voice segments to be identified.

[0050] S20. Input multiple speech segments to be recognized into the first speech recognition model to generate the first speech recognition result.

[0051] S30. Calculate the complexity feature factor of each speech segment to be identified based on the first recognition result, and classify multiple speech segments to be identified based on a preset complexity threshold to obtain the first difficulty segment and the second difficulty segment.

[0052] S40. Extract multiple sets of sub-segment speech data from the first difficulty segment, input the multiple sets of sub-segment speech data and the speech segment to be recognized from the second difficulty segment into the second speech recognition model, and generate the second speech recognition result.

[0053] S50. The first speech recognition result and the second speech recognition result are fused to obtain the target speech recognition result of the speech data to be recognized.

[0054] In this embodiment, the segmentation of the speech data to be recognized mainly employs technologies such as speech activity detection to identify speech segments and non-speech segments within the speech data, facilitating subsequent speech recognition. For example, segmentation can be based on an energy threshold; when the energy of the sound signal is below a pre-set threshold, it is considered silent. Alternatively, segmentation can be based on the amplitude of the speech signal; when the amplitude is continuously below a certain threshold, it is considered silent. Segmenting the speech data to be recognized into multiple speech segments is a technique well-known to those skilled in the art, and this embodiment does not specifically limit its application.

[0055] In this embodiment, the first speech recognition model and the second speech recognition model are a lightweight general speech recognition model and a specialized speech recognition model that can be used to recognize some professional terms in the field of power grid communication and dispatching, respectively. The lightweight general speech recognition model performs preliminary recognition processing on the speech data to be recognized, obtaining text output and related metadata, such as confidence scores and timestamps. Further analysis is then performed on multiple words in the speech data using the metadata, including the proportion of words with low confidence scores and the density of professional terms, to calculate a complexity feature factor characterizing the recognition difficulty of the speech segment to be recognized. The specialized speech recognition model performs detailed processing on the more difficult speech data. Finally, the recognition results of the two models are fused to obtain the target speech recognition result. This embodiment employs an innovative dual-model strategy, integrating a general model and a specialized model, improving the efficiency and accuracy of traditional speech recognition systems in professional fields. It exhibits significant advantages in processing speech data containing professional terms and can effectively improve the recognition efficiency of power grid communication and dispatching speech.

[0056] In a specific implementation process, for step S30, the complexity feature factor of each speech segment to be recognized is calculated based on the first recognition result, specifically including:

[0057] Based on the first recognition result, the proportion of words in each speech segment to be recognized whose confidence score is less than the first preset confidence threshold is calculated to obtain the first complexity parameter.

[0058] Words with confidence scores lower than the second preset confidence threshold in each speech segment to be identified are selected. Based on the power grid communication scheduling vocabulary library, multiple target words are determined from all the selected words. The proportion of target words in each speech segment to be identified is calculated to obtain the second complexity parameter.

[0059] In this embodiment, the first speech recognition result generated by the first speech recognition model includes at least the recognized text of each speech segment to be recognized, and the confidence score and timestamp corresponding to each word in each recognized text. The confidence score is used to quantify the model's assessment of the accuracy of recognizing each word or phrase. A high confidence score means that the model is relatively certain of the accuracy of its recognition result, while a low confidence score indicates the model's uncertainty about this part of the content. The timestamp is used to indicate the start and end time of each word or term, helping to determine the specific data segment corresponding to each word in the initial audio.

[0060] In this embodiment, the recognition difficulty of each speech segment to be recognized is measured from two aspects: the proportion of uncertain words and the density of technical terms. Specifically, using a pre-set threshold, namely the aforementioned first pre-set confidence threshold, words with confidence scores lower than the first pre-set confidence threshold are considered uncertain words. The ratio of uncertain words to the total number of words in the speech segment is calculated to obtain a first complexity parameter. A second complexity parameter is obtained by calculating the ratio of words suspected to be technical terms in each speech segment to the total number of words in the speech segment. Finally, the complexity feature factor of each speech segment to be recognized is calculated using the complexity feature factor calculation formula.

[0061] Specifically, the formula for calculating the complexity feature factor is as follows:

[0062]

[0063] In the formula, C represents the complexity feature factor of the speech segment to be recognized, M1 represents the first complexity parameter of the speech segment to be recognized, M2 represents the second complexity parameter of the speech segment to be recognized, and μ, These are the first weight parameter and the second weight parameter, respectively.

[0064] In step S30 above, after calculating the complexity feature factor of each speech segment to be recognized, the multiple speech segments to be recognized are classified based on complexity in combination with a preset complexity threshold. Specifically, speech segments to be recognized with a complexity feature factor less than the preset complexity threshold are marked as first difficulty segments, and speech segments to be recognized with a complexity feature factor not less than the preset complexity threshold are marked as second difficulty segments.

[0065] In a specific implementation process, multiple target terms are determined from all the selected terms based on the power grid communication dispatch terminology database, specifically including:

[0066] Calculate the edit distance between each selected word and multiple professional terms in the power grid communication and dispatch terminology database. Taking any selected word as an example, if there is at least one professional term in the power grid communication and dispatch terminology database whose edit distance to this word is greater than a preset similarity threshold, then this word is marked as a target word. Repeat this process for all selected words to obtain multiple target words.

[0067] In this embodiment, the similarity of words is measured by calculating the edit distance between any two words. The power grid communication scheduling vocabulary library stores multiple professional terms related to power grid communication scheduling. By reasonably setting a preset similarity threshold, words whose edit distance to at least one professional term in the power grid communication scheduling vocabulary library is greater than the preset similarity threshold are marked as target words. These target words are used to characterize the professional terms that may exist in the speech segment to be identified, thereby realizing the quantitative analysis of the professional term density of any speech segment to be identified.

[0068] In this embodiment, considering that a lightweight general-purpose speech recognition model may have low recognition accuracy for some specialized terms during the initial recognition processing of the speech data to be recognized, resulting in low confidence scores for some words, but may still recognize words similar to or with similar pronunciations of specialized terms. For example, the specialized term is "photovoltaic," but the model may recognize it as "photovoltaic recovery." Therefore, a pre-set threshold, namely the aforementioned second pre-set reliability threshold, is used to filter out words with a high probability of being similar to or with similar pronunciations of specialized terms. It is worth noting that the first pre-set reliability threshold is used to filter out words that the lightweight general-purpose speech recognition model does not recognize accurately, while the second pre-set reliability threshold is used to further analyze these inaccurately recognized words to determine which ones may be specialized terms. To improve the accuracy of specialized term density, the second pre-set reliability threshold can expand the coverage based on the first pre-set reliability threshold. For example, the first pre-set reliability threshold is set to 70%, and the second pre-set reliability threshold is set to 50%, to perform similarity analysis on a wider range of words with specialized terms.

[0069] In a specific implementation, the second speech recognition model in step S40 is specifically built based on the Transformer model architecture. The Transformer, based on an attention mechanism, abandons the traditional loop structure, enabling parallel processing of sequence data and performing well in handling long-distance dependencies, making it suitable for complex speech recognition tasks. For offline processing tasks requiring high accuracy, the Transformer can perform well. For the first speech recognition model in step S20, considering the processing speed and computing resources, a fast and lightweight model should be used. In this embodiment, an LSTM network architecture is used as an example to implement the first speech recognition model, achieving rapid processing of speech data. The first speech recognition model is mainly used for preliminary processing of speech data, prioritizing speed and lightweight operation.

[0070] It is worth noting that although a lightweight general-purpose speech recognition model cannot accurately recognize all specialized terms, it can provide a baseline recognition result to help understand the general framework of the content involved in the speech data. When a general-purpose speech recognition model shows low confidence for a certain word, it can be regarded as a signal that the word may contain specialized terms or other difficult-to-recognize elements. In this case, a specialized model can be used to re-recognize this part of the content to obtain a more detailed recognition result. This embodiment does not specifically limit the first speech recognition model. Those skilled in the art can choose other models that prioritize speed, followed by recognition accuracy, and have low training difficulty, such as DeepSpeech. DeepSpeech is an open-source speech recognition tool based on a simple RNN architecture. Its advantage lies in its relatively simple structure and ease of implementation. In actual use, the model can be appropriately simplified and the number of model layers reduced to create a lightweight version more suitable for fast recognition.

[0071] It is worth noting that in this embodiment, the classification of multiple speech segments to be recognized is achieved by calculating the complexity feature factors of different speech segments to be recognized. The first difficulty segment can be understood as containing more general vocabulary and a few specialized words, with a lower recognition difficulty. The second difficulty segment can be understood as containing more specialized words and a few specialized words, with a higher recognition difficulty. For speech data with lower recognition difficulty, the first speech recognition model can directly recognize it with high accuracy. Only a few specialized words may have lower recognition accuracy. Therefore, the second speech recognition model is used to further analyze and process the speech data corresponding to these words to improve recognition accuracy. For speech data with higher recognition difficulty, it is input as a whole into the second speech recognition model. The second speech recognition model can be used to process sequence data and can understand contextual relationships in speech recognition tasks. The Transformer model used in this embodiment is based on an attention mechanism, which enables it to capture long-distance dependencies in the input sequence and performs well in understanding contextual relationships. For the second difficulty level paragraphs, which are more difficult to recognize, the improvement of choosing to input the entire paragraph into the second speech recognition model instead of extracting the speech data corresponding to a portion of the technical words is that, for speech data involving a lot of technical words, the second speech recognition model can understand the contextual relationships, resulting in higher recognition accuracy. Therefore, the model has greater reference value for the recognition results of the entire paragraph. Compared to recognizing multiple words that do not have contextual relationships separately, the overall recognition accuracy is higher.

[0072] In this embodiment, the second speech recognition model is trained using a training dataset. The training dataset includes at least multiple sets of power grid communication dispatch speech data and transcribed text of each set. During training, the multiple sets of power grid communication dispatch speech data are converted into Mel spectrograms and used as input to the second speech recognition model. The second speech recognition model, based on the Transformer model architecture, can directly use the Mel spectrograms as input. During training, the transcribed text of each set of power grid communication dispatch speech data is used as the training target. The difference between the predicted and actual values ​​of the second speech recognition model is calculated using the cross-entropy loss function. Character error rate (CER) or word error rate (WER) can also be used as the main evaluation metrics for model performance to assess or compare model performance, resulting in a high-performance second speech recognition model. Similarly, if the first speech recognition model is based on an LSTM network architecture, its training uses a training dataset containing a general speech database and corresponding transcribed text. Training the first speech recognition model based on the LSTM network architecture is a well-known technique and will not be described further in this embodiment.

[0073] In this embodiment, the training dataset used to train the second speech recognition model includes speech data on power grid communication and dispatching in a specialized field, along with corresponding text transcriptions. This speech data covers a wide range of scenarios and speech variations, and contains a large number of specialized terms related to power grid communication and dispatching. This allows the model to learn the pronunciation and contextual relationships of these terms during training, enabling it to understand the meaning and usage of each term in a specific context. Since the model is primarily trained on a training dataset containing a large number of specialized terms, it achieves better recognition accuracy for specific terms because it has adapted to those specific pronunciation and contextual environments. In contrast, the model's recognition accuracy for general terms is lower. For the training of the first speech recognition model, the general speech database and corresponding transcribed text training dataset are characterized by containing a relatively broad range of general terms, primarily at a lightweight level. Compared to the training dataset for general models, due to the large number of general terms involved, the coherence of the speech data is secondary; that is, the emphasis on the contextual relationships between terms is relatively lower. The aim is to improve the model's training speed and achieve the recognition of more general terms, while specialized terms are handled by more targeted specialized models.

[0074] In a specific implementation process, step S40 involves extracting multiple sets of sub-segment speech data from the first difficulty segment, inputting the multiple sets of sub-segment speech data and the speech segment to be recognized from the second difficulty segment into the second speech recognition model, and generating the second speech recognition result, specifically including:

[0075] Based on the timestamp of each word in the first speech recognition result, the sub-segment speech data corresponding to each word in the first difficulty segment with a confidence score less than the first preset confidence threshold is extracted. The multiple sets of sub-segment speech data and the speech segment to be recognized in the second difficulty segment are input into the system to generate the second speech recognition result.

[0076] In this embodiment, for the speech segments to be recognized that are classified as second difficulty segments, the entire segment is input into the second speech recognition model. For the speech segments to be recognized that are classified as first difficulty segments, in order to further improve the recognition accuracy, the sub-segment speech data corresponding to each word with a confidence score lower than a first preset confidence threshold is selected. The second speech recognition model further processes these speech data that may involve professional terms. The final generated second speech recognition result includes the first recognition result of multiple sets of sub-segment speech data and the second recognition result of the speech segments to be recognized in the second difficulty segment. The first recognition result includes the recognized text of each set of sub-segment speech data and the confidence score corresponding to the recognized text. The second recognition result includes the recognized text of each speech segment to be recognized in the multiple speech segments to be recognized in the second difficulty segment, as well as the confidence score and timestamp corresponding to each word in each recognized text.

[0077] In a specific implementation process, for step S50, the first speech recognition result and the second speech recognition result are fused to obtain the target speech recognition result of the speech data to be recognized, specifically including:

[0078] For multiple sets of sub-segment speech data in the first recognition result, the confidence scores of the recognized text in the first recognition result and the confidence scores of the recognized text in the first speech recognition result are compared, and the speech text with the higher confidence score is selected as the target recognition text of the sub-segment speech data.

[0079] In this embodiment, since some speech data suspected of involving professional terms are fed into the second speech recognition model for more detailed recognition, the confidence of the text involved in these sub-segments of speech data is analyzed by combining the first recognition result and the first speech recognition result. If professional terms are involved, the confidence score of the recognized text in the first recognition result will be higher; otherwise, the confidence score of the recognized text in the first speech recognition result will be higher. For cases where both recognition is difficult, such as due to noise in the speech data or due to non-standard pronunciation, resulting in low confidence scores, the text with the higher confidence score is selected as the target recognition text. For the speech segments to be recognized that are classified as first difficulty segments, the multiple recognition texts corresponding to the speech data other than multiple sets of sub-segments of speech data are directly used as the final recognition result, i.e., marked as the target recognition text.

[0080] For multiple speech segments to be recognized in the second difficulty level, multiple sets of associated recognition texts that are related in the first speech recognition result and the second recognition result are determined based on multiple timestamps. The target recognition text of multiple speech segments to be recognized in the second difficulty level is determined based on each set of associated recognition texts.

[0081] In this embodiment, for multiple speech segments to be recognized that have been processed by the first speech recognition model and the second speech recognition model and are classified as second difficulty segments, multiple sets of associated recognition texts that are related in the first speech recognition result and the second recognition result are determined based on multiple timestamps. That is, two recognition texts that belong to the same speech data segment are analyzed. The confidence scores corresponding to the two recognition texts are analyzed, and the recognition text with the higher confidence score is selected as the target text, thereby obtaining the target speech recognition result of the speech data to be recognized.

[0082] In a specific implementation process, for the speech data input into the first speech recognition model and the second speech recognition model, in order to improve the speed of the first speech recognition model in processing speech data and improve the recognition accuracy of the second speech recognition model, multiple speech segments to be recognized are sampled at a first sampling frequency and sent to the first speech recognition model, and multiple sub-segment speech data and speech segments to be recognized in the second difficulty segment are sampled at a second sampling frequency and sent to the second speech recognition model.

[0083] In this embodiment, a higher sampling frequency can be used for the speech data input into the second speech recognition model. For example, for multiple speech segments to be recognized fed into the first speech recognition model, the sampling frequency is 8000Hz to reduce the amount of data and speed up the model's processing. For multiple sets of sub-segments of speech data and speech segments of second difficulty level to be recognized fed into the second speech recognition model, the sampling frequency is 16000Hz, which helps the model capture more details and improves the model's recognition accuracy. It is worth noting that the first and second sampling frequencies do not mean that the speech data fed into the first speech recognition model sacrifices recognition accuracy. The main purpose is to improve the recognition of the second speech recognition model to obtain more refined recognition results and improve the recognition accuracy of professional terms related to power grid communication and dispatch.

[0084] This invention provides an automatic speech recognition method for power grid communication dispatch. This method can be used to call relevant models for automatic speech recognition in intranet mode. For audio data stored in a database, the automatic speech recognition method provided in this invention processes the speech data in batches, and the output results can be stored in a table. This facilitates quick retrieval of large amounts of data using keywords, allowing for the selection of required recordings. While ensuring security, this method significantly shortens data query time, improves information query efficiency, and saves power grid companies time and manpower costs.

[0085] In this embodiment, for the text generated from batch recognition and processing of power grid audio data files, different file storage formats can be preset, such as .txt, .xlsx, and word. For the speech data to be processed, such as .wav and .mp3 files, after the data is read, it is processed by calling the first speech recognition model and the second speech recognition model. The second speech recognition model can be used for speech recognition in offline mode, adapting to audio recognition in environments without network or under intranet conditions. After obtaining the recognition result using the automatic speech recognition method provided in this embodiment, it can be stored using a preset file template. Multiple recognized texts can also be integrated and stored for easy subsequent information retrieval. The above implementation process is only one implementation scheme of this embodiment and does not represent a specific limitation of the implementation scheme.

[0086] like Figure 2 As shown, this embodiment of the invention also provides an automatic voice recognition system for power grid communication dispatch, the structure of which includes:

[0087] The voice data preprocessing module is used to acquire voice data to be recognized for power grid communication scheduling, and to segment the voice data to be recognized to obtain multiple voice segments to be recognized.

[0088] The first speech processing module is used to input multiple speech segments to be recognized into the first speech recognition model and generate the first speech recognition result. The first speech recognition result includes the recognized text of each speech segment to be recognized, as well as the confidence score and timestamp corresponding to each word in each recognized text.

[0089] The speech segment classification module is used to calculate the complexity feature factor of each speech segment to be identified based on the first recognition result, and classify multiple speech segments to be identified based on a preset complexity threshold, including marking speech segments to be identified with a complexity feature factor less than the preset complexity threshold as first difficulty segments, and marking speech segments to be identified with a complexity feature factor not less than the preset complexity threshold as second difficulty segments.

[0090] The speech segment classification module calculates the complexity feature factor for each speech segment to be identified based on the first recognition result, including:

[0091] The first complexity parameter is obtained by calculating the proportion of words with confidence scores less than the first preset confidence threshold in each speech segment to be recognized based on the first recognition result.

[0092] Words with confidence scores lower than the second preset confidence threshold in each speech segment to be identified are selected. Multiple target words are determined from all selected words based on the power grid communication scheduling vocabulary library. The proportion of target words in each speech segment to be identified is calculated to obtain the second complexity parameter.

[0093] The complexity feature factor of each speech segment to be recognized is calculated based on the following formula:

[0094]

[0095] In the formula, C represents the complexity feature factor of the speech segment to be recognized, M1 represents the first complexity parameter of the speech segment to be recognized, M2 represents the second complexity parameter of the speech segment to be recognized, and μ, These are the first weight parameter and the second weight parameter, respectively.

[0096] The second speech processing module extracts multiple sets of sub-segment speech data from the first difficulty segment, inputs the multiple sets of sub-segment speech data and the speech segment to be recognized from the second difficulty segment into the second speech recognition model, and generates the second speech recognition result.

[0097] The second speech recognition model used in the second speech processing module also includes:

[0098] A second speech recognition model was constructed based on the Transformer model architecture and trained using a training dataset. The training dataset included multiple sets of power grid communication dispatch speech data and the transcribed text of each set. During training, the multiple sets of power grid communication dispatch speech data were converted into Mel spectrograms and used as input to the second speech recognition model. The transcribed text of each set of power grid communication dispatch speech data was used as the training target. The cross-entropy loss function was employed to calculate the difference between the predicted and actual values ​​of the second speech recognition model.

[0099] The recognition result fusion module is used to fuse the features of the first speech recognition result and the second speech recognition result to obtain the target speech recognition result of the speech data to be recognized.

[0100] The above are merely specific embodiments of the present invention, enabling those skilled in the art to understand or implement the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art. Parts not described in detail in this specification are prior art known to those skilled in the art.

Claims

1. An automatic speech recognition method for power grid communication dispatch, characterized in that, include: Acquire the voice data to be identified for power grid communication scheduling, and segment the voice data to be identified to obtain multiple voice segments to be identified; Multiple speech segments to be recognized are input into the first speech recognition model to generate the first speech recognition result. The first speech recognition result includes the recognized text of each speech segment to be recognized, as well as the confidence score and timestamp corresponding to each word in each recognized text. The complexity feature factor of each speech segment to be identified is calculated based on the first recognition result. Multiple speech segments to be identified are classified based on a preset complexity threshold, including marking speech segments to be identified with a complexity feature factor less than the preset complexity threshold as first difficulty segments and speech segments to be identified with a complexity feature factor not less than the preset complexity threshold as second difficulty segments. Multiple sets of sub-segment speech data are extracted from the first difficulty segment. The multiple sets of sub-segment speech data and the speech segment to be recognized in the second difficulty segment are input into the second speech recognition model to generate the second speech recognition result. Each set of sub-segment speech data corresponds to a word in the first difficulty segment whose confidence score is less than the first preset confidence threshold. The first speech recognition result and the second speech recognition result are fused to obtain the target speech recognition result of the speech data to be recognized. The complexity feature factor for each speech segment to be identified is calculated based on the first recognition result, including: The first complexity parameter is obtained by calculating the proportion of words with confidence scores less than the first preset confidence threshold in each speech segment to be recognized based on the first recognition result. Words with confidence scores lower than the second preset confidence threshold in each speech segment to be identified are selected. Multiple target words are determined from all selected words based on the power grid communication scheduling vocabulary library. The proportion of target words in each speech segment to be identified is calculated to obtain the second complexity parameter. The complexity feature factor of each speech segment to be recognized is calculated based on the following formula: In the formula, The complexity feature factor represents the speech segment to be identified. The first complexity parameter represents the speech segment to be recognized. The second complexity parameter represents the speech segment to be recognized. , These are the first weight parameter and the second weight parameter, respectively.

2. The automatic speech recognition method for power grid communication dispatch according to claim 1, characterized in that, The second speech recognition model also includes: A second speech recognition model was constructed based on the Transformer model architecture, and the second speech recognition model was trained using a training dataset, which included multiple sets of power grid communication scheduling speech data and the transcribed text of each set of power grid communication scheduling speech data. During the training process of the second speech recognition model, multiple sets of power grid communication scheduling speech data are converted into Mel spectrograms and used as input to the second speech recognition model. The transcribed text of each set of power grid communication scheduling speech data is used as the training target of the second speech recognition model. The cross-entropy loss function is used to calculate the difference between the predicted value and the true value of the second speech recognition model.

3. The automatic speech recognition method for power grid communication dispatch according to claim 1, characterized in that, Based on the power grid communication dispatch vocabulary database, several target words were identified from all the selected words, including: The edit distance between each selected word and multiple professional words in the power grid communication and dispatch terminology library is calculated. For any selected word, if there is at least one professional word in the power grid communication and dispatch terminology library with an edit distance greater than the preset similarity threshold, the word is marked as a target word. All selected words are traversed to obtain multiple target words.

4. The automatic speech recognition method for power grid communication dispatch according to claim 1, characterized in that, Multiple sets of sub-segment speech data are extracted from the first difficulty segment. These multiple sets of sub-segment speech data, along with the speech segment to be recognized from the second difficulty segment, are input into the second speech recognition model to generate the second speech recognition result, including: Based on the timestamp of each word in the first speech recognition result, extract the sub-segment speech data corresponding to each word in the first difficulty segment whose confidence score is less than the first preset confidence threshold. Input the multiple sets of sub-segment speech data and the speech segment to be recognized in the second difficulty segment into the second speech recognition model to generate the second speech recognition result, including the first recognition result of the multiple sets of sub-segment speech data and the second recognition result of the speech segment to be recognized in the second difficulty segment. The first recognition result includes the recognized text of each group of sub-segment speech data and the confidence score corresponding to the recognized text. The second recognition result includes the recognized text of each speech segment to be recognized in the multiple speech segments to be recognized in the second difficulty segment, as well as the confidence score and timestamp corresponding to each word in each recognized text.

5. The automatic speech recognition method for power grid communication dispatch according to claim 4, characterized in that, The first speech recognition result and the second speech recognition result are fused to obtain the target speech recognition result of the speech data to be recognized, including: For multiple sets of sub-segment speech data in the first recognition result, compare the confidence scores of the recognized text in the first recognition result with the confidence scores of the recognized text in the first speech recognition result, and select the speech text with the higher confidence score as the target recognition text of the sub-segment speech data. For multiple speech segments to be recognized in the second difficulty level, multiple sets of associated recognition texts that are related in the first speech recognition result and the second recognition result are determined based on multiple timestamps. The target recognition text of multiple speech segments to be recognized in the second difficulty level is determined based on the recognition text with higher confidence score in each set of associated recognition texts. The multiple recognition texts corresponding to the speech data other than the multiple sub-segments in the first speech recognition result are marked as target recognition texts, thus obtaining the target speech recognition result of the speech data to be recognized.

6. The automatic speech recognition method for power grid communication dispatch according to claim 1, characterized in that, Multiple speech segments to be recognized are sampled at a first sampling frequency and fed into a first speech recognition model. Multiple sub-segment speech data and speech segments to be recognized in the second difficulty segment are sampled at a second sampling frequency and fed into a second speech recognition model.

7. An automatic speech recognition system for power grid communication dispatch, the system being used to implement the automatic speech recognition method for power grid communication dispatch as described in any one of claims 1-6, characterized in that, include: The voice data preprocessing module is used to acquire the voice data to be recognized for power grid communication scheduling, and to segment the voice data to be recognized to obtain multiple voice segments to be recognized. The first speech processing module is used to input multiple speech segments to be recognized into the first speech recognition model and generate the first speech recognition result. The first speech recognition result includes the recognized text of each speech segment to be recognized, as well as the confidence score and timestamp corresponding to each word in each recognized text. The speech segment classification module is used to calculate the complexity feature factor of each speech segment to be identified based on the first recognition result, and classify multiple speech segments to be identified based on a preset complexity threshold, including marking speech segments to be identified with a complexity feature factor less than the preset complexity threshold as first difficulty segments, and marking speech segments to be identified with a complexity feature factor not less than the preset complexity threshold as second difficulty segments. The second speech processing module extracts multiple sets of sub-segment speech data from the first difficulty segment, inputs the multiple sets of sub-segment speech data and the speech segment to be recognized from the second difficulty segment into the second speech recognition model, and generates the second speech recognition result. The recognition result fusion module is used to fuse the first speech recognition result and the second speech recognition result to obtain the target speech recognition result of the speech data to be recognized. The speech segment classification module calculates the complexity feature factor for each speech segment to be identified based on the first recognition result, including: The first complexity parameter is obtained by calculating the proportion of words with confidence scores less than the first preset confidence threshold in each speech segment to be recognized based on the first recognition result. Words with confidence scores lower than the second preset confidence threshold in each speech segment to be identified are selected. Multiple target words are determined from all selected words based on the power grid communication scheduling vocabulary library. The proportion of target words in each speech segment to be identified is calculated to obtain the second complexity parameter. The complexity feature factor of each speech segment to be recognized is calculated based on the following formula: In the formula, The complexity feature factor represents the speech segment to be identified. The first complexity parameter represents the speech segment to be recognized. The second complexity parameter represents the speech segment to be recognized. , These are the first weight parameter and the second weight parameter, respectively.

8. An automatic voice recognition system for power grid communication dispatch according to claim 7, characterized in that, The second speech recognition model used in the second speech processing module also includes: A second speech recognition model was constructed based on the Transformer model architecture, and the second speech recognition model was trained using a training dataset, which included multiple sets of power grid communication scheduling speech data and the transcribed text of each set of power grid communication scheduling speech data.

Images