A speech recognition method and system

Abstract

The application discloses a voice recognition method and system, wherein the method comprises the following steps: acquiring a to-be-recognized voice signal in a power dispatching scene in real time; performing frequency domain feature conversion on the to-be-recognized voice signal to obtain a mel-frequency spectrum feature; inputting the mel-frequency spectrum feature into a power dispatching voice recognition model to obtain a voice recognition result; wherein the training process of the power dispatching voice recognition model comprises the following steps: inputting an original voice data set in the power dispatching scene into at least two general voice recognition models respectively to obtain a corresponding number of inference texts; comparing each inference text with a labeled text of the original voice data set to obtain a plurality of comparison results; extracting common recognition errors in each comparison result to generate an enhanced power dispatching voice data set; and training an initial voice recognition model based on the enhanced power dispatching voice data set to obtain a trained power dispatching voice recognition model. The method improves the accuracy of voice recognition.

Description

Technical Field

[0001] This invention relates to the field of speech recognition technology, and in particular to a speech recognition method and system. Background Technology

[0002] Power dispatching, as a core component of ensuring the safe and stable operation of the power grid, relies heavily on daily voice command interactions, placing stringent requirements on the accuracy and reliability of voice recognition.

[0003] Existing speech recognition solutions often rely on general feature extraction and decoding processes, such as directly applying general models to speech transcription tasks without optimizing and adapting them for specific industry scenarios.

[0004] However, in power dispatching scenarios, existing speech recognition solutions are prone to problems such as misjudgment of terms and insufficient accent adaptation due to factors such as electromagnetic noise from equipment, differences in dispatching accents, and dense technical terms. For example, "closing the switch" may be recognized as "closing the tension" or "tripping the switch" may be recognized as "tripping the screen." This results in insufficient robustness and consistency of terminology in speech recognition under dispatching environments with multiple accents and strong noise, which restricts the implementation and application of intelligent dispatching systems. Summary of the Invention

[0005] This invention provides a speech recognition method and system to solve the technical problem of how to improve existing speech recognition methods and achieve the effect of improving speech recognition accuracy.

[0006] To address the aforementioned technical problems, the present invention provides a speech recognition method, comprising: Real-time acquisition of voice signals to be recognized in power dispatch scenarios; The speech signal to be identified is subjected to frequency domain feature transformation to obtain Mel spectrum features; The Mel spectrum features are input into the power dispatch speech recognition model to obtain the speech recognition result of the speech signal to be recognized; The training process of the power dispatch speech recognition model includes: Obtain the original speech dataset from the power dispatch scenario, and input the original speech dataset into at least two general speech recognition models to obtain the corresponding number of inference texts; Each of the inference texts is compared with the labeled text of the original speech dataset to obtain several comparison results; Extract common identification errors from the comparison results to generate an enhanced power dispatch voice dataset; The initial speech recognition model is trained based on the enhanced power dispatch speech dataset to obtain the trained power dispatch speech recognition model.

[0007] As one preferred embodiment, the step of performing frequency domain feature transformation on the speech signal to be recognized to obtain Mel spectrum features includes: The speech signal to be recognized is pre-emphasized to obtain the first speech signal after high-frequency enhancement. The first speech signal is sequentially framed and windowed to obtain a smooth and continuous second speech signal. Perform a short-time Fourier transform on the second speech signal to obtain a linear frequency domain amplitude spectrum; The linear frequency domain amplitude spectrum is input into the Mel filter bank, and logarithmic compression and normalization are performed sequentially to obtain the standardized Mel spectrum characteristics.

[0008] As one preferred embodiment, the step of inputting the Mel spectrum features into the power dispatch speech recognition model to obtain the speech recognition result of the speech signal to be recognized includes: The Mel spectrum features are input into the power dispatch speech recognition model to obtain a frame-level speech feature sequence; Based on the frame-level speech feature sequence, accent level classification is performed to obtain accent level results; Match the prompt word corresponding to the accent level result, and input the frame-level speech feature sequence after dimensional projection and the prompt word into a pre-trained large language model to obtain the speech recognition result of the speech signal to be recognized.

[0009] As one preferred embodiment, the step of inputting the frame-level speech feature sequence, after dimensional projection, together with the prompt words into a pre-trained large language model to obtain the speech recognition result of the speech signal to be recognized includes: The frame-level speech feature sequence is dimensionally mapped to obtain projection features that match the embedding space of the large language model; The system dynamically matches prompt words based on the accent level results; wherein, the accent level results include mild accent, moderate accent, and severe accent, the mild accent uses standard recognition prompt words, the moderate accent uses pronunciation variant prompt words, and the severe accent uses contextual prompt words from the power dispatching field; The projection features are concatenated with the embedding vectors of the corresponding prompt words and then input into the large language model to output the speech recognition result of the speech signal to be recognized.

[0010] As one preferred embodiment, the step of extracting common identification errors from the comparison results to generate an enhanced power dispatch voice dataset includes: Based on common identification errors in various reasoning texts, a confused word pair consisting of incorrect and correct words is constructed. Add corresponding pinyin information to each of the aforementioned pairs of confused words; The pinyin information and the confused word pairs are associated and fused into the labeled text of the original speech dataset to obtain the enhanced power dispatch speech dataset.

[0011] Another aspect of the present invention provides a speech recognition system, comprising: The acquisition module is used to acquire the voice signals to be recognized in power dispatching scenarios in real time; The conversion module is used to perform frequency domain feature conversion on the speech signal to be recognized to obtain Mel spectrum features; The recognition module is used to input the Mel spectrum features into the power dispatch speech recognition model to obtain the speech recognition result of the speech signal to be recognized; The identification module further includes: The acquisition unit is used to acquire the original speech dataset in the power dispatching scenario, and input the original speech dataset into at least two general speech recognition models to obtain the corresponding number of inference texts; The comparison unit is used to compare each of the inference texts with the labeled texts of the original speech dataset to obtain several comparison results; An extraction unit is used to extract common identification errors from the comparison results and generate an enhanced power dispatch voice dataset. The training unit is used to train the initial speech recognition model based on the enhanced power dispatch speech dataset to obtain the trained power dispatch speech recognition model.

[0012] As one preferred embodiment, the conversion module is specifically used for: The speech signal to be recognized is pre-emphasized to obtain the first speech signal after high-frequency enhancement. The first speech signal is sequentially framed and windowed to obtain a smooth and continuous second speech signal. Perform a short-time Fourier transform on the second speech signal to obtain a linear frequency domain amplitude spectrum; The linear frequency domain amplitude spectrum is input into the Mel filter bank, and logarithmic compression and normalization are performed sequentially to obtain the standardized Mel spectrum characteristics.

[0013] As one preferred embodiment, the identification module is specifically used for: The Mel spectrum features are input into the power dispatch speech recognition model to obtain a frame-level speech feature sequence; Based on the frame-level speech feature sequence, accent level classification is performed to obtain accent level results; Match the prompt word corresponding to the accent level result, and input the frame-level speech feature sequence after dimensional projection and the prompt word into a pre-trained large language model to obtain the speech recognition result of the speech signal to be recognized.

[0014] As one preferred embodiment, the identification module is further configured to: The frame-level speech feature sequence is dimensionally mapped to obtain projection features that match the embedding space of the large language model; The system dynamically matches prompt words based on the accent level results; wherein, the accent level results include mild accent, moderate accent, and severe accent, the mild accent uses standard recognition prompt words, the moderate accent uses pronunciation variant prompt words, and the severe accent uses contextual prompt words from the power dispatching field; The projection features are concatenated with the embedding vectors of the corresponding prompt words and then input into the large language model to output the speech recognition result of the speech signal to be recognized.

[0015] As one preferred embodiment, the extraction unit is specifically used for: Based on common identification errors in various reasoning texts, a confused word pair consisting of incorrect and correct words is constructed. Add corresponding pinyin information to each of the aforementioned pairs of confused words; The pinyin information and the confused word pairs are associated and fused into the labeled text of the original speech dataset to obtain the enhanced power dispatch speech dataset.

[0016] Compared with the prior art, the beneficial effects of the present invention are at least one of the following: 1) This invention effectively solves the problems of scarce training data and difficulty in identifying homonymous professional terms in the power dispatching field by comparing multiple models inference on the original speech training dataset, extracting common errors, and constructing confused word pairs with pinyin. This significantly improves the model's ability to distinguish industry-specific terms such as closing, tripping, busbar, and circuit breaker, reducing the terminology misidentification rate and making the speech recognition results more consistent with power dispatching business specifications. Simultaneously, by freezing the pre-trained model and training only the projection layer and accent classifier, the training cost is significantly reduced while ensuring recognition accuracy, thus improving model convergence efficiency.

[0017] 2) This invention introduces a quantitative classification of accent intensity and a dynamic adaptation mechanism for prompt words into the speech recognition process. This allows for automatic switching of the recognition strategy based on the accent intensity of the speech being recognized in real time, effectively adapting to acoustic feature fluctuations caused by different accents and significantly improving the robustness of recognition in mixed accent scenarios. By combining accent features with the contextual understanding capabilities of a large language model, deep alignment of speech features and semantic decoding is achieved. This maintains high accuracy and stability even in power dispatching environments characterized by strong noise, multiple accents, and high terminology density, better meeting the engineering application requirements of intelligent dispatching systems for reliable transcription of voice commands. Attached Figure Description

[0018] Figure 1 This is a flowchart illustrating a speech recognition method in one embodiment of the present invention; Figure 2 This is a flowchart illustrating the training method of a power dispatch speech recognition model in one embodiment of the present invention. Figure 3 This is a structural block diagram of a speech recognition system in one embodiment of the present invention; Figure label: Among them, 11 is the acquisition module; 12 is the conversion module; and 13 is the recognition module. Detailed Implementation

[0019] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. The purpose of providing these embodiments is to make the disclosure of the present invention more thorough and comprehensive. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention.

[0020] In the description of this invention, the terms "first," "second," "third," etc., are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, a feature defined with "first," "second," "third," etc., may explicitly or implicitly include one or more of that feature. In the description of this invention, unless otherwise stated, "a plurality of" means two or more.

[0021] In the description of this invention, it should be noted that, unless otherwise defined, all technical and scientific terms used in this invention have the same meaning as commonly understood by one of ordinary skill in the art. The terminology used in this specification is for the purpose of describing specific embodiments only and is not intended to limit the invention. Those skilled in the art can understand the specific meaning of the above terms in this invention based on the specific circumstances.

[0022] One embodiment of the present invention provides a speech recognition method; for details, please refer to [link to specific documentation]. Figure 1 , Figure 1 The diagram shown is a flowchart of a speech recognition method according to one embodiment of the present invention, which includes steps S1-S3: S1: Real-time acquisition of voice signals to be recognized in power dispatch scenarios.

[0023] It should be noted that the process of acquiring the voice signal to be recognized in real time as described in this embodiment is applicable to various power dispatching communication scenarios such as power grid dispatching centers, centralized control stations, and substation duty. It aims to provide a stable and reliable real-time voice input source for subsequent accent-adaptive speech recognition. This invention does not limit the specific voice acquisition equipment and deployment scenarios.

[0024] In this embodiment, real-time acquisition of the voice signal to be identified in the power dispatching scenario specifically refers to the real-time acquisition of voice command signals issued by dispatchers when performing switching operations, fault handling, equipment status reporting, load adjustment, and other tasks through a dedicated power dispatching communication link. This voice signal is a continuous time-domain analog signal or a digital audio signal, containing the dispatcher's natural pronunciation, the pronunciation of business terminology, and components such as environmental noise and electromagnetic interference noise within the scenario, thus fully reflecting the true voice characteristics and accent features of the current dispatching interaction.

[0025] Preferably, the acquired speech signal to be recognized is a single-channel digital audio signal with a sampling rate of 16kHz and a sampling precision of 16bit. This parameter configuration can reduce the computational load of subsequent signal processing while ensuring speech clarity, thus meeting the real-time recognition requirements of power dispatching. Those skilled in the art can adjust the sampling rate and sampling precision according to hardware performance and latency requirements; this invention does not limit such adjustments.

[0026] In this embodiment, the speech signal to be recognized is different from the power dispatch speech dataset used in the training phase. The former is online real-time acquisition, unannotated speech data used for direct inference and recognition, while the latter is offline acquisition, manually annotated speech data used for model training and data augmentation. By strictly distinguishing between the real-time speech signal to be recognized and the training dataset, the training and inference phases in the speech recognition process can be guaranteed to be independent of each other, avoiding recognition errors caused by data confusion.

[0027] It should be noted that during the real-time acquisition of the speech signal to be recognized, the signal can be preliminarily normalized in amplitude and DC offset removed to eliminate the basic interference caused by the acquisition equipment. However, this embodiment does not perform subsequent operations such as accent recognition, feature extraction, and text transcription. It only completes the real-time acquisition and standardized output of the original speech signal, providing basic input for subsequent preprocessing, Mel spectrogram conversion, accent degree classification, and the generation of recognition text by the large language model.

[0028] In this embodiment, the real-time acquired speech signal to be recognized may contain different degrees of accent features, covering light, moderate, and heavy accents. The signal may contain power dispatching terminology such as "circuit breaker," "disconnect switch," "reclosing," "busbar," "closing," and "tripping." This signal will directly serve as the sole real-time input source for the subsequent speech recognition process, supporting the realization of the core innovation of this invention: adaptive speech recognition based on accent intensity.

[0029] S2: Perform frequency domain feature transformation on the speech signal to be recognized to obtain Mel spectrum features.

[0030] It should be noted that speech signals are easily affected by environmental noise and differences in pronunciation in the time domain, resulting in poor robustness when directly used for model recognition. Therefore, it is necessary to convert the time-domain speech signal into frequency domain features that better conform to the characteristics of human hearing. In this embodiment, the speech signal to be recognized is sequentially subjected to pre-emphasis, frame windowing, short-time Fourier transform, Mel filtering, logarithmic compression, and normalization processing to finally obtain standardized Mel spectral features. This invention does not limit the optional parameters and equivalent transformation methods for each step.

[0031] In this embodiment, the speech signal to be recognized is first pre-emphasized to obtain a first speech signal with high-frequency enhancement. Pre-emphasis is achieved using a first-order high-pass filter, the mathematical expression of which is: in, This is the pre-weighting coefficient. Preferably, the pre-weighting coefficient... The value is set to 0.97, which is used to enhance the energy of the high-frequency components of the speech signal, reduce the influence of low-frequency environmental noise and electromagnetic interference, improve the signal-to-noise ratio of high-frequency details, and provide a clearer signal basis for subsequent frequency domain analysis.

[0032] The obtained first speech signal is sequentially processed by framing and windowing to obtain a smooth and continuous second speech signal. Framing involves dividing the continuous speech signal into several short frames of fixed length, maintaining a certain overlap between adjacent frames to preserve the continuity of the speech signal. Windowing multiplies each frame signal by a window function to eliminate discontinuities at frame boundaries and suppress spectral leakage. In this embodiment, a Hamming window is preferably used as the window function. Those skilled in the art can choose Hanning windows, Blackman windows, etc., according to feature requirements, and this invention does not limit this choice.

[0033] In this embodiment, a short-time Fourier transform (STFT) is performed on the second speech signal to obtain a linear frequency domain amplitude spectrum. The STFT is used to convert each frame of the time-domain speech signal into a frequency domain representation. By calculating the spectrum result in complex form, the square of its amplitude is taken to obtain the energy distribution of the frame at each frequency point, thereby converting the one-dimensional time-domain signal into two-dimensional time-frequency features, providing a standard input for the subsequent Mel-scale transform.

[0034] The linear frequency domain amplitude spectrum described above is input into a Mel filter bank for filtering. The Mel filter bank consists of multiple triangular filters evenly distributed on the Mel scale. It can simulate the nonlinear perception characteristics of human ear for sound frequencies, mapping the linear frequency axis to the Mel frequency axis, thus providing higher resolution in the low-frequency band and appropriately reducing the resolution in the high-frequency band, making it more suitable for the characteristic expression of accent differences and the pronunciation of professional terms.

[0035] In this embodiment, the output signal after Mel filtering undergoes logarithmic compression and normalization sequentially. Logarithmic compression matches the logarithmic perception of sound intensity by the human ear, compressing the feature value range; normalization eliminates feature shifts caused by recording equipment, speaker differences, and channel variations, ultimately yielding standardized Mel spectral features with stable dimensions and uniform distribution. This feature will be directly used as input to the subsequent speech encoder, supporting accent recognition and large language model speech-to-text processes.

[0036] It should be noted that the frequency domain feature conversion process described in this embodiment is performed entirely on the real-time speech signal to be recognized, without relying on the training dataset. It is consistent with the feature extraction process in the model training stage, ensuring that the feature spaces of training and inference are aligned, and providing a stable and reliable feature foundation for accent classification and adaptive prompt word recognition.

[0037] S3: Input the Mel spectrum features into the power dispatch speech recognition model to obtain the speech recognition result of the speech signal to be recognized.

[0038] It should be noted that in scenarios involving mixed accents, dense technical terms, and limited training data in power dispatching, directly using a general speech recognition model can easily lead to problems such as homonym ambiguity and insufficient accent robustness. This embodiment uses accent-level recognition, dimensional projection, and dynamic matching of prompt words, combined with a large language model to complete the final speech recognition. It fully utilizes accent severity information and contextual semantic information to improve the recognition accuracy in technical terms and accent scenarios. The specific model and parameters of the pre-trained model used in this invention are not limited here.

[0039] In this embodiment, standardized Mel-spectral features are input into the speech encoder of the power dispatch speech recognition model, and frame-level speech feature sequences are obtained through forward inference by the encoder. This speech encoder is a pre-trained general speech recognition model encoder, which has been adapted and optimized using the enhanced power dispatch speech training dataset during the training phase. It can fully extract high-dimensional temporal representations containing accent information, pronunciation details, and technical terminology features, providing a unified feature foundation for subsequent accent classification and text generation.

[0040] Accent severity is graded based on frame-level speech feature sequences to obtain accent level results. In this embodiment, the accent level results include three categories: mild accent, moderate accent, and severe accent. A multilayer perceptron structure is used to complete the graded recognition. The frame-level speech feature sequences are aggregated into fixed-dimensional feature vectors by performing average pooling, max pooling, and end-time step feature extraction. These vectors are then classified and output using a multilayer fully connected network to determine the accent level with the highest probability. This invention does not limit the feature aggregation method or the number of classification network layers, as long as it can achieve quantitative grading of accent severity.

[0041] In this embodiment, corresponding prompt words are dynamically matched based on the accent level results. Specifically, a light accent corresponds to a standard recognition prompt word for regular speech transcription; a moderate accent corresponds to a prompt word emphasizing pronunciation variations to strengthen pronunciation difference constraints; and a heavy accent corresponds to a prompt word incorporating the context of power dispatching to fully utilize the terminology context to correct recognition biases. By dynamically binding prompt words to accent levels, the large language model can adaptively adapt to different pronunciation conditions, overcoming the shortcomings of existing technologies such as fixed prompt words and poor accent adaptability.

[0042] The frame-level speech feature sequence is dimensionally projected through a projection layer and mapped to the embedding space of the pre-trained large language model. In this embodiment, the projection layer is implemented using a two-layer linear network. During the training phase, only the projection layer parameters are optimized, while the parameters of the speech encoder and the large language model are frozen, which can achieve feature space alignment while reducing training costs. The dimensionally projected speech feature sequence is concatenated with the previously matched prompt word embedding sequence and used together as the input to the pre-trained large language model.

[0043] The concatenated features and prompts are input into a pre-trained large language model, which then performs autoregressive generation to obtain the speech recognition result corresponding to the speech signal to be recognized. In this embodiment, the large language model fully utilizes contextual understanding and domain knowledge, combined with accent-adaptive prompt constraints, to accurately distinguish power dispatching professional terms such as "circuit breaker," "closing," "busbar," and "tripping," effectively reducing the error rate of homophones and ultimately outputting recognized text that conforms to power dispatching business specifications.

[0044] It should be noted that this embodiment completes forward inference entirely based on the real-time speech signal to be recognized, without relying on training data for computation. The processes of accent classification, prompt word matching, dimensional projection, and text generation are executed in a coherent manner, which can meet the requirements of real-time performance, accuracy, and reliability in power dispatching scenarios.

[0045] Another embodiment of the present invention provides a training method for a power dispatch speech recognition model. For details, please refer to [link to relevant documentation]. Figure 2 , Figure 2 The diagram shown illustrates a training method for a power dispatch speech recognition model according to one embodiment of the present invention, comprising steps S31-S34: S31: Obtain the original speech dataset in the power dispatching scenario, and input the original speech dataset into at least two general speech recognition models to obtain the corresponding number of inference texts.

[0046] It should be noted that the power dispatching field is a specialized speech recognition scenario, characterized by dense technical terminology, limited labeled data, and significant accent differences. Directly using a general-purpose model for training can easily lead to homonym recognition errors, making it difficult to meet the accuracy requirements for dispatching command recognition. This step aims to locate common recognition errors in power dispatching speech through cross-reasoning using multiple general-purpose models, providing a basis for subsequently constructing confused word pairs and generating enhanced training data. This invention does not limit the specific structure and source of the general-purpose speech recognition model.

[0047] In this embodiment, the raw voice dataset from the power dispatching scenario is first acquired. This raw voice dataset consists of real voice data from offline power dispatching, including commands issued by dispatchers in various business scenarios such as switching operations, equipment inspections, fault reporting, and load control. Each voice command is accompanied by a manually verified standard annotation text, which serves as a reference for identifying correctness. The raw voice dataset covers dispatching instructions under different accents, speaking speeds, and environmental noise levels, comprehensively reflecting the actual distribution of power dispatching voice commands.

[0048] The original speech dataset is input into at least two general-purpose speech recognition models, and offline inference is performed to obtain inferred text corresponding to the number of models. In this embodiment, three general-purpose end-to-end speech recognition models are preferably used to improve the reliability and comprehensiveness of common error extraction. The general-purpose speech recognition models are general models trained on publicly available speech datasets and have not been fine-tuned with power dispatching professional data, thus objectively exposing recognition defects under professional terminology and accent conditions.

[0049] In this embodiment, each general speech recognition model independently performs forward computation on each speech item in the original speech dataset and independently outputs the corresponding text result. The models do not share parameters, interact with features, or fuse results, ensuring the independence and comparability of the inference text. The inference text contains misidentified, omitted, or misspelled homophones in power dispatch terminology, which are the direct source of common errors in subsequent localization.

[0050] This invention does not strictly limit the number of general speech recognition models used; there can be two, three, or more. Those skilled in the art can flexibly choose according to computing resources and error extraction accuracy requirements. It should be noted that this step only completes multi-model inference and text output, without text comparison, error extraction, pinyin annotation, or model parameter updates, providing basic input for subsequent construction of confused word pairs and augmented datasets.

[0051] In this embodiment, the original speech dataset and its corresponding multiple inference texts together constitute a multi-model recognition result comparison library. This result library is specifically used to mine easily confused and misidentified word combinations in power dispatching terminology. It is different from the speech signals to be recognized in the real-time recognition process and belongs to the dedicated data in the model training stage. It can effectively alleviate the technical problems of insufficient training data and difficulty in distinguishing homonyms in power dispatching scenarios.

[0052] S32: Compare each inference text with the labeled text of the original speech dataset to obtain several comparison results.

[0053] It should be noted that after completing the inference output of multiple general speech recognition models on the original speech dataset, in order to accurately locate the recognition defects of power dispatch professional terms, it is necessary to compare the inference text output by each model with the manually labeled standard text character by character in order to objectively obtain the recognition deviation of the model in professional vocabulary, accent pronunciation, and homophones.

[0054] In this embodiment, the annotated text accompanying the original speech dataset is standard text manually verified by professionals in the power industry, containing standard expressions such as dispatch instructions, equipment names, and operational terms, which can serve as the sole criterion for determining the correctness of the recognition results. The inference text consists of the transcription results output by at least two general speech recognition models, without manual correction, and can truly reflect the recognition performance of the general models in power dispatch scenarios.

[0055] Each inference text and labeled text is compared sentence by sentence and character by character, aligning the same time position and text position to obtain several comparison results. In this embodiment, the comparison results include at least character matching results, word error types, error position indexes, and corresponding pronunciation content. Among them, word error types include homophone errors, near-homophone errors, omissions, and multiple recognitions, especially highlighting high-frequency homophone ambiguities in power dispatching such as "closing / tensioning," "busbar / wooden line," and "tripping / screen tripping."

[0056] Preferably, this embodiment employs a character-by-character edit distance comparison method to align the labeled text and the inferred text, marking inconsistent character ranges and recording the error content and location corresponding to each speech. This invention does not limit the specific text similarity calculation method or alignment algorithm; those skilled in the art can choose an appropriate text comparison method based on the required comparison accuracy, as long as the recognition errors can be accurately extracted.

[0057] In this embodiment, each speech record generates a comparison result consistent with the number of models. The comparison results from different models are independent of each other and are not pre-fused to ensure the objectivity and reliability of subsequent common error extraction. The comparison results are only used to characterize the recognition deviation of the general model on power dispatch-specific data and do not participate in the real-time speech recognition process. They are dedicated to data augmentation and model optimization during the training phase.

[0058] It should be noted that this step only completes text comparison and error information recording; it does not perform common error screening, construction of confused word pairs, or pinyin annotation. Through the above comparison process, the weaknesses of the general model in power dispatch terminology and accent pronunciation can be comprehensively and systematically identified, providing a reliable basis for subsequent targeted data augmentation, thereby alleviating the technical problems of insufficient training data and difficulty in distinguishing homophones in power dispatch scenarios.

[0059] S33: Extract common identification errors from the comparison results and generate an enhanced power dispatch voice dataset.

[0060] It should be noted that power dispatching scenarios suffer from problems such as dense technical terminology, scarce training data, and frequent homophones and ambiguities. Relying solely on the original annotated text makes it difficult for the model to learn the rules for distinguishing easily confused pronunciations. This step extracts common recognition errors from multi-model inference, constructs confused word pairs, and integrates pinyin information to form enhanced training data for the power dispatching field. This can significantly alleviate the problem of homophone misidentification. This invention does not limit the pinyin annotation rules or data fusion format.

[0061] In this embodiment, based on the comparison results obtained by comparing each inference text with the labeled text, common recognition errors appearing in all models are extracted. The common recognition errors refer to the same recognition errors produced by no fewer than two general speech recognition models on the same speech, the same character or word, which can objectively reflect the most confusing and misrecognized pronunciations and word combinations in power dispatching terminology, such as recognizing "closing" as "closing tension", recognizing "busbar" as "wooden wire", and recognizing "tripping" as "screen tripping", etc.

[0062] Based on the common identification errors mentioned above, a confusion word pair consisting of an incorrect word and a correct word is constructed. The confusion word pair is represented as follows: ,in, Error words output by multiple models To mark the correct words in the text, This refers to the sequence number of the obfuscated word pair. In this embodiment, the obfuscated word pairs cover core operational terms for power dispatching, equipment names, and status descriptions, providing the model with clear criteria for pronunciation correction and text differentiation.

[0063] For each pair of confused words, add the corresponding pinyin information to each correct character. With erroneous words The model is annotated with standard Mandarin pinyin, including initials, finals, and tones. This pinyin information can distinguish between homophones or near-homophones at the pronunciation level, allowing the model to learn by combining textual and pronunciation features during training. This enhances the model's ability to distinguish easily confused terms, and is particularly suitable for scenarios with pronunciation fluctuations caused by accent differences.

[0064] The aforementioned pinyin information and confused word pairs are fused into the labeled text of the original speech dataset, expanding and enhancing the original labeled text to obtain an enhanced power dispatch speech dataset. In this embodiment, the fusion method involves adding confused word pairs and pinyin annotations to the corresponding word positions in the labeled text, so that the dataset simultaneously contains the original standard text, easily confused error text, and pronunciation information, forming dedicated training data suitable for accent-adaptive speech recognition.

[0065] It should be noted that the enhanced power dispatch speech dataset is specifically used for the model training phase. By introducing confused word pairs and pinyin features, it effectively supplements the scarce training samples in the power dispatch field and solves the recognition defects of general models in professional terms. This invention does not limit the specific format or storage form of data fusion, as long as it can achieve joint training of confused word pairs, pinyin information and original speech data.

[0066] In this embodiment, the enhanced power dispatch speech dataset will be used for subsequent Mel spectrum feature extraction, projection layer training, accent classifier training, and large language model fine-tuning, enabling the model to have the ability to distinguish power dispatch terms and adapt to accents at the data level, providing data support for the subsequent realization of high-precision and high-robust speech recognition.

[0067] S34: The initial speech recognition model is trained based on the enhanced power dispatch speech dataset to obtain the trained power dispatch speech recognition model.

[0068] It should be noted that initial speech recognition models are typically built using general pre-trained models. When directly applied to power dispatching scenarios, due to insufficient specialized data, significant accent differences, and easy confusion of homophones, the recognition accuracy often fails to meet business requirements. This step utilizes an enhanced power dispatching speech dataset containing confused word pairs and pinyin information for model training, enabling the model to fully learn domain pronunciation rules and terminology differentiation features. This invention does not limit the specific training loss function, optimizer type, or iteration rounds.

[0069] In this embodiment, the initial speech recognition model is an LLM-based ASR architecture, which consists of four parts: a large speech recognition encoder, a projection layer, an accent recognition classifier, and a pre-trained large language model. The large speech recognition encoder is used to extract acoustic features, the projection layer is used for feature space alignment, the accent recognition classifier is used for accent severity classification, and the large language model is used for context understanding and text generation.

[0070] When training the initial speech recognition model based on the enhanced power dispatch speech dataset, a strategy of freezing the backbone network and training only lightweight branches is adopted. In this embodiment, the parameters of the large speech recognition model encoder and the pre-trained large language model are kept constant, and only the parameters of the projection layer and the accent recognition classifier are updated and iterated to reduce training overhead, accelerate convergence speed, and avoid the destruction of general knowledge by small sample data in the domain.

[0071] In this embodiment, the training process executes two subtasks simultaneously: first, the projection layer maps the speech feature sequence to the embedding space of the large language model, with text transcription loss as the optimization objective; second, the accent recognition classifier grades the accent intensity of the speech features, with classification loss as the optimization objective. The two subtasks are executed in parallel and jointly optimized, enabling the model to simultaneously possess the ability to transcribe technical terms and the ability to perceive accent strength.

[0072] During training, the model fully utilizes the confusing word pairs and pinyin information in the enhanced power dispatch speech dataset to learn the pronunciation rules and text differentiation rules of power dispatch terminology, effectively alleviating homophonic ambiguity issues such as "closing / tensioning," "bus / wooden line," and "tripping / screen tripping." Simultaneously, the model gradually improves its adaptive differentiation ability for light, moderate, and heavy accents through learning from diverse accent samples.

[0073] When the model loss converges to a preset threshold and the recognition accuracy meets the requirements of the power dispatching scenario, the training process stops, resulting in a trained power dispatching speech recognition model. This model possesses robust recognition capabilities for specialized terms, accent quantification, and dynamic adaptation of prompt words. It can be directly used for inference and recognition of real-time dispatching speech, distinguishing it from general models and baseline models trained without augmented data.

[0074] It should be noted that this step is only part of the model training phase and is independent of the inference process of real-time speech recognition. Through the training method described above, the model's professionalism and accent adaptability can be significantly improved under conditions of scarce power dispatch data, providing a stable and reliable model foundation for subsequent real-time recognition.

[0075] Another embodiment of the present invention provides a speech recognition system; for details, please refer to [link to relevant documentation]. Figure 3 , Figure 3 The diagram shown is a structural block diagram of a speech recognition system according to one embodiment of the present invention, which includes: The acquisition module 11 is used to acquire the voice signal to be recognized in the power dispatching scenario in real time; Conversion module 12 is used to perform frequency domain feature conversion on the speech signal to be recognized to obtain Mel spectrum features; The recognition module 13 is used to input the Mel spectrum features into the power dispatch speech recognition model to obtain the speech recognition result of the speech signal to be recognized.

[0076] The recognition module also includes: The acquisition unit is used to acquire the original speech dataset in the power dispatching scenario, and input the original speech dataset into at least two general speech recognition models to obtain the corresponding number of inference texts; The comparison unit is used to compare each inference text with the labeled text of the original speech dataset to obtain several comparison results; The extraction unit is used to extract common identification errors from the comparison results and generate an enhanced power dispatch voice dataset. The training unit is used to train the initial speech recognition model based on the enhanced power dispatch speech dataset to obtain the trained power dispatch speech recognition model.

[0077] Preferably, in one embodiment of the present invention, the conversion module is specifically used for: The speech signal to be recognized is pre-emphasized to obtain the first speech signal after high-frequency enhancement. The first speech signal is sequentially framed and windowed to obtain a smooth and continuous second speech signal. A short-time Fourier transform is performed on the second speech signal to obtain the linear frequency domain amplitude spectrum; The linear frequency domain amplitude spectrum is input into the Mel filter bank, and logarithmic compression and normalization are performed sequentially to obtain the standardized Mel spectrum characteristics.

[0078] Preferably, in one embodiment of the present invention, the identification module is specifically used for: The Mel spectrum features are input into the power dispatch speech recognition model to obtain a frame-level speech feature sequence; Accent level classification is performed based on frame-level speech feature sequences to obtain accent level results; Match the prompt words corresponding to the accent level results, and input the frame-level speech feature sequence and the prompt words together into the pre-trained large language model after dimensional projection to obtain the speech recognition result of the speech signal to be recognized.

[0079] Preferably, in one embodiment of the present invention, the identification module is further configured to: Dimensional mapping is performed on the frame-level speech feature sequence to obtain projected features that match the embedding space of the large language model; The system dynamically matches prompt words based on accent level results. The accent level results include mild accent, moderate accent, and severe accent. Standard recognition prompt words are used for mild accent, pronunciation variant prompt words are used for moderate accent, and contextual prompt words from the power dispatching field are used for severe accent. The projected features are concatenated with the embedding vectors of the corresponding prompt words and then input into a large language model to output the speech recognition result of the speech signal to be recognized.

[0080] Preferably, in one embodiment of the present invention, the extraction unit is specifically used for: Based on common identification errors in various reasoning texts, a confused word pair consisting of incorrect and correct words is constructed. Add the corresponding pinyin information to each pair of confused words; By associating and fusing pinyin information and confused word pairs into the labeled text of the original speech dataset, an enhanced power dispatch speech dataset is obtained.

[0081] The above embodiments merely illustrate several implementation methods of the present invention, and their descriptions are relatively specific and detailed, but they should not be construed as limiting the scope of the present invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these all fall within the protection scope of the present invention. Therefore, the protection scope of this patent should be determined by the appended claims.

Claims

1. A speech recognition method, characterized in that, include: Real-time acquisition of voice signals to be recognized in power dispatch scenarios; The speech signal to be identified is subjected to frequency domain feature transformation to obtain Mel spectrum features; The Mel spectrum features are input into the power dispatch speech recognition model to obtain the speech recognition result of the speech signal to be recognized; The training process of the power dispatch speech recognition model includes: Obtain the original speech dataset from the power dispatch scenario, and input the original speech dataset into at least two general speech recognition models to obtain the corresponding number of inference texts; Each of the inference texts is compared with the labeled text of the original speech dataset to obtain several comparison results; Extract common identification errors from the comparison results to generate an enhanced power dispatch voice dataset; The initial speech recognition model is trained based on the enhanced power dispatch speech dataset to obtain the trained power dispatch speech recognition model.

2. The speech recognition method as described in claim 1, characterized in that, The step of performing frequency domain feature transformation on the speech signal to be identified to obtain Mel spectrum features includes: The speech signal to be recognized is pre-emphasized to obtain the first speech signal after high-frequency enhancement. The first speech signal is sequentially framed and windowed to obtain a smooth and continuous second speech signal. Perform a short-time Fourier transform on the second speech signal to obtain a linear frequency domain amplitude spectrum; The linear frequency domain amplitude spectrum is input into the Mel filter bank, and logarithmic compression and normalization are performed sequentially to obtain the standardized Mel spectrum characteristics.

3. The speech recognition method as described in claim 1, characterized in that, The step of inputting the Mel spectrum features into the power dispatch speech recognition model to obtain the speech recognition result of the speech signal to be recognized includes: The Mel spectrum features are input into the power dispatch speech recognition model to obtain a frame-level speech feature sequence; Based on the frame-level speech feature sequence, accent level classification is performed to obtain accent level results; Match the prompt word corresponding to the accent level result, and input the frame-level speech feature sequence after dimensional projection and the prompt word into a pre-trained large language model to obtain the speech recognition result of the speech signal to be recognized.

4. The speech recognition method as described in claim 3, characterized in that, The step of inputting the frame-level speech feature sequence, after dimensional projection, together with the prompt words into a pre-trained large language model to obtain the speech recognition result of the speech signal to be recognized includes: The frame-level speech feature sequence is dimensionally mapped to obtain projection features that match the embedding space of the large language model; The system dynamically matches prompt words based on the accent level results; wherein, the accent level results include mild accent, moderate accent, and severe accent, the mild accent uses standard recognition prompt words, the moderate accent uses pronunciation variant prompt words, and the severe accent uses contextual prompt words from the power dispatching field; The projection features are concatenated with the embedding vectors of the corresponding prompt words and then input into the large language model to output the speech recognition result of the speech signal to be recognized.

5. The speech recognition method as described in claim 1, characterized in that, The step of extracting common identification errors from each comparison result to generate an enhanced power dispatch voice dataset includes: Based on common identification errors in various reasoning texts, a confused word pair consisting of incorrect and correct words is constructed. Add corresponding pinyin information to each of the aforementioned pairs of confused words; The pinyin information and the confused word pairs are associated and fused into the labeled text of the original speech dataset to obtain the enhanced power dispatch speech dataset.

6. A speech recognition system, characterized in that, include: The acquisition module is used to acquire the voice signals to be recognized in power dispatching scenarios in real time; The conversion module is used to perform frequency domain feature conversion on the speech signal to be recognized to obtain Mel spectrum features; The recognition module is used to input the Mel spectrum features into the power dispatch speech recognition model to obtain the speech recognition result of the speech signal to be recognized; The identification module further includes: The acquisition unit is used to acquire the original speech dataset in the power dispatching scenario, and input the original speech dataset into at least two general speech recognition models to obtain the corresponding number of inference texts; The comparison unit is used to compare each of the inference texts with the labeled texts of the original speech dataset to obtain several comparison results; An extraction unit is used to extract common identification errors from the comparison results and generate an enhanced power dispatch voice dataset. The training unit is used to train the initial speech recognition model based on the enhanced power dispatch speech dataset to obtain the trained power dispatch speech recognition model.

7. The speech recognition system as described in claim 6, characterized in that, The conversion module is specifically used for: The speech signal to be recognized is pre-emphasized to obtain the first speech signal after high-frequency enhancement. The first speech signal is sequentially framed and windowed to obtain a smooth and continuous second speech signal. Perform a short-time Fourier transform on the second speech signal to obtain a linear frequency domain amplitude spectrum; The linear frequency domain amplitude spectrum is input into the Mel filter bank, and logarithmic compression and normalization are performed sequentially to obtain the standardized Mel spectrum characteristics.

8. The speech recognition system as described in claim 6, characterized in that, The identification module is specifically used for: The Mel spectrum features are input into the power dispatch speech recognition model to obtain a frame-level speech feature sequence; Based on the frame-level speech feature sequence, accent level classification is performed to obtain accent level results; Match the prompt word corresponding to the accent level result, and input the frame-level speech feature sequence after dimensional projection and the prompt word into a pre-trained large language model to obtain the speech recognition result of the speech signal to be recognized.

9. The speech recognition system as described in claim 8, characterized in that, The identification module is also used for: The frame-level speech feature sequence is dimensionally mapped to obtain projection features that match the embedding space of the large language model; The system dynamically matches prompt words based on the accent level results; wherein, the accent level results include mild accent, moderate accent, and severe accent, the mild accent uses standard recognition prompt words, the moderate accent uses pronunciation variant prompt words, and the severe accent uses contextual prompt words from the power dispatching field; The projection features are concatenated with the embedding vectors of the corresponding prompt words and then input into the large language model to output the speech recognition result of the speech signal to be recognized.

10. The speech recognition system as described in claim 6, characterized in that, The extraction unit is specifically used for: Based on common identification errors in various reasoning texts, a confused word pair consisting of incorrect and correct words is constructed. Add corresponding pinyin information to each of the aforementioned pairs of confused words; The pinyin information and the confused word pairs are associated and fused into the labeled text of the original speech dataset to obtain the enhanced power dispatch speech dataset.

Images