Dialect speech recognition method based on global-local multi-stage training and dynamic iteration pseudo tag generation and related device

Abstract

The invention discloses a dialect speech recognition method based on global-local multi-stage training and dynamic iteration pseudo-tags and a related device, and relates to the technical field of speech recognition. The method comprises the following steps: standardizing dialect voice data to construct a corpus, and dividing sub dialect areas through acoustic feature clustering; training a ''lightweight CNN + full connection layer'' classifier to reduce cross-area interference; generating a teacher model by adopting global unified model pre-training and sub-dialect local fine tuning; constructing a multi-component collaborative loss function system, and combining with a dynamic iteration pseudo-tag generation mechanism optimization model; and finally, through endpoint detection, feature extraction and decoding post-processing, an identification result is output. Experiments show that the character error rate (CER) can be reduced by 5%-12% under the low-resource condition (1-8-hour labeled data / county), and the method is suitable for dialect protection and voice interaction application.

Description

Technical Field

[0001] This invention relates to the field of speech recognition technology, specifically a low-resource automatic speech recognition (ASR) training method and related apparatus for dialect recognition, which is particularly suitable for speech recognition tasks of fine-grained county-level dialects. Background Technology

[0002] Existing end-to-end automatic speech recognition (ASR) models have made significant progress in Mandarin or standard speech scenarios, but they still face the following challenges in dialect speech recognition, especially fine-grained county-level dialect recognition tasks:

[0003] Insufficient corpus: Obtaining annotated corpus of dialects is costly, and county-level corpus is extremely scarce, making it difficult to train the model effectively.

[0004] Insufficient modeling: Existing administrative divisions or traditional linguistic classification methods are difficult to accurately reflect the acoustic similarity between dialects, resulting in poor model generalization ability.

[0005] Pseudo-label noise problem: Although semi-supervised learning can alleviate insufficient labeling, traditional pseudo-labeling methods are prone to accumulating noise, which reduces training stability and model accuracy.

[0006] Therefore, there is an urgent need for a new technical solution that can effectively improve the speech recognition accuracy of fine-grained dialects under low-resource conditions. Summary of the Invention

[0007] The purpose of this invention is to propose a dialect speech recognition method and related device based on a combination of global-local multi-stage training and dynamic iterative pseudo-label generation, aiming to solve the problems of scarce dialect data, insufficient modeling diversity and accumulation of pseudo-label noise in the existing technology.

[0008] A dialect speech recognition method based on global-local multi-stage training and dynamic iterative pseudo-labels includes the following steps:

[0009] Standardize the speech data of the dialect of the region to be identified and output a dialect corpus;

[0010] Based on the constructed dialect corpus, feature extraction is performed in the acoustic feature space to construct a unified feature space. Unsupervised clustering is then performed by combining the similarity of acoustic features and linguistic verification to divide the dialect samples in the dialect corpus into several sub-dialect regions.

[0011] Based on sub-dialect regions, shared and discriminative features are extracted through an acoustic feature extractor, and a "lightweight CNN + fully connected layer" classifier is trained to automatically classify unlabeled data to reduce cross-dialect interference.

[0012] Based on dialect regions, a unified acoustic model is first trained using a dialect corpus to learn shared representations across dialects; then, local fine-tuning is performed for each sub-dialect region to generate a teacher model that enhances regional features.

[0013] We construct a loss function system consisting of labeled cross-entropy loss, pseudo-label cross-entropy loss, consistency loss, and minimum word error rate loss. Finally, we optimize the model convergence by dynamically adjusting the weights to balance the loss functions in the supervised training phase and the unsupervised training phase.

[0014] Based on the teacher model, high-confidence pseudo-labels are selected using the dual consistency of global and local models, and pseudo-labels are refreshed periodically through a dynamic caching mechanism. Manually labeled data is divided into regions and the selected pseudo-labels are input into supervised and unsupervised alternating training modules. First, the labeled data is used to anchor the model parameters, and then the pseudo-labels are combined to mine the features of unlabeled data. The pseudo-labels and model parameters are iteratively updated to output the final dialect recognition model.

[0015] Using the final dialect recognition model, endpoint detection, acoustic feature extraction, and post-decoding processing are performed on the speech to be recognized, and the dialect recognition text result is output.

[0016] Furthermore, the standardization process includes: audio file formatting, language recognition filtering, speech activity detection and segmentation, sentence-level reconstruction, and character rate filtering.

[0017] Furthermore, the audio file formatting, language recognition filtering, speech activity detection and segmentation, sentence-level reconstruction, and character rate filtering specifically include the following steps:

[0018] Audio file formatting: The original dialect audio collected is uniformly converted into a standardized mono 16kHz WAV format; at the same time, the audio duration is subject to regular constraints, and audio with a duration of more than 4000 seconds or less than 10 seconds is removed.

[0019] Language identification and filtering: The formatted audio data is input into a pre-trained dialect identification model to predict its language category; the long audio is divided into several segments, and the final language label is determined based on the segment voting mechanism; audio that does not belong to the target dialect or whose language is unclear is removed, so that the audio data entering the next stage are all target dialect data, forming a language-filtered audio set.

[0020] Speech Activity Detection and Segmentation: The target dialect audio obtained through language identification and filtering is input into the speech activity detection VAD module, which includes a NeMo forced aligner. The NeMo forced aligner generates a predicted transcribed text with punctuation and capitalization information based on the Paraformer model, and forces it to align with the original audio to obtain word-level timestamps. The VAD module then uses the timestamp information to identify pause intervals in the speech, thereby dividing active speech segments and providing accurate temporal basis for sentence-level reconstruction. The segmented speech segments form structured speech units.

[0021] Sentence-level reconstruction and segmentation: Based on the obtained word-level timestamps and punctuation marks, the audio is reconstructed at the sentence level, using punctuation marks as sentence boundaries. The reconstructed speech segments are then segmented into segments ranging from 1 to 20 seconds in length, ensuring that the final segments fully express the semantics and meet the training requirements of the ASR model. All audio segments are trimmed and reconstructed into audio-text pairs based on the alignment results, forming a standardized sentence-level training sample set.

[0022] Character-level filtering: Sentence-level speech segments are input into the baseline ASR model for automatic transcription; the character rate (CR) is calculated based on the transcribed text, and audio files with significantly abnormal CR are removed to exclude potential noise segments, background music interference, or samples containing a large amount of non-speech content; further, low-quality transcription results are screened out based on the character error rate (CER) of the baseline ASR model to ensure that the data finally entering the dialect corpus meets the training requirements in terms of acoustic quality, language consistency, and text consistency.

[0023] Furthermore, based on the constructed dialect corpus, feature extraction is performed in the acoustic feature space to construct a unified feature space. Unsupervised clustering is then performed, combining the similarity of acoustic features and linguistic verification, to divide the dialect samples in the dialect corpus into several sub-dialect regions. This includes the following steps:

[0024] Acoustic feature extraction and embedding vector generation: Using the trained unified dialect acoustic model of the area to be identified, high-level acoustic feature representations are extracted from the selected samples, and variable-length speech sequences are mapped to fixed-length embedding vectors through the average pooling method, thereby constructing a unified feature space for subsequent cluster analysis.

[0025] Determine the number of sub-dialect regions: Combining the geographical regionalization characteristics of the area to be identified and the opinions of linguistic experts, the search range of the number of cluster categories k is set to 2–13, and the sum of squared errors (SSE) is used as the evaluation index of clustering effect. The elbow method is used to determine the inflection point of the SSE curve, thereby dynamically determining the optimal number of cluster categories. This number of categories corresponds to the final number of sub-dialect regions to be divided.

[0026] Sub-dialect region segmentation: Based on dialect corpus samples, under the conditions of the generated unified acoustic feature space and the determined optimal number of clustering categories, the K-means++ algorithm is used to perform unsupervised clustering of speech samples to obtain preliminary sub-dialect region segmentation results, and complete the accurate segmentation of dialect samples in the dialect corpus into several sub-dialect regions.

[0027] Furthermore, based on sub-dialect regions, shared and discriminative features are extracted using an acoustic feature extractor, and a lightweight CNN + fully connected layer classifier is trained to automatically classify unlabeled data to reduce cross-dialect interference. This includes the following steps:

[0028] 31) Construction and feature extraction of acoustic feature extractor: Design an acoustic feature extractor that includes a common feature module and a discriminative feature module. The common module extracts low-level acoustic features that are common across dialects to reflect the vocal tract and spectral structure. The discriminative module extracts prosodic and intonation features that can distinguish dialects. The low-level acoustic features and prosodic and intonation features are standardized and then spliced ​​and fused to obtain the final acoustic feature vector.

[0029] 32) Dialect Classifier Construction and Training: A classifier is constructed using a "lightweight CNN + fully connected layer" structure, where the CNN contains 3 convolutional layers and the output dimension of the fully connected layer corresponds to the sub-dialect category; the cross-entropy loss is used as the optimization objective, and the Adam optimizer is used for training. The training is terminated by an early stopping strategy to obtain the final classifier.

[0030] 33) Automatic classification of unlabeled data: Input unlabeled data into the classifier, determine the sub-dialect region to which it belongs through category probability, and evaluate the classifier performance using classification accuracy and macro-average F1 value.

[0031] Furthermore, the dialect-based approach first trains a unified acoustic model using a dialect corpus to learn shared representations across dialects; then, it performs local fine-tuning for each sub-dialect region to generate a teacher model that enhances regional features, including the following steps:

[0032] 41) Unified model pre-training: Based on the dialect corpus, the unified dialect speech recognition model is pre-trained to learn the shared acoustic and linguistic representations across dialects, providing a general model foundation for different sub-dialect areas;

[0033] 42) Sub-dialect fine-tuning: On each sub-dialect dataset, the pre-trained model is fine-tuned locally to capture and enhance regional features while maintaining cross-dialect commonalities, thereby improving the model’s local adaptability and specific dialect recognition accuracy.

[0034] 43) Teacher model generation: The fine-tuned sub-dialect area models are used as teacher models to generate high-confidence pseudo-labels for unlabeled data.

[0035] Furthermore, the construction of a loss function system consisting of labeled cross-entropy loss, pseudo-label cross-entropy loss, consistency loss, and minimum word error rate loss, and the optimization of model convergence by dynamically adjusting the weights to balance the loss functions in supervised and unsupervised training, includes the following steps:

[0036] 51) Loss Function System Construction: Construct a loss function system for multi-component collaborative optimization, including four main parts: label cross-entropy loss. Pseudo-label cross-entropy loss Consistency loss and minimum word error rate loss By dynamically adjusting the weights of each part, a balance is achieved between the supervised signal and the semi-supervised signal, thereby significantly improving the dialect speech recognition performance while ensuring model convergence. The formula is shown below.

[0037] (1);

[0038] Among them, consistency loss The loss function is used to constrain the consistency of the model's predictions under different data augmentation or decoding conditions, helping the model maintain stable output under diverse input conditions. Its formula is shown below:

[0039] (2);

[0040] Among them, the minimum word error rate loss The loss function directly optimizes the word error rate, guiding model optimization at the sequence level to further improve the final recognition performance. The formula is shown below:

[0041] (3);

[0042] 52) Application of loss function in supervised training: In the supervised training phase, the labeled cross-entropy loss is used to optimize the input samples and their corresponding real labels. This loss function minimizes the negative log-likelihood of the real labels, so that the predicted distribution is as close as possible to the real distribution, thereby improving the recognition accuracy of the model on labeled data. The formula is as follows.

[0043] (4);

[0044] 53) Application of pseudo-label loss function in unsupervised training: In the unsupervised training phase, pseudo-label cross-entropy loss and consistency loss work together; pseudo-label cross-entropy loss is used to optimize the distribution of pseudo-labeled data and enhance the learning ability of unlabeled samples; in the loss function, the dynamic weights w of the samples are used. This is used to measure the confidence or importance of pseudo-labels, thereby assigning higher weights to high-quality pseudo-labels during training, further improving the robustness and accuracy of semi-supervised learning. The formula is shown below.

[0045] (5).

[0046] Furthermore, the generation of the dialect recognition model includes the following steps:

[0047] 61) Double Consistency Pseudo-Label Filtering: Based on the dialect corpus, language classification is performed to obtain an unlabeled dataset U. Based on the teacher model, pseudo-labels are generated in dataset U by combining the double consistency mechanism of the global model and the local model. Specifically, two sets of pseudo-labels are generated by the sub-dialect area model and the global model respectively, and their character error rate (CER) is calculated. Only pseudo-labels with CER values ​​below the threshold are retained as high-confidence data input for training, thereby effectively suppressing the adverse effects of noisy labels on model convergence.

[0048] 62) Dynamic caching iteration mechanism: Construct a dynamic cache pool management strategy. High-confidence pseudo-labels are stored in the cache pool after generation and refreshed periodically with the training progress. Specifically, during each training round, cached data is randomly sampled with a probability of 0.1 for updating. If an update is triggered, the old pseudo-labels of the corresponding batch are removed, and new labels are regenerated and added to the cache pool, thereby ensuring that the training process continuously benefits from the latest and high-quality pseudo-labels.

[0049] 63) Alternating Supervised and Unsupervised Training: Manually labeled data is divided into regions and then fed into a dynamic caching iteration mechanism along with initial high-quality pseudo-labels selected from double consistency pseudo-labels. Supervised and unsupervised alternating training is performed based on the Parformer model architecture. The training process is divided into two stages: first, supervised training is performed using labeled data to anchor the basic parameters of the model; then, unsupervised training is carried out in conjunction with pseudo-labels to mine dialect acoustic and linguistic features in the unlabeled data; after each round of alternation, the pseudo-labels are updated and the parameters are optimized in reverse until the specified number of training rounds is reached, and finally, a dialect recognition model optimized by the dynamic caching iteration mechanism is obtained.

[0050] Furthermore, the step of using the final dialect recognition model to perform endpoint detection, acoustic feature extraction, and decoding post-processing on the speech to be recognized, and outputting the dialect recognition text result, includes the following steps:

[0051] 71) Speech preprocessing and feature extraction: Endpoint detection is performed on the speech to be recognized to remove silence and noise segments; and acoustic features consistent with those in the training phase are extracted to form a unified acoustic feature sequence required for model input;

[0052] 72) Global-local joint recognition reasoning: The acoustic features are input into the final dialect recognition model. The model first uses the global shared representation to perform cross-dialect basic recognition, and then calls the corresponding local dialect specialization module according to the area division results to output the preliminary word-level recognition results.

[0053] 73) Decoder Language Modeling and Post-processing: The decoder is used to perform language modeling, sequence reordering and punctuation restoration on the model output. Combined with repeated word cleaning and format normalization post-processing steps, the standardized dialect recognition text is finally obtained, realizing the automatic conversion from dialect speech to standard text.

[0054] A dialect speech recognition device based on global-local multi-stage training and dynamic iterative pseudo-labels includes:

[0055] The dialect data collection and processing module is used to standardize the speech data of the dialect of the region to be identified and output a dialect corpus.

[0056] The data-driven dialect region clustering module is used to extract features from the acoustic feature space based on the dialect corpus to construct a unified feature space. It combines the similarity of acoustic features and linguistic verification to perform unsupervised clustering, dividing the dialect samples in the dialect corpus into several sub-dialect regions.

[0057] The dialect classifier construction module based on acoustic features is used to extract shared and discriminative features based on sub-dialect areas through an acoustic feature extractor, train a "lightweight CNN + fully connected layer" classifier, and automatically classify unlabeled data to reduce cross-dialect interference.

[0058] The global-local multi-stage training module is used to train a unified acoustic model based on sub-dialect regions. First, it uses a dialect corpus to train a unified acoustic model and learn cross-dialect shared representations. Then, it performs local fine-tuning for each sub-dialect region to generate a teacher model that enhances regional features.

[0059] The multi-component collaborative loss function system construction module is used to construct a loss function system consisting of labeled cross-entropy loss, pseudo-label cross-entropy loss, consistency loss, and minimum word error rate loss. Finally, the loss function in the supervised training stage and the loss function in the unsupervised training stage are balanced by dynamic weight adjustment to optimize the model convergence.

[0060] A dialect recognition model generation module, which is used to, based on a teacher model, use dual consistency of global and local models to screen high-confidence pseudo-labels, and regularly refresh the pseudo-labels through a dynamic caching mechanism; divide the manually annotated data into regions and input the screened pseudo-labels into a supervised and unsupervised alternating training module. First, anchor the model parameters with the annotated data, then combine the pseudo-labels to mine the features of the unannotated data, and iteratively update the pseudo-labels and model parameters to output the final dialect recognition model;

[0061] A dialect recognition module, which is used to use the final dialect recognition model to perform endpoint detection, acoustic feature extraction and decoding post-processing on the speech to be recognized, and output the dialect recognition text result.

[0062] A dialect speech recognition system based on global-local multi-stage training and dynamic iterative pseudo-labels, including: a computer-readable storage medium and a processor;

[0063] The computer-readable storage medium is used to store executable instructions;

[0064] The processor is used to read the executable instructions stored in the computer-readable storage medium and execute the dialect speech recognition method based on global-local multi-stage training and dynamic iterative pseudo-labels.

[0065] The beneficial effects of the present invention include the following aspects:

[0066] 1. Significantly improve the dialect recognition accuracy under low-resource conditions: Through the global-local multi-stage training (GL-MST) strategy, the present invention achieves an effective balance between cross-dialect commonality modeling and sub-dialect characteristic capture, enabling the model to achieve an overall recognition accuracy improvement of 5% - 12% in a low-resource environment with only 1 - 8 hours of annotated corpus per county, greatly alleviating the dependence of traditional methods on large-scale annotated corpora.

[0067] 2. Suppress the accumulation of pseudo-label noise and ensure training stability: Adopt dynamic iterative pseudo-label generation, combined with dual consistency screening and cache-aware iterative mechanism, which can continuously update and optimize the quality of pseudo-labels, fundamentally reducing the cumulative effect of noise labels, and significantly improving the stability and convergence speed of the training process.

[0068] 3. Enhance the adaptation ability to dialect fine-grained features and rare words: Under the alternating training mechanism, the model can not only robustly recognize the unique expressions and acoustic patterns of Hubei dialect, but also shows higher recognition ability for rare words (such as "嬢", "驺", "哈", "戚"), reducing common errors such as insertions and deletions, and effectively improving the adaptability to real speech scenarios.

[0069] 4. Constructing a scientific and rational dialect region clustering system: Combining acoustic feature embedding and cluster analysis methods, this invention achieves for the first time a dialect region clustering system that is highly consistent with traditional linguistic classifications, while also revealing the acoustic fusion phenomenon under long-term communication in some areas. This not only improves the rationality of training data organization but also provides a new quantitative analysis tool for dialectological research.

[0070] 5. Improved model robustness and cross-domain applicability: Under the dual modeling mechanism of global sharing and local adaptation, this method effectively enhances the model's ability to cope with noise in multiple scenarios, accent variations and cross-regional migration, ensuring the stable performance of the recognition system in different environments.

[0071] 6. Broad application prospects: This invention can be applied not only to industries with high requirements for dialect recognition accuracy, such as power, government affairs, transportation, and medical care, but also to provide scalable technical solutions for multilingual / multi-dialect intelligent voice interaction systems, and has strong practicality and promotional value. Attached Figure Description

[0072] Figure 1 This is a flowchart of a dialect speech recognition method based on global-local multi-stage training and dynamic iterative pseudo-labels according to the present invention.

[0073] Figure 2 This is a data processing flowchart of a dialect speech recognition method based on global-local multi-stage training and dynamic iterative pseudo-labels according to the present invention.

[0074] Figure 3 Example diagram of the reasoning result output of the present invention;

[0075] Figure 4 This is an example diagram illustrating the visualization of the clustering results of this invention. Detailed Implementation

[0076] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0077] Please see Figure 1 and Figure 2 The first aspect of this invention provides a dialect speech recognition method based on global-local multi-stage training and dynamic iterative pseudo-label generation, comprising the following steps:

[0078] Step 1: Standardize the large-scale dialect speech data of Hubei Province to ensure the cleanliness and language specificity of the training samples. This step includes audio file formatting, language identification filtering, speech activity detection and segmentation, sentence-level reconstruction, and character rate filtering, ultimately constructing the HB-39 dialect corpus containing 39 counties and cities in Hubei Province with a total duration of 194 hours.

[0079] Step 2: Data-driven dialect region clustering: Based on the Hubei dialect dataset HB-39 constructed in Step 1, unsupervised clustering is performed in the acoustic feature space. Combining the similarity of acoustic features and linguistic verification, the Hubei dialect samples are divided into five main sub-dialect regions to reduce the interference of cross-dialect differences on model training.

[0080] Step 3: Construction of a dialect classifier based on acoustic features: Based on the sub-dialect regions in Step 2, an acoustic feature extractor is constructed to extract shared tract characteristics, phoneme structure, and distinctive prosodic and intonation features of dialects; combined with the clustering results, a language classifier is further established to automatically classify unlabeled data, thereby reducing cross-dialect interference.

[0081] Step 4, Global-Local Multi-Stage Training: Based on the sub-dialect regions divided in Step 2, a unified acoustic model for Hubei dialect is first pre-trained on the full corpus to learn shared representations across dialects. Then, local fine-tuning is performed for each sub-dialect region to further enhance regional features and generate a more adaptive teacher model, providing support for subsequent pseudo-label generation.

[0082] Step 5: Construct a multi-component collaborative loss function system: A multi-component collaborative optimization system is constructed, consisting of labeled cross-entropy loss, pseudo-label cross-entropy loss, consistency loss, and minimum word error rate loss. Dynamic weight adjustment achieves a balance between supervised and semi-supervised signals, further improving training convergence and dialect recognition accuracy.

[0083] Step 6: Dynamic Iterative Pseudo-Label Generation: Based on the teacher model obtained in Step 4, a dual consistency mechanism of global and local models is used to filter high-confidence pseudo-labels. A cache-aware iterative mechanism is designed (its pseudocode implementation is shown in Table 1). By periodically refreshing the pseudo-labels, noise accumulation and label staleness are avoided, thus ensuring that the training process always benefits from high-quality pseudo-labels. The manually labeled data, after being divided into regions, and the filtered pseudo-labels are input into the supervised and unsupervised alternating training modules. First, the labeled data is used to anchor the model parameters, and then the pseudo-labels are combined to mine the features of the unlabeled data. The pseudo-labels and model parameters are iteratively updated to output the final dialect recognition model.

[0084] Step 7: Using the final dialect recognition model, perform endpoint detection, acoustic feature extraction and decoding post-processing on the speech to be recognized, and output the dialect recognition text result.

[0085] Table 1

[0086]

[0087] Step 1 includes:

[0088] Initialization processing: In the preprocessing stage, the audio files are first converted to a standardized mono 16 kHz WAV format. To avoid the impact of excessively long or short audio files on computational efficiency and modeling results, audio files longer than 4000 seconds and shorter than 10 seconds are discarded.

[0089] Language selection: Given that the collected audio contains multiple dialects, after formatting, a pre-trained dialect recognition model is used to identify the language of the audio, and a segmented voting mechanism is used to filter out non-target languages ​​to ensure that only the target language audio is retained in the final dataset.

[0090] Data Segmentation: Audio segmentation is typically achieved using speech activity detection (VAD) algorithms, which rely on detecting pauses in the speech signal to segment active speech segments. To further improve segmentation accuracy, this invention introduces the NeMo forced aligner. This tool generates predicted transcribed text (containing punctuation and capitalization information) based on a Paraformer model and forces alignment it with the original audio to obtain accurate word-level timestamps. Subsequently, using punctuation marks as sentence boundary markers, the audio is reconstructed at the sentence level and segmented into segments ranging from 1 to 20 seconds in length. All audio files are trimmed according to the alignment results, ultimately forming neat and standardized structured training data.

[0091] Character-level filtering: Since some audio contains noise, background voices, or music interference, this invention initially retains all audio to assess its overall impact on training. Subsequently, a baseline ASR model is used to perform quality filtering on the data. Specifically, this includes: calculating the character rate (CR) based on the transcribed text generated by the baseline model and removing audio files with abnormal CR values; if necessary, low-quality transcription results can be further filtered out based on the word error rate (CER).

[0092] Step 2 includes:

[0093] Speech feature extraction and embedding generation: High-level acoustic feature representations are extracted from the selected samples using the trained Hubei unified dialect acoustic model, and variable-length speech sequences are mapped to fixed-length embedding vectors through the average pooling method, thereby constructing a unified feature space for subsequent cluster analysis;

[0094] Determining the number of cluster categories: Based on the geographical regionalization characteristics of Hubei Province and the opinions of linguistic experts, the search range for the number of cluster categories k was set to 2–13. The sum of squared errors (SSE) was used as the evaluation index for clustering effect. The elbow method was used to determine the inflection point of the SSE curve, thereby dynamically determining the optimal number of cluster categories.

[0095] Clustering Execution and Result Verification: Under the given feature space and clustering parameters, the K-means++ algorithm was used to cluster the speech samples. The clustering results are highly consistent with traditional linguistic classifications, while also revealing acoustic fusion phenomena in some regions due to long-term communication. To further verify the rationality of the clustering, the t-SNE algorithm was used to perform two-dimensional dimensionality reduction and visualization of the embedding vectors. The results are as follows: Figure 4 As shown, the boundaries of different cluster categories are clear in the feature space, verifying the scientific validity and practicality of this method in dialect clustering.

[0096] Step 3 includes:

[0097] Classifier training data preparation: Based on the clustering results of the 5 sub-dialect areas output in step 2, label the samples of each area with exclusive category labels, and divide the training set and validation set from the HB-39 corpus;

[0098] Acoustic Feature Extractor Construction and Feature Extraction: An acoustic feature extractor is designed, comprising a common feature module and a discriminative feature module. The common module extracts low-level acoustic features common to all dialects (such as 24-dimensional MFCC, short-time energy, and formant parameters) to reflect the vocal tract and spectral structure; the discriminative module extracts prosodic and intonation features that can distinguish dialects (such as fundamental frequency F0, pitch curve statistics, duration, and intensity profile). The two types of features are standardized separately and then concatenated and fused to obtain the final acoustic feature vector.

[0099] Dialect classifier model construction and training: The classifier is constructed using a "lightweight CNN + fully connected layer" structure, where the CNN contains 3 convolutional layers (with BN and ReLU), and the fully connected layer has an output dimension of 5, corresponding to 5 sub-dialect categories; the cross-entropy loss is used as the optimization objective, and the Adam optimizer is used for training. The training is terminated by an early stopping strategy to obtain the final classifier.

[0100] Classifier performance verification and unlabeled data classification: Unlabeled data is input into the classifier, and the sub-dialect region is determined by the category probability. The classifier performance is evaluated using classification accuracy and macro-average F1 score. The classified data is used as input for step 5, "Dynamic Perception Iterative Pseudo-Label Generation," thereby reducing cross-dialect interference.

[0101] Step 4 includes:

[0102] Unified Model Pre-training: Based on the HB-39 dataset, a unified speech recognition model for Hubei dialect is pre-trained to learn shared acoustic and linguistic representations across dialects, providing a general model foundation for different sub-dialect regions;

[0103] Sub-dialect area fine-tuning: On each sub-dialect area dataset, the pre-trained model is fine-tuned locally to capture and enhance area features while maintaining cross-dialect commonalities, thereby improving the model's local adaptability and specific dialect recognition accuracy.

[0104] Teacher Model Generation: The finely tuned models of each sub-dialect region are used as teacher models to generate high-confidence pseudo-labels for unlabeled data. This teacher model can provide high-confidence supervision signals for different sub-dialects, thereby improving the quality of pseudo-labels and enhancing the convergence performance and recognition effect during the semi-supervised training phase.

[0105] Step 5 includes:

[0106] Loss Function System Construction: Building upon step 4, a multi-component collaborative optimization loss function system is constructed, comprising four main parts: labeled cross-entropy loss. Pseudo-label cross-entropy loss Consistency loss and minimum word error rate loss By dynamically adjusting the weights of each component, a balance is achieved between the supervised and semi-supervised signals, thereby significantly improving dialect speech recognition performance while ensuring model convergence. The formula is shown below.

[0107] (1)

[0108] Among them, consistency loss The loss function is used to constrain the consistency of the model's predictions under different data augmentation or decoding conditions, helping the model maintain stable output under diverse input conditions. Its formula is shown below:

[0109] (2)

[0110] Among them, the minimum word error rate loss The loss function directly optimizes the word error rate, guiding model optimization at the sequence level to further improve the final recognition performance. The formula is shown below:

[0111] (3)

[0112] Application of loss function in supervised training: In the supervised training phase, the labeled cross-entropy loss is used to optimize the input samples and their corresponding ground truth labels. This loss function minimizes the negative log-likelihood of the ground truth labels, making the predicted distribution as close as possible to the true distribution, thereby improving the model's recognition accuracy on labeled data. Its formula is shown below;

[0113] (4)

[0114] Application of pseudo-label loss function in unsupervised training: In the unsupervised training phase, pseudo-label cross-entropy loss and consistency loss work together. Pseudo-label cross-entropy loss is used to optimize the distribution of pseudo-labeled data, enhancing the learning ability of unlabeled samples. The loss function uses dynamic sample weights w. This is used to measure the confidence or importance of pseudo-labels, thereby assigning higher weights to high-quality pseudo-labels during training, further improving the robustness and accuracy of semi-supervised learning. The formula is shown below.

[0115] (5)

[0116] Step 6 includes:

[0117] Double Consistency Pseudo-Label Selection: Based on the teacher model obtained in step 4, pseudo-labels are generated for the unlabeled dataset U obtained in step 3 by combining the double consistency mechanism of the global and local models. Specifically, two sets of pseudo-labels are generated by the sub-dialect area model and the global model respectively, and their character error rate (CER) is calculated; only pseudo-labels with CER values ​​below the threshold are retained as high-confidence data input for training, thereby effectively suppressing the adverse effects of noisy labels on model convergence;

[0118] Dynamic caching iteration mechanism: To avoid outdated pseudo-labels and noise accumulation, a dynamic cache pool management strategy is constructed. High-confidence pseudo-labels are stored in the cache pool after generation and refreshed periodically as training progresses. Specifically, during each training round, cached data is randomly sampled with a probability of 0.1 for updating; if an update is triggered, the old pseudo-labels of the corresponding batch are removed, and new labels are regenerated and added to the cache pool, thus ensuring that the training process continuously benefits from the latest and highest-quality pseudo-labels.

[0119] Supervised and unsupervised alternating training: The labeled data sorted in step 1 and the initial high-quality pseudo labels after double-consistency pseudo label screening are jointly input into the dynamic cache iteration mechanism, and supervised and unsupervised alternating training is performed based on the Parformer model. The training process is divided into two stages: First, supervised training is carried out using the labeled data to anchor the basic parameters of the model (such as the weights of the encoder / decoder layers and the parameters of the attention mechanism); then, unsupervised training is carried out in combination with the pseudo labels to挖掘 the dialect acoustic and language features in the unlabeled data. After each round of alternation, the pseudo labels are updated and the parameters are optimized backward until the specified number of training rounds is reached, and finally a dialect recognition model optimized by the dynamic cache iteration mechanism is obtained.

[0120] The experimental results show (see Figure 3 Table 2), this training process significantly enhances the model's recognition ability for the unique expressions and rare words in Hubei dialect (such as "嬢", "驺", "哈", "戚"), and effectively reduces insertion and deletion errors; compared with the traditional unified merging training method, under the low-resource condition of only 1-8 hours of labeled data per county, the overall recognition accuracy of the proposed scheme in this invention is increased by 5%-12%, and it provides an optimized initial parameter basis for subsequent end-to-end fine-tuning.

[0121] Table 2

[0122]

[0123] This invention has the following characteristics and effects:

[0124] 1. Improve the recognition accuracy of low-resource dialects: Through global-local multi-stage training (pre-training on the full corpus + fine-tuning on sub-dialects), under the low-resource condition of only 1-8 hours of labeled data per county, the overall recognition accuracy is increased by 5%-12%, alleviating the dependence on large-scale labeled corpora in traditional methods.

[0125] 2. Suppress the accumulation of pseudo label noise: Based on dynamic iterative pseudo label generation (double-consistency screening + dynamic cache iteration), high-confidence pseudo labels are refreshed regularly to avoid noise accumulation and label obsolescence, and improve the training stability and convergence speed.

[0126] 3. Enhance the recognition ability for fine-grained features and rare words: By combining multiple-component collaborative loss functions (labeled / pseudo label cross-entropy, consistency loss, minimum word error rate loss), the model's recognition ability for unique expressions and rare words in dialects (such as "嬢", "驺") is strengthened, and insertion / deletion errors are reduced.

[0127] 4. Scientific division of dialect regions: Through data-driven dialect region clustering (unsupervised clustering of acoustic features + linguistic verification), the county-level sub-dialect region division is achieved for the first time with a high degree of consistency with traditional linguistic division, revealing the phenomenon of regional acoustic fusion and providing quantitative tools for dialect research.

[0128] A second aspect of the present invention provides a dialect speech recognition device based on global-local multi-stage training and dynamic iterative pseudo-labels, comprising:

[0129] The dialect data collection and processing module is used to standardize the speech data of the dialect of the region to be identified and output a dialect corpus.

[0130] The data-driven dialect region clustering module is used to extract features from the acoustic feature space based on the dialect corpus to construct a unified feature space. It combines the similarity of acoustic features and linguistic verification to perform unsupervised clustering, dividing the dialect samples in the dialect corpus into several sub-dialect regions.

[0131] The dialect classifier construction module based on acoustic features is used to extract shared and discriminative features based on sub-dialect areas through an acoustic feature extractor, train a "lightweight CNN + fully connected layer" classifier, and automatically classify unlabeled data to reduce cross-dialect interference.

[0132] The global-local multi-stage training module is used to train a unified acoustic model based on sub-dialect regions. First, it uses a dialect corpus to train a unified acoustic model and learn cross-dialect shared representations. Then, it performs local fine-tuning for each sub-dialect region to generate a teacher model that enhances regional features.

[0133] The multi-component collaborative loss function system construction module is used to construct a loss function system consisting of labeled cross-entropy loss, pseudo-label cross-entropy loss, consistency loss, and minimum word error rate loss. Finally, the loss function in the supervised training stage and the loss function in the unsupervised training stage are balanced by dynamic weight adjustment to optimize the model convergence.

[0134] The dialect recognition model generation module is used to filter high-confidence pseudo-labels based on the teacher model and using the dual consistency of global and local models. The pseudo-labels are refreshed periodically through a dynamic caching mechanism. Manually labeled data is divided into regions and the filtered pseudo-labels are input into the supervised and unsupervised alternating training module. First, the labeled data is used to anchor the model parameters, and then the pseudo-labels are combined to mine the features of unlabeled data. The pseudo-labels and model parameters are iteratively updated to output the final dialect recognition model.

[0135] The dialect recognition module is used to perform endpoint detection, acoustic feature extraction, and decoding post-processing on the speech to be recognized using the final dialect recognition model, and output the dialect recognition text result.

[0136] Another aspect of the present invention provides a dialect speech recognition system based on global-local multi-stage training and dynamic iterative pseudo-labels, comprising: a computer-readable storage medium and a processor;

[0137] The computer-readable storage medium is used to store executable instructions;

[0138] The processor is used to read executable instructions stored in the computer-readable storage medium and execute the dialect speech recognition method based on global-local multi-stage training and dynamic iterative pseudo-labels as described in the first aspect.

[0139] In another aspect, the present invention provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the dialect speech recognition method based on global-local multi-stage training and dynamic iterative pseudo-labels as described in the first aspect.

[0140] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0141] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0142] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0143] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0144] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit it. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the specific implementation of the present invention. Any modifications or equivalent substitutions that do not depart from the spirit and scope of the present invention should be covered within the scope of protection of the claims of the present invention.

Claims

1. A dialect speech recognition method based on global-local multi-stage training and dynamic iterative pseudo-labels, characterized in that, Includes the following steps: Standardize the speech data of the dialect of the region to be identified and output a dialect corpus; Based on the constructed dialect corpus, feature extraction is performed in the acoustic feature space to construct a unified feature space. Unsupervised clustering is then performed by combining the similarity of acoustic features and linguistic verification to divide the dialect samples in the dialect corpus into several sub-dialect regions. Based on sub-dialect regions, shared and discriminative features are extracted through an acoustic feature extractor, and a "lightweight CNN + fully connected layer" classifier is trained to automatically classify unlabeled data to reduce cross-dialect interference. Based on dialect regions, a unified acoustic model is first trained using a dialect corpus to learn shared representations across dialects; then, local fine-tuning is performed for each sub-dialect region to generate a teacher model that enhances regional features. We construct a loss function system consisting of labeled cross-entropy loss, pseudo-label cross-entropy loss, consistency loss, and minimum word error rate loss. Finally, we optimize the model convergence by dynamically adjusting the weights to balance the loss functions in the supervised training phase and the unsupervised training phase. Based on the teacher model, high-confidence pseudo-labels are filtered using the dual consistency of global and local models, and pseudo-labels are refreshed periodically through a dynamic caching mechanism. After manually labeled data is divided into regions, it is fed into a supervised and unsupervised alternating training module along with the selected pseudo-labels. First, the labeled data is used to anchor the model parameters, and then the pseudo-labels are combined to mine the features of the unlabeled data. The pseudo-labels and model parameters are iteratively updated to output the final dialect recognition model. Using the final dialect recognition model, endpoint detection, acoustic feature extraction, and post-decoding processing are performed on the speech to be recognized, and the dialect recognition text result is output.

2. The dialect speech recognition method based on global-local multi-stage training and dynamic iterative pseudo-labels according to claim 1, characterized in that, The standardization process includes: audio file formatting, language recognition filtering, speech activity detection and segmentation, sentence-level reconstruction, and character rate filtering.

3. The dialect speech recognition method based on global-local multi-stage training and dynamic iterative pseudo-labels according to claim 2, characterized in that, The audio file formatting, language recognition filtering, speech activity detection and segmentation, sentence-level reconstruction, and character rate filtering specifically include the following steps: Audio file formatting: The original dialect audio collected is uniformly converted into a standardized mono 16 kHz WAV format; at the same time, the audio duration is subject to regular constraints, and audio with a duration of more than 4000 seconds or less than 10 seconds is removed. Language identification and filtering: The formatted audio data is input into a pre-trained dialect identification model to predict its language category; the long audio is divided into several segments, and the final language label is determined based on the segment voting mechanism; audio that does not belong to the target dialect or whose language is unclear is removed, so that the audio data entering the next stage are all target dialect data, forming a language-filtered audio set. Speech Activity Detection and Segmentation: The target dialect audio obtained through language identification and filtering is input into the speech activity detection VAD module, which includes a NeMo forced aligner. The NeMo forced aligner generates a predicted transcribed text with punctuation and capitalization information based on the Paraformer model, and forces it to align with the original audio to obtain word-level timestamps. The VAD module then uses the timestamp information to identify pause intervals in the speech, thereby dividing active speech segments and providing accurate temporal basis for sentence-level reconstruction. The segmented speech segments form structured speech units. Sentence-level reconstruction and segmentation: Based on the obtained word-level timestamps and punctuation marks, the audio is reconstructed at the sentence level, using punctuation marks as sentence boundaries. The reconstructed speech segments are then segmented into segments ranging from 1 to 20 seconds in length, ensuring that the final segments fully express the semantics and meet the training requirements of the ASR model. All audio segments are trimmed and reconstructed into audio-text pairs based on the alignment results, forming a standardized sentence-level training sample set. Character-level filtering: Sentence-level speech segments are input into the baseline ASR model for automatic transcription; the character rate (CR) is calculated based on the transcribed text, and audio files with significantly abnormal CR are removed to exclude potential noise segments, background music interference, or samples containing a large amount of non-speech content; further, low-quality transcription results are screened out based on the character error rate (CER) of the baseline ASR model to ensure that the data finally entering the dialect corpus meets the training requirements in terms of acoustic quality, language consistency, and text consistency.

4. The dialect speech recognition method based on global-local multi-stage training and dynamic iterative pseudo-labels according to claim 1, characterized in that, Based on the constructed dialect corpus, feature extraction is performed in the acoustic feature space to construct a unified feature space. Unsupervised clustering is then performed by combining the similarity of acoustic features and linguistic verification to divide the dialect samples in the dialect corpus into several sub-dialect regions. Includes the following steps: Acoustic feature extraction and embedding vector generation: Using the trained unified dialect acoustic model of the area to be identified, high-level acoustic feature representations are extracted from the selected samples, and variable-length speech sequences are mapped to fixed-length embedding vectors through the average pooling method, thereby constructing a unified feature space for subsequent cluster analysis. Determine the number of sub-dialect regions: Combining the geographical regionalization characteristics of the area to be identified and the opinions of linguistic experts, the search range of the number of cluster categories k is set to 2–13, and the sum of squared errors (SSE) is used as the evaluation index of clustering effect. The elbow method is used to determine the inflection point of the SSE curve, thereby dynamically determining the optimal number of cluster categories. This number of categories corresponds to the final number of sub-dialect regions to be divided. Sub-dialect region segmentation: Based on dialect corpus samples, under the conditions of the generated unified acoustic feature space and the determined optimal number of clustering categories, the K-means++ algorithm is used to perform unsupervised clustering of speech samples to obtain preliminary sub-dialect region segmentation results, and complete the accurate segmentation of dialect samples in the dialect corpus into several sub-dialect regions.

5. The dialect speech recognition method based on global-local multi-stage training and dynamic iterative pseudo-labels according to claim 1, characterized in that, The process, based on sub-dialect regions, extracts shared and discriminative features using an acoustic feature extractor, trains a lightweight CNN + fully connected layer classifier, and automatically categorizes unlabeled data to reduce cross-dialect interference. This includes the following steps: 31) Construction and feature extraction of acoustic feature extractor: Design an acoustic feature extractor that includes a common feature module and a discriminative feature module. The common module extracts low-level acoustic features that are common across dialects to reflect the vocal tract and spectral structure. The discriminative module extracts prosodic and intonation features that can distinguish dialects. The low-level acoustic features and prosodic and intonation features are standardized and then spliced ​​and fused to obtain the final acoustic feature vector. 32) Dialect Classifier Construction and Training: A classifier is constructed using a "lightweight CNN + fully connected layer" structure, where the CNN contains 3 convolutional layers and the output dimension of the fully connected layer corresponds to the sub-dialect category; the cross-entropy loss is used as the optimization objective, and the Adam optimizer is used for training. The training is terminated by an early stopping strategy to obtain the final classifier. 33) Automatic classification of unlabeled data: Input unlabeled data into the classifier, determine the sub-dialect region to which it belongs through category probability, and evaluate the classifier performance using classification accuracy and macro-average F1 value.

6. The dialect speech recognition method based on global-local multi-stage training and dynamic iterative pseudo-labels according to claim 1, characterized in that, The dialect-based approach first trains a unified acoustic model using a dialect corpus to learn shared representations across dialects; then, it performs local fine-tuning for each sub-dialect region to generate a teacher model that enhances regional features, including the following steps: 41) Unified model pre-training: Based on the dialect corpus, the unified dialect speech recognition model is pre-trained to learn the shared acoustic and linguistic representations across dialects, providing a general model foundation for different sub-dialect areas; 42) Sub-dialect fine-tuning: On each sub-dialect dataset, the pre-trained model is fine-tuned locally to capture and enhance regional features while maintaining cross-dialect commonalities, thereby improving the model’s local adaptability and specific dialect recognition accuracy. 43) Teacher model generation: The fine-tuned sub-dialect area models are used as teacher models to generate high-confidence pseudo-labels for unlabeled data.

7. The dialect speech recognition method based on global-local multi-stage training and dynamic iterative pseudo-labels according to claim 1, characterized in that, The construction of a loss function system consisting of labeled cross-entropy loss, pseudo-label cross-entropy loss, consistency loss, and minimum word error rate loss, and the optimization of model convergence by dynamically adjusting the weights to balance the loss functions in supervised and unsupervised training, includes the following steps: 51) Loss Function System Construction: Construct a loss function system for multi-component collaborative optimization, including four main parts: label cross-entropy loss. Pseudo-label cross-entropy loss Consistency loss and minimum word error rate loss By dynamically adjusting the weights of each part, a balance is achieved between the supervised signal and the semi-supervised signal, thereby significantly improving the dialect speech recognition performance while ensuring model convergence. The formula is shown below. (1)； Among them, consistency loss The loss function is used to constrain the consistency of the model's predictions under different data augmentation or decoding conditions, helping the model maintain stable output under diverse input conditions. Its formula is shown below: (2)； Among them, the minimum word error rate loss The loss function directly optimizes the word error rate, guiding model optimization at the sequence level to further improve the final recognition performance. The formula is shown below: (3)； 52) Application of loss function in supervised training: In the supervised training phase, the labeled cross-entropy loss is used to optimize the input samples and their corresponding real labels. This loss function minimizes the negative log-likelihood of the real labels, so that the predicted distribution is as close as possible to the real distribution, thereby improving the recognition accuracy of the model on labeled data. The formula is as follows. (4)； 53) Application of pseudo-label loss function in unsupervised training: In the unsupervised training phase, pseudo-label cross-entropy loss and consistency loss work together; pseudo-label cross-entropy loss is used to optimize the distribution of pseudo-labeled data and enhance the learning ability of unlabeled samples; in the loss function, the dynamic weights w of the samples are used. To measure the confidence or importance of pseudo-labels, higher weights are assigned to high-quality pseudo-labels during training, thereby further improving the robustness and accuracy of semi-supervised learning. The formula is shown below. (5)。 8. The dialect speech recognition method based on global-local multi-stage training and dynamic iterative pseudo-labels according to claim 1, characterized in that, The generation of the dialect recognition model includes the following steps: 61) Double Consistency Pseudo-Label Filtering: Based on the dialect corpus, language classification is performed to obtain an unlabeled dataset U. Based on the teacher model, pseudo-labels are generated in dataset U by combining the double consistency mechanism of the global model and the local model. Specifically, two sets of pseudo-labels are generated by the sub-dialect area model and the global model respectively, and their character error rate (CER) is calculated. Only pseudo-labels with CER values ​​below the threshold are retained as high-confidence data input for training, thereby effectively suppressing the adverse effects of noisy labels on model convergence. 62) Dynamic caching iteration mechanism: Construct a dynamic cache pool management strategy. High-confidence pseudo-labels are stored in the cache pool after generation and refreshed periodically with the training progress. Specifically, during each training round, cached data is randomly sampled with a probability of 0.1 for updating. If an update is triggered, the old pseudo-labels of the corresponding batch are removed, and new labels are regenerated and added to the cache pool, thereby ensuring that the training process continuously benefits from the latest and high-quality pseudo-labels. 63) Alternating Supervised and Unsupervised Training: Manually labeled data is divided into regions and then fed into a dynamic caching iteration mechanism along with initial high-quality pseudo-labels selected from double consistency pseudo-labels. Supervised and unsupervised alternating training is performed based on the Parformer model architecture. The training process is divided into two stages: first, supervised training is performed using labeled data to anchor the basic parameters of the model; then, unsupervised training is carried out in conjunction with pseudo-labels to mine dialect acoustic and linguistic features in the unlabeled data; after each round of alternation, the pseudo-labels are updated and the parameters are optimized in reverse until the specified number of training rounds is reached, and finally, a dialect recognition model optimized by the dynamic caching iteration mechanism is obtained.

9. The dialect speech recognition method based on global-local multi-stage training and dynamic iterative pseudo-labels according to claim 1, characterized in that, The process of using the final dialect recognition model to perform endpoint detection, acoustic feature extraction, and decoding post-processing on the speech to be recognized, and outputting the dialect recognition text result, includes the following steps: 71) Speech preprocessing and feature extraction: Endpoint detection is performed on the speech to be recognized to remove silence and noise segments; and acoustic features consistent with those in the training phase are extracted to form a unified acoustic feature sequence required for model input; 72) Global-local joint recognition reasoning: The acoustic features are input into the final dialect recognition model. The model first uses the global shared representation to perform cross-dialect basic recognition, and then calls the corresponding local dialect specialization module according to the area division results to output the preliminary word-level recognition results. 73) Decoder Language Modeling and Post-processing: The decoder is used to perform language modeling, sequence reordering and punctuation restoration on the model output. Combined with repeated word cleaning and format normalization post-processing steps, the standardized dialect recognition text is finally obtained, realizing the automatic conversion from dialect speech to standard text.

10. A dialect speech recognition device based on global-local multi-stage training and dynamic iterative pseudo-labels, characterized in that, include: The dialect data collection and processing module is used to standardize the speech data of the dialect of the region to be identified and output a dialect corpus. The data-driven dialect region clustering module is used to extract features from the acoustic feature space based on the dialect corpus to construct a unified feature space. It combines the similarity of acoustic features and linguistic verification to perform unsupervised clustering, dividing the dialect samples in the dialect corpus into several sub-dialect regions. The dialect classifier construction module based on acoustic features is used to extract shared and discriminative features based on sub-dialect areas through an acoustic feature extractor, train a "lightweight CNN + fully connected layer" classifier, and automatically classify unlabeled data to reduce cross-dialect interference. The global-local multi-stage training module is used to train a unified acoustic model based on sub-dialect regions. First, it uses a dialect corpus to train a unified acoustic model and learn cross-dialect shared representations. Then, it performs local fine-tuning for each sub-dialect region to generate a teacher model that enhances regional features. The multi-component collaborative loss function system construction module is used to construct a loss function system consisting of labeled cross-entropy loss, pseudo-label cross-entropy loss, consistency loss, and minimum word error rate loss. Finally, the loss function in the supervised training stage and the loss function in the unsupervised training stage are balanced by dynamic weight adjustment to optimize the model convergence. The dialect recognition model generation module is used to filter high-confidence pseudo-labels based on the teacher model, utilizing the dual consistency of global and local models, and periodically refreshing the pseudo-labels through a dynamic caching mechanism. After manually labeled data is divided into regions, it is fed into a supervised and unsupervised alternating training module along with the selected pseudo-labels. First, the labeled data is used to anchor the model parameters, and then the pseudo-labels are combined to mine the features of the unlabeled data. The pseudo-labels and model parameters are iteratively updated to output the final dialect recognition model. The dialect recognition module is used to perform endpoint detection, acoustic feature extraction, and decoding post-processing on the speech to be recognized using the final dialect recognition model, and output the dialect recognition text result.

Images