Course reinforcement learning method, device and equipment for low-resource dialect recognition

Abstract

The invention provides a course reinforcement learning method, device and equipment for low-resource dialect recognition, relates to the technical field of artificial intelligence, and aims to solve the problems that in a low-resource dialect speech recognition scene, existing reinforcement learning enables a model not to perform effective learning and performance improvement is limited. The method comprises the following steps: performing supervised fine tuning on an initial speech recognition model by adopting a low-resource dialect speech data set to obtain a preliminary optimization model; recognizing the voice data based on the preliminary optimization model, and calculating a complexity index for reflecting the recognition difficulty of the voice data according to a voice recognition result and the labeled reference answer; according to the complexity index, sorting the voice data from low to high according to the recognition difficulty to form a training data sequence of course learning; and applying reinforcement learning on the training data sequence, and optimizing, iteratively updating the preliminary optimization model through a dynamic reward function and a group relative strategy until a preset convergence condition is met, thereby obtaining a target dialect recognition model.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and in particular to a curriculum reinforcement learning method, apparatus, and device for low-resource dialect recognition. Background Technology

[0002] Speech recognition technology, especially large-scale speech models based on pre-trained models, has become a key entry point in the field of human-computer interaction.

[0003] Currently, several improvement schemes have been proposed for model training under low-resource conditions. One approach combines curriculum learning with reinforcement learning, improving training efficiency by having the model learn from easy to difficult samples. However, this approach often uses simple binary reward signals (e.g., 1 for success, 0 for failure), which are sparse in complex speech recognition tasks, resulting in inefficient use of scarce samples and potential underlearning of difficult samples. Another approach introduces a phased training strategy, using data-rich source languages ​​(e.g., Mandarin) for initial training, followed by fine-tuning on low-resource target dialects. However, this approach still relies on additional large-scale source language datasets, failing to completely eliminate dependence on non-target language resources.

[0004] Therefore, in the current low-resource dialect recognition scenario, there is an urgent need for a solution that can efficiently utilize limited data and achieve stable performance improvement of the model without the need for external language resources. Summary of the Invention

[0005] This invention provides a curriculum reinforcement learning method, apparatus, and device for low-resource dialect recognition, which addresses the shortcomings of existing reinforcement learning methods in low-resource dialect speech recognition scenarios, where models cannot learn effectively and have limited performance improvements, thereby achieving stable performance enhancement of the model.

[0006] This invention provides a course reinforcement learning method for low-resource dialect recognition, comprising the following steps: Supervised fine-tuning of an initial speech recognition model using a low-resource dialect speech dataset to obtain a preliminary optimized model. The low-resource dialect speech dataset refers to a set of dialect data with an annotated speech data volume not exceeding a preset threshold. Based on the preliminary optimized model, speech data is recognized to obtain speech recognition results. A complexity index is calculated for each speech data based on the speech recognition results and annotated reference answers. The complexity index reflects the recognition difficulty of the speech data. According to the complexity index, the speech data are sorted from low to high recognition difficulty to form a training data sequence for course learning. Reinforcement learning is applied to the training data sequence, and the preliminary optimized model is iteratively updated through a dynamic reward function and a group-relative strategy until a preset convergence condition is met, resulting in a target dialect recognition model.

[0007] According to the curriculum reinforcement learning method for low-resource dialect recognition provided by the present invention, the initial speech recognition model includes a speech encoding module, a feature adaptation module, and a text decoding module; wherein, the speech encoding module is used to extract feature vectors of dialect speech, the feature adaptation module is used to realize the mapping between feature vectors and text semantic space, and the text decoding module is used to convert semantically aligned features into text recognition results.

[0008] According to the course reinforcement learning method for low-resource dialect recognition provided by the present invention, the above-mentioned supervised fine-tuning of the initial speech recognition model using a low-resource dialect speech dataset to obtain a preliminary optimized model includes: extracting features from the speech data through a speech coding module to obtain speech feature vectors adapted to the characteristics of dialect speech; mapping the speech feature vectors to the text semantic space through a feature adaptation module to obtain semantic alignment features; and fine-tuning the text decoding module based on the semantic alignment features and labeled reference answers to obtain a preliminary optimized model.

[0009] According to the course-based reinforcement learning method for low-resource dialect recognition provided by the present invention, the process of generating the dynamic reward value of the aforementioned dynamic reward function includes: for each speech data, calculating a basic reward value reflecting the overall recognition accuracy based on the word error rate between the basic speech recognition result and the labeled reference answer; calculating an adjusted reward value reflecting the local error distribution based on the number of error markers in the speech recognition result; and generating the dynamic reward value for each speech data by dynamically weighting and combining the basic reward value and the adjusted reward value.

[0010] According to the present invention, a curriculum-based reinforcement learning method for low-resource dialect recognition is provided. The method applies reinforcement learning to a training data sequence, iteratively updating the initial optimized model through a dynamic reward function and group-relative policy optimization until a preset convergence condition is met, thereby obtaining a target dialect recognition model. The method includes: employing group-relative policy optimization, grouping speech data to calculate intra-group reward values, and adjusting the policy gradient corresponding to each speech data point in conjunction with the dynamic reward values ​​to obtain the adjusted policy gradient; iteratively updating the parameters of the initial optimized model using the adjusted policy gradient; and determining that the preset convergence condition is met when the performance fluctuation after the model parameter update is less than a preset threshold, thereby obtaining the target dialect recognition model.

[0011] According to the present invention, a course-based reinforcement learning method for low-resource dialect recognition is provided. The method employs a group-relative strategy optimization to group speech data and calculate the intra-group reward value. This includes: dividing the speech data into multiple data groups according to a complexity index; calculating the average reward value and reward dispersion of the dynamic reward value within each data group to obtain the intra-group reward value.

[0012] According to the curriculum reinforcement learning method for low-resource dialect recognition provided by the present invention, the above-mentioned method of adjusting the policy gradient corresponding to each speech data by combining dynamic reward value to obtain the adjusted policy gradient includes: standardizing the dynamic reward value of each speech data based on the average reward and reward dispersion to obtain a standardized reward value; calculating the dominance function using the standardized reward value, and adjusting the initial policy gradient based on the dominance function to obtain the adjusted policy gradient.

[0013] According to the present invention, a course reinforcement learning method for low-resource dialect recognition is provided. The above-mentioned method calculates an advantage function using standardized reward values ​​and adjusts the initial policy gradient based on the advantage function to obtain the adjusted policy gradient. The method includes: constructing an advantage function that reflects the relative advantage of the data based on the difference between the standardized reward value and the average reward value; calculating the initial policy gradient based on the advantage function and constraining the initial policy gradient to obtain the adjusted policy gradient, so as to ensure that the policy update magnitude is within a preset reasonable range.

[0014] According to the low-resource dialect recognition course reinforcement learning method provided by the present invention, before calculating the complexity index of each speech data based on the speech recognition results and the labeled reference answer, the word error rate of the speech data can also be calculated for each speech data; and the number of erroneous markers in the speech recognition results that are inconsistent with the labeled reference answer can be counted.

[0015] According to the course reinforcement learning method for low-resource dialect recognition provided by the present invention, the above-mentioned calculation of the complexity index of each speech data based on the speech recognition results and the labeled reference answers includes: normalizing the word error rate and the number of error tags respectively to obtain the standardized word error rate and the number of standardized error tags; and weighting and summing the standardized word error rate and the number of standardized error tags according to a preset weight ratio to generate the complexity index.

[0016] According to the present invention, a course reinforcement learning method for low-resource dialect recognition is provided, wherein the above-mentioned method sorts speech data from low to high recognition difficulty according to the complexity index to form a training data sequence for course learning, including: dividing the complexity index into multiple consecutive numerical intervals, each numerical interval corresponding to a difficulty level; and combining speech data in order of difficulty level from low to high to form a training data set from low complexity to high complexity.

[0017] This invention also provides a course reinforcement learning device for low-resource dialect recognition, comprising the following modules: an adjustment module, used to perform supervised fine-tuning of an initial speech recognition model using a low-resource dialect speech dataset to obtain a preliminary optimized model, wherein the low-resource dialect speech dataset refers to a set of dialect data with an annotated speech data amount not exceeding a preset threshold; a generation module, used to recognize the speech data based on the preliminary optimized model to obtain speech recognition results, and calculate the complexity index of each speech data according to the speech recognition results and the annotated reference answers, wherein the complexity index is used to reflect the recognition difficulty of the speech data; a construction module, used to sort the speech data from low to high recognition difficulty according to the complexity index to form a training data sequence for course learning; and a training module, used to apply reinforcement learning on the training data sequence, and iteratively update the preliminary optimized model through a dynamic reward function and a group relative strategy until a preset convergence condition is met to obtain a target dialect recognition model.

[0018] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the course reinforcement learning method for low-resource dialect recognition as described above.

[0019] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the course reinforcement learning method for low-resource dialect recognition as described above.

[0020] The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements a course reinforcement learning method for low-resource dialect recognition as described above.

[0021] The present invention provides a course-based reinforcement learning method, apparatus, and device for low-resource dialect recognition. Through supervised fine-tuning, it establishes basic dialect recognition capabilities under low-resource conditions. Subsequently, by quantifying the recognition difficulty of speech samples and constructing a course learning sequence from easy to difficult, it improves the utilization efficiency of limited training data. Then, it introduces a reinforcement learning mechanism that integrates dynamic rewards and group-based policy optimization on this ordered learning path. This mechanism not only accurately guides the model optimization direction through fine-grained reward signals but also effectively maintains training stability through inter-group comparison strategies. This overcomes the training oscillation or overfitting problems that traditional methods are prone to in low-resource scenarios, thereby achieving a steady improvement in the model's recognition performance under strictly limited dialect data conditions. Attached Figure Description

[0022] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0023] Figure 1 This is a flowchart illustrating a course reinforcement learning method for low-resource dialect recognition provided by the present invention.

[0024] Figure 2 This is a schematic diagram of the structure of a low-resource dialect recognition course reinforcement learning device provided by the present invention.

[0025] Figure 3 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation

[0026] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0027] The following is combined Figure 1 This invention describes a curriculum reinforcement learning method for low-resource dialect recognition.

[0028] Figure 1 This is a flowchart illustrating a low-resource dialect recognition course reinforcement learning method provided by the present invention, as shown below. Figure 1 As shown, the method includes the following: S101. The initial speech recognition model is fine-tuned in a supervised manner using a low-resource dialect speech dataset to obtain a preliminary optimized model.

[0029] In this embodiment of the application, a low-resource dialect speech dataset refers to a set of dialect data whose labeled speech data does not exceed a preset threshold.

[0030] For example, the preset threshold can be a value set arbitrarily, which can be flexibly adjusted according to the actual scenario. For instance, the preset threshold can be 100 hours, because low-resource definitions in the dialect field typically do not exceed a corpus size of 100 hours.

[0031] In specific implementation, the preset threshold can be determined through cross-validation. For example, in the model development stage, experiments are carried out using datasets of different sizes (such as 50h, 100h, 150h), and the curve of the model performance (such as character accuracy rate) changing with the data volume is observed. The data volume when the performance growth tends to be flat can be set as the threshold.

[0032] Optionally, the preset threshold can also be adjusted according to the dialect type. For dialects with complex phonetic systems (such as Cantonese), the threshold can be set higher (such as 120h), while for dialects with simple phonetic systems (such as some Mandarin variants), the threshold can be set lower (such as 80h).

[0033] In the embodiment of the present application, the initial speech recognition model includes a speech encoding module, a feature adaptation module, and a text decoding module.

[0034] Exemplarily, the speech encoding module is used to extract the feature vector of the dialect speech; the feature adaptation module is used to realize the mapping between the feature vector and the text semantic space; the text decoding module is used to convert the semantic alignment feature into a text recognition result.

[0035] In some embodiments, the speech data can be first subjected to feature extraction through the speech encoding module to obtain a speech feature vector suitable for the characteristics of the dialect speech.

[0036] Exemplarily, the speech encoding module can select the encoder of a pre-trained large speech model with multi-task and multi-lingual speech processing capabilities, such as the encoder of the Whisper Large model.

[0037] Specifically, taking the speech data of a certain low-resource Cantonese in the Guangfu area of Guangdong as an example. The Whisper Large encoder can convert the input Cantonese speech segment with a sampling rate of 16kHz (such as "今日去街市买餸", which means "Today go to the market to buy groceries") into a speech feature vector with dimensions of [300, 512]. This vector can effectively capture the dialect characteristics such as the "nine tones and six pitches" and rhythm of Cantonese.

[0038] Among them, 300 is the time step, calculated from 3 seconds of speech at a frame length of 25ms and a frame shift of 10ms; 512 is the feature dimension.

[0039] In some embodiments, the speech feature vector is then mapped to the text semantic space through the feature adaptation module to obtain a semantic alignment feature.

[0040] Exemplarily, the feature adaptation module can be implemented by a lightweight adapter module, which usually includes two linear layers (a down-project layer and an up-project layer) and an activation function (such as the gaussian error linear unit, GELU).

[0041] Among them, the down-project layer is used to reduce the dimension of the high-dimensional feature vector output by the speech encoding module (for example, reducing the dimension from 512 to 128). After introducing a non-linear transformation (enhancing the feature expression ability) through the GELU activation function, the dimension is then restored to the dimension matching the input dimension of the text decoding module (for example, restored to 768) through the up-project layer.

[0042] Specifically, taking the Cantonese speech feature vector as an example, the lightweight adapter module can map the [300, 512] feature vector output by the speech encoding module to a [300, 768] semantic alignment feature, so that this feature can be accurately understood by the text decoding module. For example, it maps the speech feature of the Sichuan dialect sentence "今日去街市买餸" (meaning "Today go to the wet market to buy groceries") to a feature vector that semantically matches the text "今日去街市买餸".

[0043] Furthermore, based on the semantic alignment feature and the annotated reference answer, the text decoding module is fine-tuned to obtain a preliminary optimized model.

[0044] Exemplarily, the text decoding module can select a pre-trained large language model, such as the Bidirectional and Auto-Regressive Transformers (BART) model. Its input is the semantic alignment feature output by the feature adaptation module, and the output is a dialect text sequence.

[0045] Specifically, taking a low-resource Cantonese speech dataset as an example, the annotated reference answer is "今日去街市买餸" (meaning "Today go to the wet market to buy groceries"), and the semantic alignment feature is the [300, 768] feature vector corresponding to this speech. During fine-tuning, this semantic alignment feature is input into the BART model. The initial output of the model may be "今日去XXX买餸" ("XX" is the inaccurately recognized "wet market"). By calculating the cross-entropy loss between the model output and the annotated reference answer (the initial loss value is about 1.2), the low-rank matrix parameters of the low-rank adaptation (LoRA) are updated through backpropagation. After 20 rounds of iteration, the model output gradually approaches "今日去街市买餸".

[0046] In the embodiment of the present application, when the character error rate (CER) of the sample drops below 5%, it indicates that the fine-tuning effect corresponding to the sample meets the standard. At this time, the initial speech recognition model of this iteration is used as the preliminary optimized model.

[0047] In this way, through supervised fine-tuning, the present application uses limited low-resource dialect annotation data to enable the initial speech recognition model to quickly adapt to the speech characteristics (such as tones and pronunciation habits) and semantic rules (such as dialect vocabulary and grammar structures) of the target dialect, establishing the basic ability of dialect recognition, and avoiding the problem of low training efficiency caused by the too low initial performance of the model when directly entering the reinforcement learning stage, providing a good model initialization basis for subsequent curriculum reinforcement learning (CRL).

[0048] S102. Recognize the speech data based on the preliminary optimized model to obtain a speech recognition result.

[0049] In some embodiments, the low-resource dialect speech data to be recognized can be input into the preliminary optimized model. The speech feature vector is extracted through the speech encoding module in sequence, the semantic alignment feature is obtained through the feature adaptation module, and the dialect text sequence is output through the text decoding module, so as to obtain the speech recognition result.

[0050] Exemplarily, taking the low-resource Fuzhou dialect speech data as an example, the speech content to be recognized is "伊去闽江公园散步" (meaning "He goes for a walk in Minjiang Park"), the original sampling rate of this speech is 22 kHz and the duration is 3 seconds. First, it is converted into a speech segment with a sampling rate of 16 kHz and no noise through preprocessing.

[0051] Furthermore, after the converted speech segment is input into the preliminary optimized model, the speech encoding module (WhisperLarge) outputs a speech feature vector of [300, 512], the feature adaptation module maps it to a semantic alignment feature of [300, 768], and the text decoding module (BART) outputs the recognition result based on this feature in an autoregressive generation manner.

[0052] Specifically, if there are slight errors in the model, it may output "伊去闽江XX散步" (where "XX" is the "公园" that is not accurately recognized). The text containing the complete character sequence is the speech recognition result of this speech data. At the same time, the model will output the confidence of each character, and the characters with a confidence lower than 0.5 can be marked as potential errors for subsequent complexity metric calculation.

[0053] For example, the confidence of "伊" is 0.98, the confidence of "闽江" is 0.92, and the confidence of "XX" is 0.35.

[0054] S103. Calculate the complexity index of each voice data according to the voice recognition result and the marked reference answer.

[0055] Among them, the complexity index is used to reflect the recognition difficulty of the voice data.

[0056] In some embodiments, for each voice data, the word error rate of the voice data can be calculated first, and the number of error marks inconsistent with the marked reference answer in the voice recognition result can be counted.

[0057] Exemplarily, taking the low-resource Suzhou dialect voice data as an example, the marked reference answer is "Suzhou Gardens are a world cultural heritage" (total number of characters is 11, including "Su", "zhou", "gar", "den", "are", "a", "world", "cul", "tur", "al", "heritage"), and the voice recognition result output by the preliminary optimization model is "Suzhou garXX are a world cultural heritage", where "XX" is an incorrect recognition and should actually be "den").

[0058] Specifically, when calculating the CER, first count the number of errors in three categories: insertion (characters that are in the recognition result but not in the marked answer), deletion (characters that are in the marked answer but not in the recognition result), and substitution (cases where the characters in the recognition result are different from the marked characters): Here, "den" is replaced by "XX", which belongs to 1 substitution error (regardless of the number of characters replaced, it is counted as 1 substitution error), there are no insertion and deletion errors, and the total number of error characters is 1; According to the CER calculation formula: CER = (number of insertion errors + number of deletion errors + number of substitution errors) / total number of marked characters, the CER of this sample can be obtained as (0 + 0 + 1) / 11 ≈ 9.09%.

[0059] At the same time, count the number of error marks: "XX" in the recognition result is an error mark, with a total of 2 error characters, so the number of error marks for this sample is 2.

[0060] Furthermore, the word error rate and the number of error marks can be normalized respectively to obtain the standardized word error rate and the standardized number of error marks.

[0061] Exemplarily, assume that a low-resource dialect voice data set contains 1000 samples. By statistics, the CER range of all samples is [1.2%, 45.6%], and the number of error marks range is [1, 15].

[0062] Specifically, when normalizing the CER, first count the minimum word error rate and the maximum word error rate of the entire low-resource dialect voice data set, then subtract the minimum word error rate from the word error rate of each sample, and divide the obtained difference by the difference between the maximum word error rate and the minimum word error rate. Finally, the standardized word error rate is obtained. The value range of this standardized word error rate falls within the interval [0, 1], which is convenient for subsequent weighted calculation with other indicators.

[0063] For example, taking the Suzhou dialect sample with a CER of 9.09% as an example, the calculation process of its standardized character error rate is as follows: subtract 1.2% from 9.09% to get 7.89%, then divide 7.89% by the difference between 45.6% and 1.2%, which is 44.4%, and finally get a standardized character error rate of about 0.178.

[0064] Specifically, when normalizing the number of mislabeled items, we first count the minimum and maximum number of mislabeled items in the dataset. Then, we subtract the minimum number of mislabeled items from the number of mislabeled items for each sample, and divide the result by the difference between the maximum and minimum number of mislabeled items to obtain the standardized number of mislabeled items, which also ranges from [0,1]. If the number of mislabeled items for a sample is 2, the standardized number of mislabeled items is calculated as follows: subtract 1 from 2 to get 1, and then divide 1 by the difference between 15 and 14, resulting in a standardized number of mislabeled items of approximately 0.071.

[0065] Furthermore, a complexity index can be generated by weighting and summing the standardized character error rate and the number of standardized error tags according to a preset weight ratio.

[0066] In this embodiment, the preset weight ratio can be a dynamic reward coefficient α:β (e.g., 0.6:0.4), where α is the weight of the reverse indicator corresponding to the standardized character error rate, β is the weight of the reverse indicator corresponding to the number of standardized error marks, and α+β=1. The reverse indicator refers to "1-standardized character error rate" and "1-standardized error mark number", which reflect the proportion of correct characters and the proportion of correct marks in the sample, respectively. The higher the reverse indicator value, the lower the difficulty of sample recognition.

[0067] For example, the preset weight ratio can be dynamically adjusted according to the recognition performance during the model training process. For instance, when the model's ability to recognize incorrect labels is weak, β can be adjusted to 0.5.

[0068] Specifically, taking the Suzhou dialect sample with a standardized character error rate of 0.178 and a standardized error marker count of 0.071 as an example, we first calculate the reverse index: 1 - 0.178 = 0.822, 1 - 0.071 = 0.929; then, we calculate the dynamic reward value by weighting according to the following formula: .

[0069] in: r represents the dynamic reward value of a single speech sample, with a value range of [0, 1]. The higher the value, the higher the accuracy of sample recognition and the lower the difficulty. α and β are dynamic reward coefficients that satisfy α>0, β>0 and α+β=1, and are used to balance the contribution of word error rate and number of error tags; E[·] represents the mathematical expectation symbol, due to the fact that in single-sample calculations... "and" "All of these are definite values, therefore the expected result is equal to itself; The annotations are for reference answers to the samples; The model inference results for the samples; The word error rate (CER) of the model's reasoning results relative to the labeled reference answer is calculated using the formula: "CER = (Number of insertion errors + Number of deletion errors + Number of replacement errors) / Total number of labeled characters". This is a mapping function for the number of error tags, here "1-normalized error tag count", used to convert the number of error tags into a positive index of the same order of magnitude as "1-CER"; This represents the number of incorrect labels in the sample inference results.

[0070] Subsequently, by substituting the numerical values ​​of the Suzhou dialect sample above into the formula, we can obtain the dynamic reward value r = 0.6 × 0.822 + 0.4 × 0.929 = 0.8648, and the complexity index is 1 - r = 0.1352.

[0071] The higher the dynamic reward value, the higher the accuracy of sample recognition, the fewer the error labels, the lower the corresponding complexity index, and the easier the sample recognition. Conversely, the lower the dynamic reward value, the higher the complexity index, and the more difficult the sample recognition.

[0072] Thus, by combining word error rate and number of error tags to calculate complexity index, this application effectively solves the evaluation bias problem that exists when only CER is used to evaluate the difficulty of samples in the traditional way. It can more accurately quantify the recognition difficulty of each low-resource dialect speech sample, lay the foundation for the subsequent construction of a course learning sequence from easy to difficult, ensure that the model can learn step by step in the reinforcement learning stage, and improve the utilization efficiency of limited training data.

[0073] S104. Based on the complexity index, sort the speech data from low to high recognition difficulty to form a training data sequence for course learning.

[0074] In some embodiments, the complexity index can be divided into multiple consecutive numerical ranges.

[0075] Each numerical range corresponds to a difficulty level.

[0076] For example, assuming the complexity index ranges from [0,1], it can be divided into three consecutive numerical intervals: [0, 0.3) corresponds to low difficulty level, [0.3, 0.7) corresponds to medium difficulty level, and [0.7, 1] corresponds to high difficulty level.

[0077] Specifically, taking a low-resource Hakka speech dataset as an example, if the complexity index of a sample is 0.1352 (such as the Suzhou dialect sample mentioned above), it falls into the interval [0, 0.3) and is considered a low-difficulty sample; if the complexity index of a sample is 0.52, it falls into the interval [0.3, 0.7) and is considered a medium-difficulty sample; if the complexity index of a sample is 0.85, it falls into the interval [0.7, 1] and is considered a high-difficulty sample.

[0078] Optionally, the number and boundaries of the numerical interval divisions can be adjusted according to the size of the dataset. For example, when the dataset is large (more than 5,000 samples), it can be divided into 5 difficulty levels to improve the precision of the difficulty division.

[0079] Furthermore, speech data can be combined in order of increasing difficulty to form a training data set ranging from low to high complexity.

[0080] For example, taking a low-resource Minnan dialect speech dataset containing 3000 samples as an example, there are 900 low-difficulty samples, 1200 medium-difficulty samples, and 900 high-difficulty samples. When combining them according to difficulty level from low to high, first, the 900 low-difficulty samples are arranged in random order to form a low-difficulty training subset; then, the 1200 medium-difficulty samples are randomly arranged to form a medium-difficulty training subset; finally, the 900 high-difficulty samples are randomly arranged to form a high-difficulty training subset.

[0081] Specifically, the three training subsets can be concatenated in the order of "low difficulty → medium difficulty → high difficulty" to obtain a training data sequence with a total length of 3000. At different stages of the course, the model will learn the low, medium, and high difficulty subsets in sequence. For example, the first week will learn the low difficulty subset, the second week will learn the medium difficulty subset, and the third week will learn the high difficulty subset, thus achieving a progressive increase in difficulty.

[0082] Thus, this application constructs a training data sequence that conforms to the course learning concept by dividing speech data into different difficulty levels according to complexity indicators and combining them in an order of "from easy to difficult". This enables the model to first master the dialect features of simple samples (such as common words and standard pronunciation) in the reinforcement learning stage, and then gradually learn the features of complex samples (such as rare words and tone sandhi). This avoids the problem of the model being biased towards simple samples and ignoring complex samples due to directly learning samples of all difficulty levels, and improves the utilization efficiency of low-resource data.

[0083] S105. Apply reinforcement learning to the training data sequence, and iteratively update the preliminary optimization model through a dynamic reward function and a group relative policy until the preset convergence condition is met, obtaining the target dialect recognition model.

[0084] In some embodiments, for each speech data, based on the word error rate between the speech recognition result and the annotated reference answer, a basic reward value reflecting the overall recognition accuracy can be calculated.

[0085] Among them, the basic reward value is used to reflect the overall recognition accuracy of the sample, and the higher the value, the better the overall recognition effect.

[0086] Exemplarily, taking the speech sample of the Nanchang dialect in Gan language with low resources as an example, the annotated reference answer is "今朝礼拜六,去八一广场玩" (meaning "Today is Saturday, go to Bayi Square to play"), and the recognition result is "今朝礼拜六,去八一XX玩" ("XX" corresponds to "Square"). The calculated CER=(0 + 0 + 1) / 12≈8.33%, then the basic reward value is "1 - CER = 0.9167.

[0087] In some embodiments, for each speech data, based on the number of error markers in the speech recognition result, an adjustment reward value reflecting the local error distribution can be calculated.

[0088] Among them, the adjustment reward value is used to reflect the local error distribution of the sample, and the higher the value, the fewer the error markers and the more accurate the local recognition; it is used to supplement the deficiency of the basic reward value in local error evaluation.

[0089] Exemplarily, continuing the above Gan language sample, "XX" in the recognition result is an error marker, and the number of error markers = 2. The range of the number of error markers in the dataset to which this sample belongs is [1, 10]. After normalization, the standardized number of error markers is (2 - 1) / (10 - 1)≈0.111, then the adjustment reward value is "1 - standardized number of error markers = 0.889.

[0090] Furthermore, by dynamically weighting and combining the basic reward value and the adjustment reward value, a dynamic reward value for each speech data is generated.

[0091] Exemplarily, continuing the above Gan language sample, set the dynamic reward coefficients α = 0.6, β = 0.4, the basic reward value is 0.9167 (corresponding to " "), the adjustment reward value is 0.889 (corresponding to " "), substitute into the dynamic reward function formula, and get r = 0.9056.

[0092] In some embodiments, after obtaining the dynamic reward value, group-relative policy optimization can be adopted to group the speech data, calculate the reward value within the group, and adjust the policy gradient corresponding to each speech data in combination with the dynamic reward value to obtain the adjusted policy gradient.

[0093] For example, the 3000 samples of the Gan language dataset can be divided into 10 data groups according to the complexity index (each group of samples has similar complexity), and a certain data group (complexity index range [0.4, 0.5)) can be selected to perform group relative policy optimization (GRPO) calculation.

[0094] Specifically, the dynamic reward values ​​of the 300 samples within the group are first calculated to obtain the dynamic reward value set {r1, r2, ..., r...}. 300}, and their average value is: mean(r) = (r1 + r2 + ... + r 300 ) / 300≈0.82.

[0095] At the same time, calculate the set of error marker counts within the group {T1, T2, ..., T}. 300 Its standard deviation is: std(T) = ≈1.5.

[0096] Where mean(T) is the average number of mislabeled samples within the group, which is 3. Selecting the i-th sample within the group (e.g., dynamic reward value rᵢ=0.9056, number of mislabeled samples Tᵢ=2), first calculate its within-group dominance function Ai. The formula for Ai is as follows: .

[0097] in: It represents the advantage function value of the i-th speech sample within its data group, which measures the relative advantage of the sample's dynamic reward value compared to other samples in the group; This represents the dynamic reward value for the i-th speech sample, which is 0.9056 in this case. Let G represent the average dynamic reward value of all c samples in the data group to which the i-th sample belongs, where G=300 and mean(r)=0.82; T represents the number of mislabeled samples among all c samples in the dataset to which the i-th sample belongs. cnt The standard deviation is 1.5 in this case. These represent the dynamic reward values ​​of the 1st to cth samples in the data set, respectively. T represents the number of mislabeled samples from the 1st to the cth in the dataset. cnt .

[0098] Subsequently, by substituting the numerical values, A was obtained. i =(0.9056-0.82) / 1.5≈0.0571.

[0099] For example, after calculating the advantage function A i Then, based on the advantage function A i The policy gradient of GRPO is calculated, and the policy gradient adjustment formula is as follows: .

[0100] in: The gradient of the objective function θ represents the policy network parameter θ, which is used to update the model parameters and optimize model performance. This indicates that the gradient of the objective function is positively correlated with the expectation term on the right side. E[·] represents the mathematical expectation, which is used to average the policy gradient contribution of all samples in the group to reduce random fluctuations; This represents the semantic alignment features of a given speech sample under the current policy network parameters θ. At that time, the model takes action. (i.e., output the recognition result of this sample) The log probability reflects the model's confidence in the action; here, we assume the log probability is -0.12, and the confidence level is approximately 0.887. This represents the action taken by the model for the i-th sample, i.e., the speech recognition result of that sample. ; This represents the state of the i-th sample, i.e., the semantic alignment feature obtained after the sample has been processed by the speech coding module and the feature adaptation module; This indicates that, given semantic alignment features under policy network parameters θ, Take action at the time The probability of this is 0.887 in this case. This refers to the intragroup dominance function value of the i-th sample calculated earlier, which is 0.0571 here.

[0101] Subsequently, the contribution of this sample to the policy gradient is calculated as: log(0.887)×0.0571≈−0.00685. The average of the contribution values ​​of all 300 samples in the group is used to obtain the policy gradient adjustment value corresponding to the group, which is the adjusted policy gradient.

[0102] Furthermore, after obtaining the adjusted policy gradient, the parameters of the preliminary optimized model can be iteratively updated using the adjusted policy gradient. If the performance fluctuation after the model parameter update is less than a preset threshold, it is determined that the preset convergence condition is met, and the target dialect recognition model is obtained.

[0103] In this application embodiment, performance fluctuation less than a preset threshold can mean that the word error rate (CER) of the model on the validation set remains stable for a preset number of rounds (e.g., 3 rounds) and does not exceed a preset threshold (e.g., 15%), or that the fluctuation of the model's loss value for a preset number of rounds (e.g., 5 rounds) does not exceed a preset range (e.g., 0.01).

[0104] For example, taking a low-resource Changsha dialect speech dataset of Xiang dialect as an example, the preset convergence condition is set as "validation set CER ≤ 15% for 3 consecutive rounds". If the validation set CER is 18.2% in the first round, the condition is not met; if it drops to 16.5% in the second round, the condition is still not met; if it drops to 14.8% in the third round, the condition is met; if it remains at 14.5% in the fourth round and 14.3% in the fifth round, and is ≤ 15% for 3 consecutive rounds, then the model is determined to meet the convergence condition and training is stopped.

[0105] For example, continuing the training of the Gan language model described above, an Adam optimizer with weight decay (adamwith weight decay, AdamW) can be used (with a learning rate of 1e-4 and a weight decay coefficient of 1e-5). The adjusted policy gradient is used as the input to the optimizer. Only the LoRA low-rank matrix parameters and the lightweight adapter module parameters of the model are updated, without updating the pre-training parameters of the speech coding module, to ensure the stability of the model's basic capabilities.

[0106] Specifically, after the first parameter update, the model's CER on the validation set decreased from 18.2% to 16.5%; after the fifth update, it decreased to 14.8%; after the eighth update, it decreased to 14.3%, and remained between 14.3% and 14.5% for three consecutive rounds, satisfying the preset convergence condition, and the update stopped. At this point, the model is the target dialect recognition model for the Nanchang dialect of Gan language.

[0107] Thus, this application provides comprehensive feedback signals through a dynamic reward function, and ensures training stability by combining group-relative strategy optimization. The model is iteratively optimized step by step on the training data sequence of course learning, and finally obtains a target recognition model with stable performance and adaptability to low-resource dialects.

[0108] In the course-based reinforcement learning method for low-resource dialect recognition provided in this application embodiment, supervised fine-tuning is used to establish basic dialect recognition capabilities under low-resource conditions. Subsequently, by quantifying the recognition difficulty of speech samples and constructing a course learning sequence from easy to difficult, the utilization efficiency of limited training data is improved. Then, a reinforcement learning mechanism that integrates dynamic rewards and group-based policy optimization is introduced on this ordered learning path. This not only accurately guides the model optimization direction through fine-grained reward signals, but also effectively maintains training stability through inter-group comparison strategies. This overcomes the training oscillation or overfitting problems that traditional methods are prone to in low-resource scenarios, thereby achieving a steady improvement in the model's recognition performance under strictly limited dialect data conditions.

[0109] Alternatively, the above-mentioned adjusted policy gradient can be implemented in the following way.

[0110] In some embodiments, the voice data can be divided into multiple data groups according to the complexity index, and the average reward value and reward dispersion of the dynamic reward value within each data group can be calculated to obtain the reward value within the group.

[0111] Within the same group, the complexity levels of the data are similar.

[0112] For example, similar complexity levels mean that the difference between the maximum and minimum complexity index of all samples in the same data group does not exceed a preset threshold (such as 0.2), ensuring that the difficulty difference of samples in the group is small.

[0113] Specifically, taking a low-resource dataset of the Huangshan dialect of Hui language (e.g., 4000 samples, complexity index range [0,1]) as an example, the dataset can be divided into 5 groups at intervals of 0.2 based on the complexity index: Group 1 ([0, 0.2)), Group 2 ([0.2, 0.4)), Group 3 ([0.4, 0.6)), Group 4 ([0.6, 0.8)), and Group 5 ([0.8, 1]), with 800 samples in each group. The maximum difference in complexity index between samples within a group is 0.2, which meets the requirement of similar complexity levels.

[0114] In this embodiment, the reward dispersion is used to reflect the degree of concentration of reward values ​​within a group. The smaller the dispersion, the more uniform the reward distribution of the samples within the group.

[0115] For example, to calculate the average reward value of group 3: sum the dynamic reward values ​​of 800 samples in group 3, divide by 800, and you get an average reward value of approximately 0.75.

[0116] For example, to calculate the reward dispersion (i.e., standard deviation): calculate the sum of squares of the differences between the dynamic reward value and the average reward value for each sample, divide by the number of samples, and take the square root to obtain a reward dispersion of approximately 0.08.

[0117] Furthermore, the dynamic reward value of each speech data can be standardized based on the average reward and the reward dispersion to obtain a standardized reward value.

[0118] For example, continuing the calculation of the above-mentioned symbol group 3, the dynamic reward value r of a certain sample i =0.82, the mean reward value within the group is mean(r)=0.75, and the reward dispersion (standard deviation) is std(r)=0.08. Therefore, the calculation logic for the standardized reward value is: (r i -mean(r)) / std(r), that is (0.82-0.75) / 0.08=0.875.

[0119] Optionally, if the calculated standardized reward value exceeds the set range (e.g., [-3, 3]), it can be determined as an outlier.

[0120] Furthermore, the advantage function can be calculated using standardized reward values, and the initial policy gradient can be adjusted based on the advantage function to obtain the adjusted policy gradient.

[0121] In some embodiments, an advantage function reflecting the relative advantage of the data can be constructed based on the difference between the standardized reward value and the average reward value.

[0122] For example, continuing with the above symbolic sample, if the standardized reward value is 0.875 and the standardized value corresponding to the average reward value within the group is 0, then the dominance function is: standardized reward value - average standardized reward value, that is, 0.875 - 0 = 0.875.

[0123] It should be noted that for a detailed description of the calculation of the advantage function and the policy gradient adjustment, please refer to the relevant descriptions of the advantage function and policy gradient adjustment in S105 above. To avoid repetition, they will not be repeated here.

[0124] In some embodiments, the initial policy gradient can be calculated based on the advantage function, and the initial policy gradient can be constrained to serve as the adjusted policy gradient, so as to ensure that the policy update magnitude is within a preset reasonable range.

[0125] For example, continuing with the above-mentioned emblem sample, the dominant function value A i =0.875, the model's log probability of recognizing the action for this sample is logπ = -0.15, then the initial policy gradient is: logπ × A i That is, -0.15 × 0.875 = -0.13125.

[0126] At this point, if the set policy update magnitude constraint range is [-0.2, 0.2], since the initial policy gradient -0.13125 is within the constraint range, it can be directly used as the adjusted policy gradient.

[0127] Thus, by grouping by complexity, standardizing reward values, calculating the advantage function, and constraining the policy gradient, this application obtains an adjusted policy gradient that can accurately guide the optimization direction of the model while ensuring training stability. In particular, for the characteristics of low-resource dialect samples, it prioritizes the optimization of samples with relatively high advantages, gradually improving the model's ability to recognize samples of different difficulties, and ultimately improving the performance of low-resource dialect recognition.

[0128] The following describes the course reinforcement learning device for low-resource dialect recognition provided by the present invention. The course reinforcement learning device for low-resource dialect recognition described below can be referred to in correspondence with the course reinforcement learning method for low-resource dialect recognition described above.

[0129] Figure 2 This is a structural diagram of a low-resource dialect recognition curriculum reinforcement learning device provided in an embodiment of this application. The low-resource dialect recognition curriculum reinforcement learning device 200 includes: an adjustment module 201, a generation module 202, a construction module 203, and a training module 204.

[0130] The system comprises: an adjustment module 201, used to perform supervised fine-tuning of the initial speech recognition model using a low-resource dialect speech dataset to obtain a preliminary optimized model; the low-resource dialect speech dataset refers to a set of dialect data with an annotated speech data volume not exceeding a preset threshold; a generation module 202, used to recognize the speech data based on the preliminary optimized model to obtain speech recognition results, and to calculate the complexity index of each speech data based on the speech recognition results and the annotated reference answers, the complexity index being used to reflect the recognition difficulty of the speech data; a construction module 203, used to sort the speech data from low to high recognition difficulty according to the complexity index, forming a training data sequence for course learning; and a training module 204, used to apply reinforcement learning on the training data sequence, using a dynamic reward function and a group-relative strategy to iteratively update the preliminary optimized model until a preset convergence condition is met, thus obtaining the target dialect recognition model.

[0131] In some embodiments, the initial speech recognition model includes a speech encoding module, a feature adaptation module, and a text decoding module; wherein, the speech encoding module is used to extract feature vectors of dialect speech, the feature adaptation module is used to map the feature vectors to the text semantic space, and the text decoding module is used to convert the semantically aligned features into text recognition results.

[0132] In some embodiments, the adjustment module 201 is specifically used to: extract features from the speech data through the speech encoding module to obtain a speech feature vector adapted to the speech characteristics of the dialect; map the speech feature vector to the text semantic space through the feature adaptation module to obtain semantic alignment features; and fine-tune the text decoding module based on the semantic alignment features and the labeled reference answer to obtain a preliminary optimized model.

[0133] In some embodiments, the training module 204 is specifically used to: calculate a basic reward value reflecting the overall recognition accuracy for each speech data based on the word error rate of the basic speech recognition result and the labeled reference answer; calculate an adjusted reward value reflecting the local error distribution based on the number of error markers in the speech recognition result; and generate a dynamic reward value for each speech data by dynamically weighting and combining the basic reward value and the adjusted reward value.

[0134] In some embodiments, the training module 204 is specifically used for: employing group-based relative policy optimization, grouping speech data to calculate intra-group reward values, and adjusting the policy gradient corresponding to each speech data in combination with the dynamic reward values ​​to obtain the adjusted policy gradient; using the adjusted policy gradient to iteratively update the parameters of the preliminary optimized model; and determining that the preset convergence condition is met when the performance fluctuation after the model parameter update is less than a preset threshold, thereby obtaining the target dialect recognition model.

[0135] In some embodiments, the training module 204 is specifically used to: divide the speech data into multiple data groups according to the complexity index; calculate the average reward value and reward dispersion of the dynamic reward value within each data group to obtain the reward value within the group.

[0136] In some embodiments, the training module 204 is specifically used to: standardize the dynamic reward value of each speech data based on the average reward and the reward dispersion to obtain a standardized reward value; calculate the advantage function using the standardized reward value, and adjust the initial policy gradient based on the advantage function to obtain the adjusted policy gradient.

[0137] In some embodiments, the training module 204 is specifically used to: construct an advantage function that reflects the relative advantage of the data based on the difference between the standardized reward value and the average reward value; calculate the initial policy gradient based on the advantage function, and perform constraint processing on the initial policy gradient as the adjusted policy gradient to ensure that the policy update magnitude is within a preset reasonable range.

[0138] In some embodiments, the generation module 204 is further configured to: calculate the word error rate of each speech data; and count the number of erroneous markers in the speech recognition results that are inconsistent with the annotated reference answer.

[0139] In some embodiments, the generation module 204 is specifically used to: normalize the word error rate and the number of error tags respectively to obtain the standardized word error rate and the number of standardized error tags; and perform a weighted summation of the standardized word error rate and the number of standardized error tags according to a preset weight ratio to generate a complexity index.

[0140] In some embodiments, the above-mentioned construction module 204 is specifically used to: divide the complexity index into multiple consecutive numerical intervals, each numerical interval corresponding to a difficulty level; and combine the speech data in order of difficulty level from low to high to form a training data set from low complexity to high complexity.

[0141] In the low-resource dialect recognition course reinforcement learning device provided in this application embodiment, the basic ability of dialect recognition is established under low-resource conditions through supervised fine-tuning. Subsequently, by quantifying the recognition difficulty of speech samples and constructing a course learning sequence from easy to difficult, the utilization efficiency of limited training data is improved. Then, a reinforcement learning mechanism that integrates dynamic rewards and group-to-group policy optimization is introduced on this ordered learning path. This not only accurately guides the model optimization direction through fine-grained reward signals, but also effectively maintains training stability through inter-group comparison strategies. This overcomes the training oscillation or overfitting problems that are prone to occur in low-resource scenarios in traditional methods, thereby achieving a steady improvement in the model's recognition performance under strictly limited dialect data conditions.

[0142] Figure 3 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 3 As shown, the electronic device 300 may include: a processor 310, a communication interface 320, a memory 320, and a communication bus 330, wherein the processor 310, the communication interface 320, and the memory 320 communicate with each other through the communication bus 330. The processor 310 can call logic instructions in the memory 320 to execute a course reinforcement learning method for low-resource dialect recognition. This method includes: supervising the fine-tuning of an initial speech recognition model using a low-resource dialect speech dataset to obtain a preliminary optimized model; the low-resource dialect speech dataset refers to a set of dialect data with labeled speech data not exceeding a preset threshold; recognizing the speech data based on the preliminary optimized model to obtain speech recognition results, and calculating a complexity index for each speech data point based on the speech recognition results and labeled reference answers, the complexity index reflecting the recognition difficulty of the speech data; sorting the speech data from low to high recognition difficulty according to the complexity index to form a training data sequence for course learning; applying reinforcement learning to the training data sequence, iteratively updating the preliminary optimized model through a dynamic reward function and group-relative strategy until a preset convergence condition is met to obtain the target dialect recognition model.

[0143] Furthermore, the logical instructions in the aforementioned memory 320 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, essentially, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0144] On the other hand, the present invention also provides a computer program product, which includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the course reinforcement learning method for low-resource dialect recognition provided by the above methods. The method includes: supervising the fine-tuning of an initial speech recognition model using a low-resource dialect speech dataset to obtain a preliminary optimized model. The low-resource dialect speech dataset refers to a set of dialect data with an annotated speech data amount not exceeding a preset threshold; recognizing the speech data based on the preliminary optimized model to obtain speech recognition results, and calculating the complexity index of each speech data according to the speech recognition results and the annotated reference answers. The complexity index is used to reflect the recognition difficulty of the speech data; sorting the speech data from low to high recognition difficulty according to the complexity index to form a training data sequence for course learning; applying reinforcement learning on the training data sequence, and iteratively updating the preliminary optimized model through a dynamic reward function and a group relative strategy until a preset convergence condition is met to obtain a target dialect recognition model.

[0145] In another aspect, the present invention also provides a non-transitory computer-readable storage medium storing a computer program thereon. When executed by a processor, the computer program implements a course reinforcement learning method for low-resource dialect recognition provided by the methods described above. This method includes: supervising the fine-tuning of an initial speech recognition model using a low-resource dialect speech dataset to obtain a preliminary optimized model. The low-resource dialect speech dataset refers to a set of dialect data with an annotated speech data volume not exceeding a preset threshold. Based on the preliminary optimized model, the speech data is recognized to obtain speech recognition results. According to the speech recognition results and the annotated reference answers, a complexity index is calculated for each speech data. The complexity index is used to reflect the recognition difficulty of the speech data. According to the complexity index, the speech data is sorted from low to high recognition difficulty to form a training data sequence for course learning. Reinforcement learning is applied to the training data sequence, and the preliminary optimized model is iteratively updated through a dynamic reward function and a group-relative strategy until a preset convergence condition is met to obtain a target dialect recognition model.

[0146] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0147] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0148] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A curriculum reinforcement learning method for low-resource dialect recognition, characterized in that, The method includes: The initial speech recognition model is fine-tuned in a supervised manner using a low-resource dialect speech dataset to obtain a preliminary optimized model. The low-resource dialect speech dataset refers to a set of dialect data with an annotated speech data volume not exceeding a preset threshold. Based on the preliminary optimization model, the speech data is recognized to obtain the speech recognition result. Based on the speech recognition result and the labeled reference answer, the complexity index of each speech data is calculated. The complexity index is used to reflect the difficulty of speech data recognition. Based on the complexity index, the speech data are sorted from low to high recognition difficulty to form a training data sequence for course learning. Reinforcement learning is applied to the training data sequence, and the preliminary optimization model is iteratively updated through dynamic reward function and group relative policy optimization until the preset convergence condition is met, thus obtaining the target dialect recognition model.

2. The method according to claim 1, characterized in that, The initial speech recognition model includes a speech encoding module, a feature adaptation module, and a text decoding module; The speech encoding module is used to extract feature vectors of dialect speech, the feature adaptation module is used to map the feature vectors to the text semantic space, and the text decoding module is used to convert the semantic alignment features into text recognition results.

3. The method according to claim 2, characterized in that, The process involves supervised fine-tuning of the initial speech recognition model using a low-resource dialect speech dataset to obtain a preliminary optimized model, including: The speech coding module extracts features from the speech data to obtain a speech feature vector that is suitable for the characteristics of the dialect speech. The feature adaptation module maps the speech feature vector to the text semantic space to obtain semantic alignment features. Based on the semantic alignment features and the labeled reference answers, the text decoding module is fine-tuned to obtain the preliminary optimized model.

4. The method according to claim 1, characterized in that, The process of generating the dynamic reward value of the dynamic reward function includes: For each voice data point, a basic reward value reflecting the overall recognition accuracy is calculated based on the word error rate between the voice recognition result and the labeled reference answer. Based on the number of error markers in the speech recognition results, an adjustment reward value reflecting the local error distribution is calculated; A dynamic reward value for each voice data point is generated by dynamically weighting and combining the base reward value and the adjusted reward value.

5. The method according to claim 4, characterized in that, The step involves applying reinforcement learning to the training data sequence, iteratively updating the initial optimized model through a dynamic reward function and a group-relative policy until a preset convergence condition is met, to obtain the target dialect recognition model, including: The group-relative strategy optimization is adopted to group the speech data, calculate the reward value within the group, and adjust the policy gradient corresponding to each speech data in combination with the dynamic reward value to obtain the adjusted policy gradient. The parameters of the initial optimization model are iteratively updated using the adjusted policy gradient. If the performance fluctuation after the model parameter update is less than a preset threshold, the preset convergence condition is satisfied, and the target dialect recognition model is obtained.

6. The method according to claim 5, characterized in that, The optimization using the group-relative strategy, which involves grouping speech data and calculating intra-group reward values, includes: Based on the complexity index, the voice data is divided into multiple data groups; Calculate the average reward value and reward dispersion within each data group to obtain the reward value within the group.

7. The method according to claim 6, characterized in that, The step of adjusting the policy gradient corresponding to each voice data point in conjunction with the dynamic reward value to obtain the adjusted policy gradient includes: The dynamic reward value of each voice data is standardized based on the average reward and the reward dispersion to obtain a standardized reward value. The advantage function is calculated using the standardized reward value, and the initial policy gradient is adjusted based on the advantage function to obtain the adjusted policy gradient.

8. The method according to claim 4, characterized in that, The step of calculating the advantage function using the standardized reward value and adjusting the initial policy gradient based on the advantage function to obtain the adjusted policy gradient includes: Based on the difference between the standardized reward value and the average reward value, a dominance function reflecting the relative advantage of the data is constructed. The initial policy gradient is calculated based on the advantage function, and the initial policy gradient is constrained to serve as the adjusted policy gradient, so as to ensure that the policy update magnitude is within a preset reasonable range.

9. The method according to claim 1, characterized in that, Before calculating the complexity index of each voice data based on the speech recognition results and the labeled reference answers, the method further includes: For each piece of speech data, calculate the word error rate of the speech data; The number of erroneous markers in the speech recognition results that are inconsistent with the annotated reference answer is counted.

10. The method according to claim 9, characterized in that, The step of calculating the complexity index of each voice data based on the speech recognition results and the labeled reference answers includes: The word error rate and the number of error tags are normalized to obtain the standardized word error rate and the standardized number of error tags. The complexity index is generated by weighting and summing the standardized character error rate and the number of standardized error tags according to a preset weight ratio.

11. The method according to claim 1, characterized in that, The step of sorting the speech data from low to high recognition difficulty according to the complexity index to form a training data sequence for course learning includes: The complexity index is divided into multiple consecutive numerical intervals, and each numerical interval corresponds to a difficulty level; The speech data are combined in order of increasing difficulty to form the training data set from low complexity to high complexity.

12. A curriculum reinforcement learning device for low-resource dialect recognition, characterized in that, The device includes: The adjustment module is used to perform supervised fine-tuning of the initial speech recognition model using a low-resource dialect speech dataset to obtain a preliminary optimized model. The low-resource dialect speech dataset refers to a set of dialect data with an annotated speech data volume not exceeding a preset threshold. The generation module is used to recognize speech data based on the preliminary optimization model, obtain speech recognition results, and calculate the complexity index of each speech data according to the speech recognition results and the labeled reference answer. The complexity index is used to reflect the recognition difficulty of the speech data. The module is used to sort the speech data from low to high recognition difficulty according to the complexity index, so as to form a training data sequence for course learning; The training module is used to apply reinforcement learning to the training data sequence, and to iteratively update the preliminary optimization model through a dynamic reward function and a group-relative policy until a preset convergence condition is met, thereby obtaining the target dialect recognition model.

13. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that, When the processor executes the computer program, it implements the curriculum reinforcement learning method for low-resource dialect recognition as described in any one of claims 1 to 11.

14. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the curriculum reinforcement learning method for low-resource dialect recognition as described in any one of claims 1 to 11.

15. A computer program product comprising a computer program that, when executed by a processor, implements the curriculum reinforcement learning method for low-resource dialect recognition as described in any one of claims 1 to 11.

Images