Voice data dynamic screening method and device, equipment and storage medium

Abstract

The invention discloses a voice data dynamic screening method, device and equipment and a storage medium, and belongs to the technical field of voice data processing, and the method comprises the steps: obtaining a candidate voice data set which comprises a plurality of voice data; screening the voice data in the candidate voice data set based on a strategy network, and storing the reserved voice data in a training set in the screening process; a near-end strategy optimization algorithm is adopted, and the parameters of the strategy network are updated by taking the maximum accumulated reward as a target; wherein the accumulated reward is determined based on the instant reward and the delayed reward, the delayed reward is determined based on the performance index of the downstream model, and the downstream model is trained by adopting a training set. The method can adapt to different task requirements, performs dynamic screening on the voice data, and has relatively good generalization.

Description

Technical Field

[0001] The present disclosure relates to the technical field of voice data processing, and in particular to a method, apparatus, device and storage medium for dynamic voice data screening. Background Art

[0002] In the field of artificial intelligence speech processing, such as Automatic Speech Recognition (ASR) models and Text-To-Speech (TTS) models, the performance of such models is highly dependent on high-quality speech datasets.

[0003] Commonly used voice data screening methods in related technologies include: heuristic rule screening method, which statically screens voice data based on fixed screening rules (such as signal-to-noise ratio threshold, duration threshold).

[0004] However, the rules of the heuristic rule screening method are fixed and difficult to adapt to different task requirements. Different screening rules need to be set when facing different tasks, and the generalization is poor. Summary of the Invention

[0005] The present disclosure provides a method, apparatus, device, and storage medium for dynamic voice data screening, which can adapt to different task requirements and dynamically screen voice data, and has good generalization. The technical solution includes at least the following solutions: In a first aspect, a method for dynamic screening of speech data is provided, comprising: obtaining a candidate speech data set, the candidate speech data set comprising a plurality of speech data; screening the speech data in the candidate speech data set based on a policy network, and storing the retained speech data in a training set during the screening process; and using a proximal policy optimization algorithm to update the parameters of the policy network with the goal of maximizing the cumulative reward; wherein the cumulative reward is determined based on an immediate reward and a delayed reward, the delayed reward is determined based on a performance indicator of a downstream model, and the downstream model is trained using the training set.

[0006] Optionally, the method further includes: preprocessing the speech data in the candidate speech data set, wherein the preprocessing includes: unifying the sampling rate, noise suppression processing, speech data normalization processing, and speech feature extraction.

[0007] Optionally, the method further includes: constructing a state space of speech data and an action space of speech data; wherein, the state space includes a state vector of each speech data in the candidate speech data set, and the state vector of the tth iteration includes the speech features and the downstream model performance indicators in the t-1th iteration; the action space includes a retention action and a rejection action, the retention action is used to retain the speech data in the training set, and the rejection action is used to remove the speech data from the candidate speech data set.

[0008] Optionally, screening the speech data in the candidate speech data set based on the policy network includes: determining an action to be performed on the first speech data based on a probability of performing a retention action on the first speech output by the policy network.

[0009] Optionally, the immediate reward is calculated based on the speech features; the delayed reward is calculated based on the downstream model performance index of the t-1th iteration and the downstream model performance index of the tth iteration.

[0010] Optionally, when the downstream model is an automatic speech recognition ASR model and the performance indicator of the downstream model is a word error rate WER, the reward function of the delay reward is expressed by the following formula:

[0011] in, is the reward function of the delayed reward, is the downstream model performance indicator for the t-1th iteration, is the downstream model performance indicator for the tth iteration, 、 is a positive constant, Used to balance the magnitude of positive rewards, Used to balance the magnitude of negative rewards.

[0012] In the second aspect, a dynamic screening device for speech data is also provided, including: an acquisition module for acquiring a candidate speech data set, wherein the candidate speech data set includes multiple speech data; a screening module for screening the speech data in the candidate speech data set based on a policy network, and during the screening process, storing the retained speech data in a training set; a policy optimization module for adopting a proximal policy optimization algorithm to update the parameters of the policy network with the goal of maximizing the cumulative reward; wherein the cumulative reward is determined based on the immediate reward and the delayed reward, and the delayed reward is determined based on the performance indicator of the downstream model, and the downstream model is trained using the training set.

[0013] Optionally, the device further includes: a preprocessing module, which is used to preprocess the speech data in the candidate speech data set, and the preprocessing includes: unifying the sampling rate, noise suppression processing, speech data normalization processing, and speech feature extraction.

[0014] Optionally, the screening module is also used to construct a state space of speech data and an action space of speech data; wherein, the state space includes a state vector of each speech data in the candidate speech data set, and the state vector of the tth iteration includes the speech features and the downstream model performance indicators in the t-1th iteration; the action space includes retention actions and rejection actions, the retention action is used to retain the speech data in the training set, and the rejection action is used to remove the speech data from the candidate speech data set.

[0015] Optionally, the screening module is further configured to determine an action to be performed on the first speech data based on a probability of performing a reserved action on the first speech output by the policy network.

[0016] Optionally, in the screening module, the immediate reward is calculated based on the speech features; the delayed reward is calculated based on the downstream model performance index of the t-1th iteration and the downstream model performance index of the tth iteration.

[0017] Optionally, in the strategy optimization module, when the downstream model is an automatic speech recognition ASR model and the performance indicator of the downstream model is a word error rate WER, the reward function of the delay reward is expressed by the following formula:

[0018] in, is the reward function of the delayed reward, is the downstream model performance indicator for the t-1th iteration, is the downstream model performance indicator for the tth iteration, 、 is a positive constant, Used to balance the magnitude of positive rewards, Used to balance the magnitude of negative rewards.

[0019] In a third aspect, a computer device is also provided, comprising: a memory and a processor, wherein the memory stores at least one computer program, and the at least one computer program is loaded and executed by the processor, thereby executing the dynamic voice data screening method described in the above embodiment.

[0020] In a fourth aspect, a computer-readable storage medium is also provided, in which at least one computer program is stored. The at least one computer program is loaded and executed by a processor, thereby executing the method for dynamic screening of voice data described in the above embodiment.

[0021] In a fifth aspect, a computer program product is provided, comprising a computer program / instruction, which implements the method described in the first aspect when executed by a processor.

[0022] The beneficial effects of the technical solutions provided by the embodiments of the present disclosure include at least: In an embodiment of the present disclosure, a candidate speech dataset comprising multiple speech data is obtained; the speech data in the candidate speech dataset is screened using a policy network. During the screening process, the retained speech data is stored in a training set; and a proximal policy optimization algorithm is used to update the parameters of the policy network with the goal of maximizing the cumulative reward. The cumulative reward is determined based on an immediate reward and a delayed reward, and the delayed reward is determined based on the performance indicators of a downstream model, which is trained using the training set. A closed-loop feedback loop is established between data screening and model training, enabling continuous optimization of the data screening process (optimizing the parameters of the policy network) based on feedback from the downstream model (e.g., changes in performance indicators). By incorporating feedback from the downstream model into the screening process, the speech data can be dynamically screened based on the needs of the downstream model, effectively improving the accuracy of speech data screening while reducing the training cycle of the downstream model. Compared with static rules, the speech data screening method of the present disclosure also has better generalization when applied to different downstream models. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] In order to more clearly illustrate the technical solutions in the embodiments of the present disclosure, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0024] Figure 1 A flowchart of a method for dynamic voice data screening provided by an exemplary embodiment of the present disclosure is shown; Figure 2 A flowchart of a method for dynamically screening voice data provided by another exemplary embodiment of the present disclosure is shown; Figure 3 A schematic structural diagram of a device for dynamic voice data screening provided by an exemplary embodiment of the present disclosure is shown; Figure 4 It is a structural diagram of a computer device provided in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0025] Unless otherwise defined, the technical or scientific terms used herein shall have the usual meanings understood by persons of ordinary skill in the field to which the present disclosure belongs. The words “first”, “second”, “third” and similar terms used in the patent application specification and claims of the present disclosure do not indicate any order, quantity or importance, but are only used to distinguish different components. Similarly, words such as “one” or “a” do not indicate a quantity limitation, but rather indicate the presence of at least one. Words such as “include” or “comprising” and similar terms mean that the elements or objects appearing before “include” or “comprising” cover the elements or objects listed after “include” or “comprising” and their equivalents, and do not exclude other elements or objects.

[0026] In order to make the objectives, technical solutions and advantages of the present disclosure more clear, the embodiments of the present disclosure will be further described in detail below with reference to the accompanying drawings.

[0027] Figure 1 A flowchart of a method for dynamic voice data screening provided by an exemplary embodiment of the present disclosure is shown, and the method can be executed by a computer device. Figure 1 , the method comprising: In step 101, a candidate speech data set is obtained.

[0028] The candidate speech dataset includes multiple speech data sets. After screening the candidate speech dataset, the speech data retained during the screening process can be stored in the training set of downstream models, which are used to train the downstream models. This downstream model includes but is not limited to ASR, TTS, and other models that require speech data for training.

[0029] Before executing step 101, the speech data in the candidate speech dataset must first be preprocessed. The preprocessing includes: unifying the sampling rate, noise suppression, speech data normalization, and multi-dimensional speech feature extraction.

[0030] Unifying the sampling rate, for example, to 16kHz, ensures that all speech data in the candidate speech dataset are in the same format. Noise suppression primarily removes background noise from the speech data. Speech data normalization involves removing overly long or short speech segments and normalizing the volume, ensuring that the length and volume of each speech data in the candidate speech dataset are more uniform. Multidimensional speech feature extraction extracts speech features from speech data across multiple dimensions. Speech features include, but are not limited to, signal-to-noise ratio (SNR), volume energy, Mel-spectrogram (Mel-spectrogram), MFCC (Mel-frequency cepstral coefficient), duration, silence ratio, pitch distribution, phoneme entropy, speech rate, and diversity features (such as timbre diversity entropy, timbre embedding vector, and emotion label distribution entropy).

[0031] In step 102, the speech data in the candidate speech data set is screened based on the policy network.

[0032] During the screening process, the retained speech data are stored in the training set.

[0033] The essence of step 102 is to use reinforcement learning to screen the speech data in the candidate speech dataset. Reinforcement learning tasks are typically described using a Markov decision process (MDP). Therefore, before executing step 102, an MDP for the speech data must be constructed.

[0034] Typically, an MDP consists of a state space, an action space, and a reward function.

[0035] The state space includes all possible states. The state in the embodiment of the present disclosure is represented by the state vector of the speech data, and the state space includes the state vector of each speech data in the candidate speech data set. Exemplarily, the state vector of the t-th iteration includes the speech features and the downstream model performance index in the t-1-th iteration. t is a positive integer. When t is 1, the downstream model performance index in the t-1-th iteration does not exist, that is, the state vector of the 1st iteration only includes the speech features.

[0036] Depending on different downstream models, the speech features in the state vector and the downstream model performance indicators in the t-1th iteration may be different.

[0037] For example, when the downstream model is an ASR model, the speech features in the state vector of the t-th iteration include NR, MFCC dynamic range, phoneme entropy, and diversity indicators of the past 100 samples. The downstream model performance indicators in the t-1-th iteration in the state vector of the t-th iteration include the change in WER (Word Error Rate).

[0038] When the downstream model is a TTS model, the speech features in the state vector of the t-th iteration include the fundamental frequency dynamic range (for example, F0 standard deviation), MCD (Mel cepstral distortion), timbre embedding vector (which can be extracted using ECAPA-TDNN), sentiment label distribution entropy, sentence length, punctuation density, proportion of rare words in the speech data, and timbre diversity entropy of the past 50 samples. The downstream model performance indicators in the t-1th iteration in the state vector of the t-th iteration include the change in MOS (Mean opinion score) and the MCD trend.

[0039] During implementation, the state vector can be designed based on the characteristics of the downstream model. For example, the ASR model focuses on the language content (pronunciation and semantics) in the speech and needs to distinguish different phonemes or words; the TTS model focuses on how to generate natural and expressive speech, including pitch, rhythm, emotion, etc.

[0040] The action space includes all possible actions. In the disclosed embodiment, the possible actions for speech data in the candidate speech dataset include retaining and rejecting. Retaining the first speech data in the training set is to retain the first speech data, while rejecting the first speech data is to remove the first speech data from the candidate speech dataset. The first speech data is any speech data in the candidate speech dataset.

[0041] Considering that directly retaining or rejecting data with borderline quality (e.g., data with a WER close to a threshold) may lead to misjudgment, in this case, to reduce the possibility of misjudgment and further improve speech data utilization, a buffering action is introduced in the action space. Buffering the first speech data means storing the first speech data in a buffered dataset.

[0042] The speech data in the buffered data set can be processed in the following manner: when the amount of speech data in the buffered data set is greater than the buffer threshold, or after the parameters of the policy network are updated, the speech data in the buffered data set is processed using the policy network.

[0043] In the disclosed embodiment, the policy network is used to map speech data from the state space to the action space. That is, the policy network is used to output the execution probability of each action of the speech data in the action space based on the state vector of the speech data in the state space.

[0044] In one possible implementation, a probability threshold can be set. If the probability of the policy network outputting the first speech data performing the retain action is greater than the probability threshold, the retain action is performed on the first speech data. In other words, the action to be performed on the first speech data is determined solely based on the probability of the policy network outputting the first speech data performing the retain action.

[0045] In another possible implementation, either a probability threshold or a speech feature threshold can be considered, and speech data that meets the probability threshold or the speech feature threshold can be retained. That is, the action to be performed on the first speech data can be determined based on both the probability of the first speech data being retained output by the policy network and whether the first speech data meets the speech feature threshold.

[0046] The reward function is the feedback given by the environment to the model during reinforcement learning. The reward function can be based on the model's positive and negative rewards. The reward function can indicate the strengths and weaknesses of the policy network. During reinforcement learning, the policy network parameters are usually optimized with the goal of maximizing the cumulative reward.

[0047] Optionally, the reward function includes both an immediate reward and a delayed reward. The immediate reward is primarily based on the speech feature values ​​of a single piece of speech data and guides the policy to prioritize speech data of objectively higher quality when making decisions. This reward is derived directly from the speech sample itself, independent of downstream model training. The delayed reward provides feedback and evaluation of the degree to which the selected data improves the actual training results of the downstream model, guiding the long-term optimization of the policy network.

[0048] Optionally, the immediate reward is calculated based on speech features; the delayed reward is calculated based on the downstream model performance index of the t-1th iteration and the downstream model performance index of the tth iteration.

[0049] For example, when the downstream model is an ASR model, the immediate reward can be designed based on SNR, duration range, and silence ratio. Reward +1, The reward is -1. If the audio duration is within the set range, the reward is +0.5 (for example, if the audio duration is between 2 and 15 seconds, the reward is +0.5). If the silence ratio is less than the set threshold, the reward is +0.5 (for example, if the silence ratio is less than 30%, the reward is +0.5).

[0050] The values ​​in the above examples, such as 20dB, 10dB, reward +1, reward -1, etc., can be set as needed. They are only examples and are not limiting.

[0051] When the downstream model is a TTS model, rewards can also be set in a similar way, that is, some speech feature thresholds that meet the requirements are preset. When the speech data meets the corresponding speech feature thresholds, positive rewards are given. When the speech data does not meet the speech feature thresholds, no rewards are given or negative rewards are given.

[0052] Optionally, when the downstream model is an ASR model, the reward function of the delayed reward is expressed by formula (1).

[0053] (1) In formula (1), is the reward function for delayed reward, is the downstream model performance indicator for the t-1th iteration, is the downstream model performance indicator for the tth iteration, 、 is a positive constant, Used to balance the magnitude of positive rewards, Used to balance the magnitude of negative rewards.

[0054] The essence of formula (1) is that after the tth iteration, when the downstream model is the ASR model, the performance index of the ASR model is WER. The lower the WER, the better the performance of the downstream model. A positive value indicates that the performance of the downstream model has improved, and a positive reward is given. Similarly, a higher WER indicates that the performance of the downstream model is worse, that is, A positive value indicates that the performance of the downstream model has degraded, and a negative reward is given.

[0055] When the downstream model is a TTS model, the performance indicator of the TTS model is MOS. The higher the MOS, the better the performance of the downstream model. In this case, the reward function of the delayed reward is expressed using formula (2).

[0056] (2) The meanings of the parameters in formula (2) are the same as those in formula (1), and their detailed description is omitted here.

[0057] The essence of formula (2) is that when the performance indicator of the TTS model is MOS, the higher the MOS, the better the performance of the downstream model. A positive value indicates that the performance of the downstream model has improved, and a positive reward is given; A positive value indicates that the performance of the downstream model has degraded, and a negative reward is given.

[0058] The delayed reward function design enables the reinforcement learning model to dynamically adjust the screening strategy based on downstream task feedback, encouraging the selection of high-quality data that can effectively improve model performance, while inhibiting the selection of data that leads to performance degradation.

[0059] After constructing the MDP process of speech data, each speech data in the candidate speech data set can be mapped to the state space (that is, the state vector of each speech data is calculated), and then the policy network is used to make a decision for each state vector in the state space to determine whether the speech corresponding to each state vector needs to be retained in the training set, so that the training set can be obtained.

[0060] Assuming the tth round of training is the current round, after obtaining the training set, it is necessary to use the training set to train the downstream model so that the delayed reward can be calculated using formula (1) or formula (2). The immediate reward can be calculated directly from the speech features of the candidate speech dataset, and the details are omitted here.

[0061] Finally, the immediate reward and delayed reward of the tth round of training can be obtained, and then the cumulative reward of the tth round of training can be calculated.

[0062] Optionally, the method further includes: using the attention mechanism to adaptively judge the quality of the voice data, performing weighted screening on the voice data, and further improving the accuracy of voice data screening. In step 103, a proximal policy optimization algorithm is used to update the parameters of the policy network with the goal of maximizing the cumulative reward.

[0063] Among them, the cumulative reward is determined based on the immediate reward and the delayed reward. The delayed reward is determined based on the performance indicators of the downstream model, and the downstream model is trained using the training set.

[0064] There are many implementation methods of the proximal policy optimization algorithm (PPO) in related technologies, so detailed description is omitted here.

[0065] By looping through steps 102 and 103, the policy network parameters can be dynamically updated based on the performance of the downstream model, and the updating of the policy network parameters can be paused until a termination condition is reached. The termination condition includes convergence of the downstream model performance or reaching a preset number of iterations.

[0066] The method in the embodiment of the present disclosure can be used in a scenario where the downstream model is continuously learning. In this scenario, new speech data will continue to be added to the candidate speech data set. In this case, in addition to screening the candidate speech data set in the manner of steps 101 to 103, after the policy network reaches the termination condition, a cycle time can also be set, and the speech data in the training set is used to train a round of downstream models after each cycle time. If the change in the performance index of the downstream model in two adjacent rounds of training indicates that the degree of performance change of the downstream model meets the performance change threshold (for example, compared with the previous round of training, the WER of the downstream model after this round of training decreases by less than 1%), it indicates that the quality of the speech data screened by the policy network is poor, and the parameters of the policy network need to be updated. At this time, steps 102 to 103 can be re-executed to update the parameters of the policy network.

[0067] In the disclosed embodiment, steps 101 through 103 establish a closed-loop feedback loop between data screening and model training, enabling continuous optimization of the data screening process (and thus the parameters of the policy network) based on feedback from downstream models (e.g., changes in performance metrics). By incorporating feedback from downstream models into the screening process, voice data can be dynamically screened based on the needs of the downstream models, effectively improving the accuracy of voice data screening while reducing the training cycle of the downstream models. Compared to static rules, the voice data screening method of the disclosed embodiment also exhibits superior generalization across different downstream models.

[0068] Figure 2 A flowchart of a method for dynamically filtering voice data provided by another exemplary embodiment of the present disclosure is shown. The method can be executed by a computer device. In this embodiment, the method for dynamically filtering voice data in the embodiment of the present disclosure is described by taking the downstream model as an ASR model as an example.

[0069] In step 201, a candidate speech data set is obtained.

[0070] In this embodiment, the speech data screened out from the candidate speech data set can be retained in the training set, which is used to train the ASR model.

[0071] In step 202, the speech data in the candidate speech data set is preprocessed.

[0072] When the candidate speech dataset is used to train the ASR model, preprocessing includes: unifying the speech sampling rate to 16kHz and performing mono conversion; frame processing (frame length 25ms, frame shift 10ms); and normalizing the volume of the speech data to -23dBFS.

[0073] In step 203, speech features are extracted from the speech data in the candidate speech dataset.

[0074] When the downstream model is an ASR model, speech features include: signal-to-noise ratio, silence ratio, perceptual evaluation of speech quality (PESQ), phoneme entropy (calculated based on the mean and variance of MFCC), speaker name of the speech data, and recording device type.

[0075] In step 204, a state space and an action space are constructed.

[0076] In step 205, a policy network is used to map the speech data from the state space to the action space, so as to screen the speech data in the candidate speech data set to obtain a training set.

[0077] The relevant contents of step 204 to step 205 refer to the aforementioned step 102, and detailed description is omitted here.

[0078] For example, when the downstream model is an ASR model, when using a policy network to screen speech data, the probability threshold can be set to 0.7, and the speech feature threshold can be a phoneme entropy threshold, which can be set to 110% of the historical average of the phoneme entropy value. In other words, if the probability of retaining a particular speech data item is greater than 0.7 or the phoneme entropy value is higher than the historical average by 10%, the speech data item is retained.

[0079] If any of the above conditions is not met, for example, if the probability of retaining a certain voice data is less than or equal to 0.7 and the phoneme entropy value is less than the phoneme entropy threshold, a rejection action is performed on the voice data.

[0080] In step 206, a proximal policy optimization algorithm is used to update the parameters of the policy network with the goal of maximizing the cumulative reward.

[0081] When implementing the update of the policy network parameters, a phased update approach can be adopted. In the first phase, steps 202 to 206 are executed in a loop, and the policy network parameters are continuously optimized with the goal of maximizing the cumulative reward. The first phase ends when the termination condition is reached. Then, in the second phase, a cycle time is set (e.g., 100 hours). Each time the current policy network is used to screen for a cycle time, the downstream model is trained using the data in the training set. If the change in the performance indicators of the downstream model in two adjacent rounds of training indicates that the degree of performance change of the downstream model meets the performance change threshold (for example, compared with the previous round of training, the WER of the downstream model after the current round of training decreases by less than 1%), the update of the policy network parameters is triggered (that is, returning to the first phase). In this way, the policy network parameters can be continuously optimized to improve the accuracy of speech data screening.

[0082] When the downstream model is a TTS model, the process of steps 201 to 206 remains unchanged. It is only necessary to modify some parameter settings and replace the ASR in steps 201 to 206 with TTS.

[0083] Here, the parameter settings that need to be modified include: during preprocessing, the voice sampling rate needs to be unified to 24kHz and converted to mono; the volume needs to be normalized to -3dBFS, and frame processing needs to be performed (frame length 50ms, frame shift 12.5ms).

[0084] When the downstream model is a TTS model, the speech features include: F0 standard deviation, MCD, timbre embedding vector, sentiment label distribution entropy, sentence length, punctuation density, and rare word ratio.

[0085] When the downstream model is a TTS model, when using a policy network to screen speech data, the probability threshold can be set to 0.75, and the speech feature threshold can be the cosine similarity threshold of the timbre embedding vector, which can be set to 0.7. In other words, if the probability of retaining a particular speech data item is greater than 0.75 or the cosine similarity between the timbre embedding vector of the speech data item and the average timbre embedding vector of the speech data in the training set is less than 0.7, the speech data item is retained.

[0086] If any of the above conditions is not met, for example, if the probability threshold of a certain speech data is less than or equal to 0.75, and the cosine similarity between the timbre embedding vector of the speech data and the average value of the timbre embedding vector of the speech data in the training set is greater than or equal to 0.7, the speech data is rejected.

[0087] When the downstream model is a TTS model, the cycle length can be set to 50, and the performance change threshold can be determined based on MOS. If the MOS drops by more than 0.3, it means that the performance change meets the performance change threshold and the parameters of the policy network need to be updated.

[0088] The following are device embodiments of the present application. For details not described in detail in the device embodiments, reference may be made to the above method embodiments.

[0089] Figure 3 FIG2 shows a schematic diagram of the structure of a dynamic voice data screening device provided by an exemplary embodiment of the present disclosure. Figure 3 The voice data dynamic screening device 300 includes: an acquisition module 301, a screening module 302, and a strategy optimization module 303.

[0090] The acquisition module 301 is used to acquire a candidate speech data set, where the candidate speech data set includes a plurality of speech data.

[0091] The screening module 302 is used to screen the speech data in the candidate speech data set based on the policy network. During the screening process, the retained speech data is stored in the training set.

[0092] The policy optimization module 303 is configured to use a proximal policy optimization algorithm to update the parameters of the policy network with the goal of maximizing the cumulative reward. The cumulative reward is determined based on the immediate reward and the delayed reward, and the delayed reward is determined based on the performance indicator of the downstream model, and the downstream model is trained using the training set.

[0093] Optionally, the device further includes: a preprocessing module 304, which is used to preprocess the speech data in the candidate speech data set, and the preprocessing includes: unifying the sampling rate, noise suppression processing, speech data normalization processing, and speech feature extraction.

[0094] Optionally, the screening module 302 is also used to construct a state space of speech data and an action space of speech data; wherein, the state space includes a state vector of each speech data in the candidate speech data set, and the state vector of the tth iteration includes the speech features and the downstream model performance indicators in the t-1th iteration; the action space includes a retention action and a rejection action, the retention action is used to retain the speech data in the training set, and the rejection action is used to remove the speech data from the candidate speech data set.

[0095] Optionally, the screening module 302 is further configured to determine an action to be performed on the first speech data based on a probability of performing a reserved action on the first speech output by the policy network.

[0096] Optionally, in the screening module 302, the immediate reward is calculated based on the speech features; the delayed reward is calculated based on the downstream model performance index of the t-1th iteration and the downstream model performance index of the tth iteration.

[0097] Optionally, in the strategy optimization module 303, when the downstream model is an automatic speech recognition ASR model and the performance indicator of the downstream model is a word error rate WER, the reward function of the delay reward is expressed by the following formula:

[0098] in, is the reward function of the delayed reward, is the downstream model performance indicator for the t-1th iteration, is the downstream model performance indicator for the tth iteration, 、 is a positive constant, Used to balance the magnitude of positive rewards, Used to balance the magnitude of negative rewards.

[0099] It should be noted that the dynamic voice data screening device provided in the above embodiment only uses the division of the above-mentioned functional modules as an example to illustrate the dynamic voice data screening. In actual applications, the above-mentioned functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the dynamic voice data screening device provided in the above embodiment and the dynamic voice data screening method embodiment are based on the same concept. The specific implementation process is detailed in the method embodiment and will not be repeated here.

[0100] The division of modules in the embodiments of the present disclosure is illustrative and represents only a logical functional division. In actual implementation, other division methods may be used. Furthermore, the functional modules in the various embodiments of the present disclosure may be integrated into a single processor, exist physically as separate modules, or be integrated into a single module. The integrated modules may be implemented in either hardware or software functional modules.

[0101] If the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present disclosure, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a terminal device (which can be a personal computer, mobile phone, or communication device, etc.) or a processor to execute all or part of the steps of the method of each embodiment of the present disclosure. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk, and other media that can store program code.

[0102] Figure 4 Schematic diagram of the structure of the computer device provided by the embodiment of the present disclosure. Figure 4 As shown, the computer device 400 includes a processor 401 and a memory 402 .

[0103] Processor 401 may include one or more processing cores, such as a quad-core processor or an octa-core processor. Processor 401 may be implemented in hardware using at least one of the following: a DSP (Digital Signal Processing), an FPGA (Field-Programmable Gate Array), or a PLA (Programmable Logic Array). Processor 401 may also include a main processor and a coprocessor. The main processor is used to process data in the awake state, also known as a CPU (Central Processing Unit); the coprocessor is a low-power processor used to process data in the standby state. In some embodiments, processor 401 may be integrated with a GPU (Graphics Processing Unit), which is responsible for rendering and drawing content displayed on the display screen. In some embodiments, processor 401 may also include an AI (Artificial Intelligence) processor, which is used to handle computational operations related to machine learning.

[0104] The memory 402 may include one or more computer-readable storage media, which may be non-transitory. The memory 402 may also include high-speed random access memory and non-volatile memory, such as one or more disk storage devices and flash memory storage devices. In some embodiments, the non-transitory computer-readable storage medium in the memory 402 is used to store at least one instruction, which is executed by the processor 401 to implement the dynamic voice data screening method provided in the embodiments of the present disclosure.

[0105] Those skilled in the art will understand that Figure 4 The structure shown in the figure does not constitute a limitation on the computer device 400, and the computer device 400 may include more or fewer components than shown in the figure, or combine some components, or adopt a different component arrangement.

[0106] The embodiment of the present disclosure also provides a non-transitory computer-readable storage medium. When the instructions in the storage medium are executed by the processor of a computer device, the computer device can execute the method for dynamic screening of voice data provided in the embodiment of the present disclosure.

[0107] The embodiments of the present disclosure further provide a computer program product, including a computer program / instruction, which, when executed by a processor, implements the method for dynamic voice data screening provided in the embodiments of the present disclosure.

[0108] The above description is merely an optional embodiment of the present disclosure and is not intended to limit the present disclosure. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present disclosure shall be included in the scope of protection of the present disclosure.

Claims

1. A method for dynamic screening of voice data, characterized in that: The method comprises: Acquire a candidate speech data set, where the candidate speech data set includes a plurality of speech data; Screening the speech data in the candidate speech data set based on the policy network, and storing the retained speech data in a training set during the screening process; Adopting a proximal policy optimization algorithm to update the parameters of the policy network with the goal of maximizing the cumulative reward; The cumulative reward is determined based on the immediate reward and the delayed reward, the delayed reward is determined based on the performance indicator of the downstream model, and the downstream model is trained using the training set.

2. The method according to claim 1, characterized in that The method further comprises: The speech data in the candidate speech data set is preprocessed, and the preprocessing includes: unifying the sampling rate, noise suppression processing, speech data normalization processing, and speech feature extraction.

3. The method according to claim 2, characterized in that The method further comprises: Construct the state space of speech data and the action space of speech data; The state space includes the state vector of each speech data in the candidate speech data set, and the state vector of the t-th iteration includes the speech features and the downstream model performance index in the t-1-th iteration; The action space includes a retaining action and a rejecting action. The retaining action is used to retain the speech data in the training set, and the rejecting action is used to remove the speech data from the candidate speech data set.

4. The method according to claim 3, characterized in that The screening of the speech data in the candidate speech data set based on the policy network includes: An action to be performed on the first speech data is determined based on the probability of the first speech performing the reserved action output by the policy network.

5. The method according to any one of claims 1 to 4, characterized in that The instant reward is calculated based on the voice feature; The delayed reward is calculated based on the downstream model performance index of the t-1th round iteration and the downstream model performance index of the tth round iteration.

6. The method according to claim 5, characterized in that When the downstream model is an automatic speech recognition ASR model and the performance indicator of the downstream model is word error rate WER, the reward function of the delay reward is expressed by the following formula: in, is the reward function of the delayed reward, is the downstream model performance indicator for the t-1th iteration, is the downstream model performance indicator for the tth iteration, 、 is a positive constant, Used to balance the magnitude of positive rewards, Used to balance the magnitude of negative rewards.

7. A device for dynamic screening of voice data, characterized in that: The device comprises: An acquisition module is used to acquire a candidate speech data set, wherein the candidate speech data set includes a plurality of speech data; A screening module, configured to screen the speech data in the candidate speech data set based on a policy network, and store the retained speech data in a training set during the screening process; A policy optimization module, configured to use a proximal policy optimization algorithm to update the parameters of the policy network with the goal of maximizing the cumulative reward; The cumulative reward is determined based on the immediate reward and the delayed reward, the delayed reward is determined based on the performance indicator of the downstream model, and the downstream model is trained using the training set.

8. A computer device, characterized in that: The computer device includes: a memory and a processor, wherein at least one computer program is stored in the memory, and the at least one computer program is loaded and executed by the processor to implement the method according to any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that The computer-readable storage medium stores at least one computer program, and the at least one computer program is loaded and executed by a processor to implement the method according to any one of claims 1 to 6.

10. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instructions are executed by a processor, the method according to any one of claims 1 to 6 is implemented.

Images