Speech recognition model training method, speech recognition method, and device

Abstract

The application discloses a speech recognition model training method, a speech recognition method and device, wherein a current speech training sample is obtained; the current speech training sample is input into a speech recognition model for training, and a training loss value corresponding to the current speech training sample is obtained; the training loss value of the speech training sample is subjected to nonlinear mapping, and an enhancement parameter of the speech training sample is obtained, wherein the enhancement parameter comprises an operation parameter of at least one preset enhancement operation; the current speech training sample is subjected to data enhancement based on the enhancement parameter of the current speech training sample, and an enhanced speech training sample is obtained; and the speech recognition model is trained based on the enhanced speech training sample until a preset stop condition is met. The application can improve the robustness of the speech recognition model obtained through training and improve the accuracy of speech recognition.

Description

Technical Field

[0001] This application relates to the field of speech processing technology, specifically to a training method for a speech recognition model, a speech recognition method, and an apparatus. Background Technology

[0002] Automatic Speech Recognition (ASR) is a technology that converts speech signals into text. ASR has a wide range of applications, such as voice input and voice control. Data augmentation is a common technique in ASR used to expand the size and diversity of the training dataset. Data augmentation aims to generate more speech samples by performing a series of transformations on the original speech data, thereby improving the quality and quantity of the training dataset and ultimately enhancing the performance of the ASR system. Data augmentation methods include time-domain transformation, frequency-domain transformation, feature transformation, and simulating environmental phonemes such as echoes and noise. Current techniques, after calculating the sample loss value, use a linear mapping method to map the sample loss values ​​of all samples before performing sample augmentation. This does not consider the fluctuations in sample loss values ​​during training, which can cause the model to overfit to some data and underfit to others, ultimately reducing model robustness and leading to inaccurate speech recognition.

[0003] In other words, speech recognition in existing technologies is inaccurate. Summary of the Invention

[0004] This application provides a method for training a speech recognition model, a speech recognition method, and an apparatus, which can improve the accuracy of speech recognition.

[0005] Firstly, the training method for the speech recognition model provided in this application includes:

[0006] Get the current speech training sample;

[0007] The current speech training sample is input into the speech recognition model for training, and the training loss value corresponding to the current speech training sample is obtained.

[0008] The training loss value of the current speech training sample is nonlinearly mapped to obtain the enhancement parameters of the speech training sample, wherein the enhancement parameters include operation parameters of at least one preset enhancement operation.

[0009] Based on the enhancement parameters of the current speech training sample, data augmentation is performed on the current speech training sample to obtain an enhanced speech training sample;

[0010] The speech recognition model is trained based on the enhanced speech training samples until a preset stopping condition is met.

[0011] In an optional embodiment, the step of performing a non-linear mapping on the training loss value of the current speech training sample to obtain the enhancement parameters of the speech training sample includes:

[0012] The training loss value of the current speech training sample is normalized to obtain the normalized value of the current speech training sample.

[0013] The normalized value of the current speech training sample is input into a preset nonlinear mapping function to obtain the mapping parameters of the current speech training sample.

[0014] The enhancement parameters of the current speech training sample are determined based on the mapping parameters of the current speech training sample.

[0015] In an optional embodiment, normalizing the training loss value of the current speech training sample to obtain a normalized value for the current speech training sample includes:

[0016] Obtain the training loss value of historical speech training samples, wherein the training loss value of the historical speech training samples is obtained by inputting the historical speech training samples into the speech recognition model before the current speech training samples;

[0017] Obtain the maximum and minimum loss values ​​among the training loss values ​​of the historical speech training samples and the training loss values ​​of the current speech training samples;

[0018] The training loss value of the current speech training sample is normalized based on the maximum loss value and the minimum loss value to obtain the normalized value of the current speech training sample.

[0019] In an optional embodiment, determining the enhancement parameters of the current speech training sample based on the mapping parameters of the current speech training sample includes:

[0020] Obtain the first parameter mapping relationship for each preset enhanced operation;

[0021] The mapping parameters of the current speech training sample are mapped based on the first parameter mapping relationship of each preset enhancement operation to obtain the operation parameters of each preset enhancement operation for the current speech training sample, and thus the enhancement parameters of the current speech training sample are obtained.

[0022] In an optional embodiment, the step of performing data augmentation on the current speech training sample based on the augmentation parameters of the current speech training sample to obtain augmented speech training samples includes:

[0023] Obtain the probability of selecting each preset enhancement operation for the current speech training sample;

[0024] Preset augmentation operations with a selection probability greater than a preset threshold are identified as target augmentation operations, resulting in multiple target augmentation operations.

[0025] Data augmentation is performed on the current speech training sample based on each target augmentation operation to obtain augmented speech training samples.

[0026] In an optional embodiment, obtaining the selection probability of each preset enhancement operation for the current speech training sample includes:

[0027] Obtain the average first training loss of the current speech training sample in the current iteration and the average second training loss of the current speech training sample in the previous iteration.

[0028] The rate of change of loss is determined based on the average value of the first training loss and the average value of the second training loss;

[0029] The probability of selecting each preset enhancement operation for the current speech training sample is determined based on the loss change rate.

[0030] In an optional embodiment, obtaining the current speech training sample includes:

[0031] Obtain the raw audio data;

[0032] Feature extraction is performed on the raw audio data to obtain speech features;

[0033] The speech features are processed to obtain the current speech training sample.

[0034] Secondly, the speech recognition method provided in this application includes:

[0035] Obtain the speech information to be recognized;

[0036] Obtain a speech recognition model, wherein the speech recognition model is trained by the method described in any one of the first aspects;

[0037] The speech information to be recognized is input into the speech recognition model to obtain the speech recognition result.

[0038] Thirdly, the training device for the speech recognition model provided in this application includes:

[0039] The acquisition module is used to acquire the current speech training samples;

[0040] The first training module is used to input the current speech training sample into the speech recognition model for training, and obtain the training loss value corresponding to the current speech training sample.

[0041] The mapping module is used to perform nonlinear mapping on the training loss value of the current speech training sample to obtain the enhancement parameters of the speech training sample, wherein the enhancement parameters include operation parameters of at least one preset enhancement operation.

[0042] The data augmentation module is used to perform data augmentation on the current speech training sample based on the augmentation parameters of the current speech training sample to obtain an augmented speech training sample;

[0043] The second training module is used to train the speech recognition model based on the enhanced speech training samples until a preset stopping condition is met.

[0044] In an optional embodiment, the mapping module is configured to:

[0045] The training loss value of the current speech training sample is normalized to obtain the normalized value of the current speech training sample.

[0046] The normalized value of the current speech training sample is input into a preset nonlinear mapping function to obtain the mapping parameters of the current speech training sample.

[0047] The enhancement parameters of the current speech training sample are determined based on the mapping parameters of the current speech training sample.

[0048] In an optional embodiment, the mapping module is configured to:

[0049] Obtain the training loss value of historical speech training samples, wherein the training loss value of the historical speech training samples is obtained by inputting the historical speech training samples into the speech recognition model before the current speech training samples;

[0050] Obtain the maximum and minimum loss values ​​among the training loss values ​​of the historical speech training samples and the training loss values ​​of the current speech training samples;

[0051] The training loss value of the current speech training sample is normalized based on the maximum loss value and the minimum loss value to obtain the normalized value of the current speech training sample.

[0052] In an optional embodiment, the mapping module is configured to:

[0053] Obtain the first parameter mapping relationship for each preset enhanced operation;

[0054] The mapping parameters of the current speech training sample are mapped based on the first parameter mapping relationship of each preset enhancement operation to obtain the operation parameters of each preset enhancement operation for the current speech training sample, and thus the enhancement parameters of the current speech training sample are obtained.

[0055] In an optional embodiment, the data enhancement module is configured to:

[0056] Obtain the probability of selecting each preset enhancement operation for the current speech training sample;

[0057] Preset augmentation operations with a selection probability greater than a preset threshold are identified as target augmentation operations, resulting in multiple target augmentation operations.

[0058] Data augmentation is performed on the current speech training sample based on each target augmentation operation to obtain augmented speech training samples.

[0059] In an optional embodiment, the data enhancement module is configured to:

[0060] Obtain the average first training loss of the current speech training sample in the current iteration and the average second training loss of the current speech training sample in the previous iteration.

[0061] The rate of change of loss is determined based on the average value of the first training loss and the average value of the second training loss;

[0062] The probability of selecting each preset enhancement operation for the current speech training sample is determined based on the loss change rate.

[0063] In an optional embodiment, the acquisition module is configured to:

[0064] Obtain the raw audio data;

[0065] Feature extraction is performed on the raw audio data to obtain speech features;

[0066] The speech features are processed to obtain the current speech training sample.

[0067] Fourthly, the voice recognition device provided in this application includes:

[0068] The voice acquisition module is used to acquire the voice information to be recognized.

[0069] A model acquisition module is used to acquire a speech recognition model, wherein the speech recognition model is trained by the method described in any one of the first aspects;

[0070] The speech recognition module is used to input the speech information to be recognized into the speech recognition model to obtain the speech recognition result.

[0071] Fifthly, the electronic device provided in this application includes a memory and a processor. The memory stores a computer program, and the processor runs the computer program in the memory to implement the steps in the training method of the speech recognition model or the steps in the speech recognition method provided in this application.

[0072] Sixthly, the computer-readable storage medium provided in this application stores a plurality of instructions, which are adapted for a processor to load and implement the steps in the training method of the speech recognition model or the steps in the speech recognition method provided in this application.

[0073] Seventhly, the computer program product provided in this application includes a computer program or instructions that, when executed by a processor, implement the steps in the training method of the speech recognition model provided in this application or the steps in the speech recognition method.

[0074] In this application, compared with related technologies, after calculating the training loss value of the current speech training sample, the training loss value of the current speech training sample is nonlinearly mapped to obtain the enhancement parameters of the current speech training sample. Then, the current speech training sample is enhanced using the enhancement parameters of the current speech training sample and then the model is trained. Since the enhancement parameters are obtained in a nonlinear way, the fluctuation of the sample loss value during the training process can be fully considered, which can improve the robustness of the trained speech recognition model and improve the accuracy of speech recognition. Attached Figure Description

[0075] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0076] Figure 1 This is a schematic diagram of a scenario for the speech recognition system provided in an embodiment of this application;

[0077] Figure 2 This is a flowchart illustrating one embodiment of the training method for the speech recognition model provided in this application.

[0078] Figure 3 This is a flowchart illustrating another embodiment of the training method for the speech recognition model provided in this application.

[0079] Figure 4 This is a flowchart illustrating one embodiment of the speech recognition method provided in this application.

[0080] Figure 5This is a schematic diagram of the structure of the training device for the speech recognition model provided in the embodiments of this application;

[0081] Figure 6 This is a schematic diagram of the structure of the speech recognition device provided in the embodiments of this application;

[0082] Figure 7 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0083] It should be noted that the principles of this application are illustrated by example in a suitable computing environment. The following description is based on the specific embodiments of this application that are illustrated, and should not be regarded as limiting other specific embodiments not detailed herein.

[0084] In the following description of this application, "some embodiments" are referred to, which describe a subset of all possible embodiments. However, it is understood that "some embodiments" may be the same subset or different subset of all possible embodiments, and may be combined with each other without conflict.

[0085] In the following description of this application, the terms "first, second, third" are used merely to distinguish similar objects and do not represent a specific ordering of objects. It is understood that "first, second, third" may be interchanged in a specific order or sequence where permitted, so that the embodiments of this application described herein can be implemented in an order other than that illustrated or described herein.

[0086] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit this application.

[0087] To improve the performance of speech recognition, embodiments of this application provide a speech recognition model training method, a speech recognition model training device, a speech recognition method, a speech recognition device, an electronic device, a computer-readable storage medium, and a computer program product. The speech recognition model training method can be executed by the speech recognition model training device or by an electronic device integrating the speech recognition model training device; the speech recognition method can be executed by the speech recognition device or by an electronic device integrating the speech recognition device.

[0088] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0089] Please refer to Figure 1 This application also provides a speech recognition system, such as Figure 1 As shown, the speech recognition system includes an electronic device 100, which integrates a training device for the speech recognition model provided in this application and / or a speech recognition device.

[0090] Among them, electronic device 100 can be any device equipped with a processor and having processing capabilities, such as mobile electronic devices with processors such as smartphones, tablets, PDAs, laptops, and smart speakers, or fixed electronic devices with processors such as desktop computers, televisions, servers, and industrial equipment.

[0091] In addition, such as Figure 1 As shown, the speech recognition system may also include a memory 200 for storing raw data, intermediate data and result data.

[0092] In this embodiment of the application, the memory 200 can be a cloud memory. Cloud storage is a new concept that is extended and developed from the concept of cloud computing. A distributed cloud storage system (hereinafter referred to as a storage system) refers to a storage system that uses cluster applications, grid technology and distributed storage file system functions to bring together a large number of storage devices of various types in the network (storage devices are also called storage nodes) through application software or application interfaces to work together to provide data storage and business access functions to the outside world.

[0093] Currently, the storage method of storage systems is as follows: Logical volumes are created. During the creation of a logical volume, physical storage space is allocated to each logical volume. This physical storage space may consist of a single storage device or the disks of several storage devices. Clients store data on a logical volume, which means storing the data on the file system. The file system divides the data into many parts, each part being an object. Each object contains not only the data but also additional information such as a data identifier (ID, ID entity). The file system writes each object to the physical storage space of that logical volume and records the storage location information of each object. Therefore, when a client requests access to data, the file system can allow the client to access the data based on the storage location information of each object.

[0094] The process by which a storage system allocates physical storage space to a logical volume is as follows: the physical storage space is pre-divided into strips according to the capacity estimate of the objects stored in the logical volume (this estimate often has a large margin relative to the actual capacity of the objects to be stored) and the grouping of Redundant Array of Independent Disks (RAID). A logical volume can be understood as a strip, thus allocating physical storage space to the logical volume.

[0095] It should be noted that, Figure 1 The schematic diagram of the speech recognition system shown is merely an example. The speech recognition system and scenarios described in this application are intended to more clearly illustrate the technical solutions of this application and do not constitute a limitation on the technical solutions provided in this application. As those skilled in the art will know, with the evolution of speech recognition systems and the emergence of new business scenarios, the technical solutions provided in this application are also applicable to similar technical problems.

[0096] The following sections provide detailed descriptions of each example. It should be noted that the sequence numbers of the following embodiments are not intended to limit the preferred order of the embodiments.

[0097] Please refer to Figure 2 , Figure 2 This is a flowchart illustrating one embodiment of the training method for the speech recognition model provided in this application. Figure 2 As shown, the training method for the speech recognition model provided in this application follows this process:

[0098] 201. Obtain the current speech training sample.

[0099] In this embodiment of the application, obtaining the current speech training sample includes:

[0100] (1) Obtain the raw audio data.

[0101] In this embodiment, the original audio data can be audio data recorded by various recording devices. The original audio data can be crawled from the web or read from various recording devices; no limitation is made here.

[0102] (2) Extract features from the original audio data to obtain speech features.

[0103] In this embodiment, the original audio data undergoes pre-emphasis processing, frame segmentation, windowing, fast Fourier transform, and filtering using a Mel filter bank to obtain filter bank-based features. Of course, in other embodiments, the speech features can also be Mel frequency cepstrum coefficients (MFCC) features.

[0104] Specifically, the speech data is first pre-emphasized. Pre-emphasis enhances the high-frequency components of the speech signal by passing it through a high-pass filter, ensuring the same signal-to-noise ratio for spectral analysis across the entire frequency band from low to high frequencies. The purpose of pre-emphasis is to eliminate the effects of the vocal cords and lips during phonation, compensating for the suppression of high-frequency components by the vocal system, and highlighting high-frequency formants. The pre-emphasized speech data is then framed. Frame segmentation involves dividing the given speech data into segments of a fixed time length; each segment is called a frame or sub-frame, and these sub-frames are used for Fbank feature extraction. Each frame of speech data is then windowed. After framing, each frame needs to be windowed to increase the continuity between the left and right edges of the frame, thus reducing spectral leakage. The Hamming window is a commonly used window function for Fbank feature extraction. Finally, a Fast Fourier Transform (FFT) is performed on the windowed speech data. Since it's often difficult to discern signal characteristics from time-domain transformations, it's typically converted to an energy distribution in the frequency domain for observation. Different energy distributions represent different speech characteristics. Therefore, after windowing the speech data, a Fast Fourier Transform (FFT) is required to obtain the spectral energy distribution. FFTs are performed on each frame of windowed speech data to obtain the spectrum of each frame, and the power spectrum of the speech data is obtained by taking the square of the modulus of the spectrum. A Mel filter bank is used to filter the obtained energy spectrum. The Fbank feature considers human auditory characteristics, first mapping the linear spectrum to a Mel nonlinear spectrum based on auditory perception, and then converting it to the cepstrum. In the Mel frequency domain, human perception of pitch is linear. Logarithmic calculations are performed on the energy spectrum after Mel filter bank processing to obtain the Fbank feature.

[0105] (3) Process the speech features to obtain the current speech training sample.

[0106] Among them, speech features can be features based on filter banks (Fbank) or Mel frequency cepstrum coefficients (MFCC) features, etc.

[0107] In other embodiments, the current speech training sample may also be processed by other devices and then directly read as input.

[0108] 202. Input the current speech training sample into the speech recognition model for training, and obtain the training loss value corresponding to the current speech training sample.

[0109] In this embodiment, the speech recognition model can be the Wenet model, the Kaldi model, other CNN models, or RNN models, etc., depending on the specific situation, and is not limited here.

[0110] It's important to clarify that Artificial Intelligence (AI) is the theory, methods, technology, and application systems that utilize digital computers or computers-controlled machines to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to achieve optimal results. In other words, AI is a comprehensive technology within computer science that attempts to understand the essence of intelligence and produce new intelligent machines capable of reacting in a manner similar to human intelligence. AI essentially studies the design principles and implementation methods of various intelligent machines, enabling them to possess perception, reasoning, and decision-making capabilities.

[0111] Artificial intelligence (AI) is a comprehensive discipline encompassing a wide range of fields, including both hardware and software technologies. Fundamental AI technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing, operating / interactive systems, and mechatronics. AI software technologies primarily include machine learning (ML), with deep learning (DL) being a relatively new research direction within ML. It has been introduced into machine learning to bring it closer to its original goal: artificial intelligence. Currently, deep learning is mainly applied in areas such as machine vision and natural language processing.

[0112] Deep learning learns the inherent patterns and hierarchical representations of sample data. The information gained during this learning process greatly aids in the interpretation of data such as text, images, and sound. Using deep learning techniques and corresponding sample sets, it is possible to train network models that perform different functions.

[0113] In this embodiment, the current speech training sample can be input into the speech recognition model for multiple rounds of iterative training to obtain the training loss value of the current speech training sample. Each round of iteration involves the current speech training sample being input into the speech recognition model for one forward propagation and one backward propagation.

[0114] Each training iteration outputs the training loss value for the current speech training sample. For example, among N current speech training samples, the training loss value for each current speech training sample is loss_i, where i takes values ​​from 1 to N. The training loss value for each current speech training sample can be calculated using loss functions such as cross-entropy loss function and exponential loss function, which can be set according to specific circumstances.

[0115] 203. Perform a nonlinear mapping on the training loss value of the current speech training sample to obtain the enhancement parameters of the current speech training sample.

[0116] The enhancement parameters include operational parameters for at least one preset enhancement operation. The number of preset enhancement operations can be one, two, or more; this application does not limit this. The at least one preset enhancement operation includes frequency masking (freq_mask), time masking (time_mask), and spectrum substitution (spec_sub). Of course, the at least one preset enhancement operation may also include operations such as simulated echo, simulated noise, and random scaling.

[0117] Among them, the parameters of frequency masking include the number of frequency maskings, which refers to the number of times the frequency domain interval of the sample's spectrum is masked; the parameters of time masking include the number of time maskings, which refers to the number of times the time domain interval of the sample's spectrum is masked; and the parameters of spectrum replacement include the number of spectrum replacements, which refers to the number of times the time domain interval of the sample's spectrum is replaced.

[0118] Specifically, the current speech training sample, after feature processing, yields a two-dimensional vector, with one dimension being time and the other frequency. Frequency masking (freq_mask) and time masking (time_mask) mean masking the two-dimensional vector according to different dimensions, that is, setting the values ​​in the corresponding dimensions of the two-dimensional vector to 0. Spectrum replacement is performed on the time dimension after obtaining the two-dimensional spectrum vector, for example, replacing the values ​​at indices 8 to 10 with the values ​​at indices 1 to 3, that is, "replacing" the vectors at times 1 to 3 in the spectrum with the vectors at times 8 to 10. In Python, this is written as: array arr[1:3] = arr[8:10].

[0119] In one specific embodiment, performing a non-linear mapping on the training loss value of the current speech training sample to obtain the enhancement parameters of the current speech training sample may include:

[0120] (1) Normalize the training loss value of the current speech training sample to obtain the normalized value of the current speech training sample.

[0121] In one specific embodiment, the training loss value of the current speech training sample is normalized using the min-max normalization algorithm to obtain the normalized value of the current speech training sample. In other embodiments, other normalization algorithms such as linear function normalization and zero-mean normalization can also be used to normalize the training loss value of the current speech training sample to obtain the normalized value of the current speech training sample.

[0122] (2) Input the normalized value of the current speech training sample into the preset nonlinear mapping function to obtain the mapping parameters of the current speech training sample.

[0123] In one specific embodiment, the preset nonlinear mapping function is an incomplete beta function. For example, the mapping parameter for the current speech training sample is λ. Of course, in other embodiments, the preset nonlinear mapping function can also be other monotonically decreasing nonlinear functions.

[0124] (3) Determine the enhancement parameters of the current speech training sample based on the mapping parameters of the current speech training sample.

[0125] In one specific embodiment, the mapping parameters of the current speech training sample are determined as the enhancement parameters of the current speech training sample, thus obtaining the enhancement parameters of the current speech training sample. For example, the mapping parameters of the current speech training sample are λ. The number of frequency masking times, the number of time masking times, and the number of spectral substitution times are all λ.

[0126] The larger the training loss value of the current speech training sample, the more likely it is to be a difficult sample to learn from. The smaller the mapping parameter λ obtained by inputting the normalized value of the sample into the preset nonlinear mapping function, the fewer the number of data augmentations applied to that sample, making it more suitable for the model to learn the information contained in the current sample. Conversely, the smaller the training loss value of the current speech training sample, the larger the mapping parameter λ, and the more data augmentations are applied, prompting the model to learn the more complex information within this simple sample.

[0127] Furthermore, in order to utilize the information from the previous batch of samples to normalize the training loss value of the current speech training sample and obtain the normalized value of the current speech training sample, it can include:

[0128] (1) Obtain the training loss value of historical speech training samples.

[0129] The training loss value of historical speech training samples is obtained by inputting historical speech training samples into the speech recognition model before the current speech training samples.

[0130] Specifically, the raw audio data is processed to obtain multiple batches of training sample sets, each batch containing N speech training samples. These batches are then sequentially input into the speech recognition model for training, yielding the training loss value for each batch's speech training samples. The training sample set from the previous batch in two adjacent batches is designated as the historical speech training sample, and its training loss value is obtained. Finally, the training sample set from the next batch in two adjacent batches is designated as the current speech training sample, and its training loss value is obtained.

[0131] (2) Obtain the maximum and minimum loss values ​​among the training loss values ​​of historical speech training samples and the training loss values ​​of current speech training samples.

[0132] Specifically, the training loss values ​​of historical speech training samples and the training loss values ​​of current speech training samples are combined into a set of loss values. The maximum value in the set of loss values ​​is determined as the maximum loss value (max), and the minimum value in the set of loss values ​​is determined as the minimum loss value (min).

[0133] (3) Normalize the training loss value of the current speech training sample based on the maximum and minimum loss values ​​to obtain the normalized value of the current speech training sample.

[0134] In this embodiment, the maximum-minimum normalization algorithm is used to normalize the training loss value of the current speech training sample based on the maximum and minimum loss values ​​to obtain the normalized value of the current speech training sample.

[0135] Specifically, the formula for calculating the normalized value of the current speech training sample is as follows:

[0136] I-(loss_i-min) / (max-min)

[0137] Where I is the normalized value of the current speech training sample, loss_i is the training loss value of the i-th speech training sample in the set composed of the training loss values ​​of the historical speech training samples and the current speech training sample, i takes the value of 1-2N, max is the maximum loss value, and min is the minimum loss value.

[0138] Furthermore, considering the characteristics of different preset enhancement operations, different mapping relationships are set for each preset enhancement operation. The enhancement parameters of the current speech training sample are determined based on the mapping parameters of the current speech training sample, including:

[0139] (1) Obtain the first parameter mapping relationship of each preset enhanced operation.

[0140] In one specific embodiment, the preset enhancement operation is frequency masking (freq_mask), and the first parameter mapping relationship of frequency masking is: when the mapping parameter is λ, the number of frequency masking is 1+λ; the preset enhancement operation is time masking (time_mask), and the first parameter mapping relationship of time masking is: when the mapping parameter is λ, the number of time masking is 1+λ; the preset enhancement operation is spectrum replacement, and the first parameter mapping relationship of spectrum replacement is: when the mapping parameter is λ, the number of spectrum replacements is λ.

[0141] (2) Map the mapping parameters of the current speech training sample based on the first parameter mapping relationship of each preset enhancement operation to obtain the operation parameters of each preset enhancement operation of the current speech training sample, and obtain the enhancement parameters of the current speech training sample.

[0142] Specifically, by mapping according to the first parameter mapping relationship of each preset enhancement operation, the operation parameters of each preset enhancement operation of the current speech training sample are obtained, thus obtaining the enhancement parameters of the current speech training sample. The enhancement parameters of the current speech training sample can be obtained based on the same principle.

[0143] 204. Based on the augmentation parameters of the current speech training sample, perform data augmentation on the current speech training sample to obtain augmented speech training samples.

[0144] After obtaining the augmentation parameters of the current speech training sample, data augmentation is performed on the current speech training sample according to the augmentation parameters to obtain augmented speech training samples.

[0145] Furthermore, in order to perform different preset enhancement operations on different samples, in this embodiment of the application, data enhancement is performed on the current speech training sample based on the enhancement parameters of the current speech training sample to obtain enhanced speech training samples, which may include:

[0146] (1) Obtain the probability of each operation selection for each preset enhancement operation on the current speech training sample.

[0147] In one specific embodiment, the selection probability of each preset enhancement operation can be manually input. For example, the selection probability of frequency masking (freq_mask) is P1, the selection probability of time masking (time_mask) is P2, and the selection probability of spectrum substitution (spec_sub) is P3.

[0148] (2) Select the preset enhancement operation with an operation selection probability greater than the preset threshold as the target enhancement operation to obtain multiple target enhancement operations.

[0149] The preset thresholds for each preset enhancement operation can be determined in advance by human experience or generated randomly.

[0150] (3) Perform data augmentation on the current speech training sample based on each target augmentation operation to obtain augmented speech training sample.

[0151] When the rate of change of loss increases, it reflects a large change in model parameters, indicating that the model is learning and fitting the training data too quickly. In this case, the data augmentation probability needs to be reduced to avoid overfitting. When a sample undergoes each preset augmentation operation, the model determines whether to perform the corresponding operation based on the operation selection probability value. This process prevents the model from over-focusing on local features of the sample, leading to overfitting. In each round of training initialization, the model calculates the rate of change of loss. This rate of change is input into a preset nonlinear mapping function to obtain a coefficient of change. The new operation selection probability is calculated using this coefficient. A larger rate of change of loss results in a larger coefficient of change (cof), and a smaller operation selection probability, thus reducing the probability of performing data augmentation and preventing overfitting.

[0152] 205. Based on the enhanced speech training samples, train the speech recognition model until the preset stopping condition is met.

[0153] Specifically, the enhanced speech training samples are input into the speech recognition model to obtain the training loss value of the enhanced speech training samples. If the training loss value of the enhanced speech training samples is greater than the preset value, the preset stopping condition is not met. Then, the network parameters of the speech recognition model are updated and iterative training is performed using the enhanced speech training samples. When the training loss value of the enhanced speech training samples is not greater than the preset value, the preset stopping condition is met, and the speech recognition model at this time is output.

[0154] Please refer to Figure 3 , Figure 3 This is a flowchart illustrating another embodiment of the training method for the speech recognition model provided in this application, as shown below. Figure 3 As shown, the flow of the speech recognition method provided in this application is as follows:

[0155] 301. Obtain the raw audio data.

[0156] In this embodiment, the original audio data can be audio data recorded by various recording devices. The original audio data can be crawled from the web or read from various recording devices; no limitation is made here.

[0157] 302. Extract features from the original audio data to obtain speech features.

[0158] In this embodiment, the original audio data undergoes pre-emphasis processing, frame segmentation, windowing, fast Fourier transform, and filtering using a Mel filter bank to obtain filter bank-based features. Of course, in other embodiments, the speech features can also be Mel frequency cepstrum coefficients (MFCC) features.

[0159] 303. Process the speech features to obtain historical speech training samples and current speech training samples.

[0160] Specifically, in this embodiment, the speech features are shuffled, sorted, and padded to obtain multiple speech training samples. In other embodiments, batch processing is performed after the padded process to obtain multiple batches of training sample sets. For example, each batch of training sample sets includes N speech training samples. After processing the original audio data to obtain multiple batches of training sample sets, the multiple batches of training sample sets are sequentially input into the speech recognition model for training, obtaining the training loss value of the current speech training sample in each batch. The training sample set in the previous batch of two adjacent batches is determined as the historical speech training sample. The training sample set in the next batch of two adjacent batches is determined as the current speech training sample.

[0161] 304. Obtain the maximum and minimum loss values ​​among the training loss values ​​of historical speech training samples and the training loss values ​​of the current speech training sample.

[0162] Specifically, historical speech training samples are input into the speech recognition model for training, yielding the training loss value for the historical speech training samples. Then, the current speech training sample is input into the speech recognition model for training, yielding the training loss value for the current speech training sample. The training loss values ​​of the historical speech training samples and the training loss value of the current speech training sample are combined into a loss value set. The maximum value in the loss value set is determined as the maximum loss value (max), and the minimum value in the loss value set is determined as the minimum loss value (min).

[0163] 305. Normalize the training loss value of the current speech training sample based on the maximum and minimum loss values ​​to obtain the normalized value of the current speech training sample.

[0164] In this embodiment, the maximum-minimum normalization algorithm is used to normalize the training loss value of the current speech training sample based on the maximum and minimum loss values ​​to obtain the normalized value of the current speech training sample.

[0165] Specifically, the formula for calculating the normalized value of the current speech training sample is as follows:

[0166] I-(loss_i-min) / (max-min)

[0167] Where I is the normalized value of the current speech training sample, loss_i is the training loss value of the i-th speech training sample in the set composed of the training loss values ​​of the historical speech training samples and the current speech training sample, i takes the value of 1-2N, max is the maximum loss value, and min is the minimum loss value.

[0168] 306. Input the normalized value of the current speech training sample into the preset nonlinear mapping function to obtain the mapping parameters of the current speech training sample.

[0169] In one specific embodiment, the preset nonlinear mapping function is an incomplete beta function. For example, the mapping parameter for the current speech training sample is λ. Of course, in other embodiments, the preset nonlinear mapping function can also be other monotonically decreasing nonlinear functions.

[0170] 307. Obtain the first parameter mapping relationship for each preset enhanced operation.

[0171] In one specific embodiment, the preset enhancement operation is frequency masking (freq_mask), and the first parameter mapping relationship of frequency masking is: when the mapping parameter is λ, the number of frequency masking is 1+λ; the preset enhancement operation is time masking (time_mask), and the first parameter mapping relationship of time masking is: when the mapping parameter is λ, the number of time masking is 1+λ; the preset enhancement operation is spectrum replacement, and the first parameter mapping relationship of spectrum replacement is: when the mapping parameter is λ, the number of spectrum replacements is λ.

[0172] 308. Map the mapping parameters of the current speech training sample based on the first parameter mapping relationship of each preset enhancement operation to obtain the operation parameters of each preset enhancement operation of the current speech training sample, and obtain the enhancement parameters of the current speech training sample.

[0173] 309. Obtain the probability of selecting each preset enhancement operation for the current speech training sample.

[0174] In a specific embodiment, to accurately determine the selection probability of each preset enhancement operation and obtain the selection probability of each preset enhancement operation for the current speech training sample, it may include:

[0175] (1) Obtain the average first training loss of the current speech training sample in the current round of training iteration and the average second training loss of the current speech training sample in the previous round of training iteration.

[0176] (2) Determine the rate of change of loss based on the average value of the first training loss and the average value of the second training loss.

[0177] Specifically, the ratio of the average value of the first training loss to the average value of the second training loss is defined as the loss change rate.

[0178] (3) Determine the probability of selecting each operation for each preset enhancement operation on the current speech training sample based on the loss change rate.

[0179] The greater the rate of change of loss, the lower the probability of selecting the operation.

[0180] Specifically, the absolute value of the loss change rate is input into a preset nonlinear mapping function to obtain the loss change mapping value. The output loss change mapping value is then subtracted from 1 to obtain the change coefficient cof. Based on the mapping relationship of the second parameter of each preset enhancement operation and the change coefficient cof, the selection probability of each operation for each preset enhancement operation is determined.

[0181] For example, the second parameter mapping relationship for frequency masking is: the change coefficient is cof, and the operation selection probability of frequency masking is P1 = 1 - cof. The second parameter mapping relationship for time masking is: the change coefficient is cof, and the operation selection probability of time masking is P2 = 1 - cof. The second parameter mapping relationship for time replacement is: the change coefficient is cof, and the operation selection probability of time replacement is P3 = 0.5 - cof.

[0182] S310. Preset enhancement operations with a selection probability greater than a preset threshold are identified as target enhancement operations, resulting in multiple target enhancement operations.

[0183] In one specific embodiment, the preset threshold is a random number randomly generated within the range (0,1). Specifically, each preset enhancement operation is determined as a candidate enhancement operation. A random number is randomly generated within the range (0,1), and the probability of selecting the candidate enhancement operation is determined to be greater than the random number. If the probability of selecting the candidate enhancement operation is greater than the random number, the candidate enhancement operation is determined as the target enhancement operation; otherwise, the probability of selecting the candidate enhancement operation is not greater than the random number, the candidate enhancement operation is not determined as the target enhancement operation.

[0184] S311. Perform data augmentation on the current speech training sample based on each target augmentation operation to obtain augmented speech training samples.

[0185] S312. Train the speech recognition model based on the enhanced speech training samples until the preset stopping condition is met.

[0186] For further details, please refer to Figure 4 , Figure 4 This is a flowchart illustrating the speech recognition method provided in an embodiment of this application, as shown below. Figure 4As shown, the flow of the speech recognition method provided in this application is as follows:

[0187] S401. Obtain the speech information to be recognized.

[0188] S402. Obtain the speech recognition model.

[0189] The speech recognition model can be any of the above-mentioned speech recognition models.

[0190] S403. Input the speech information to be recognized into the speech recognition model to obtain the speech recognition result.

[0191] To facilitate better implementation of the speech recognition model training method provided in this application embodiment, this application embodiment also provides a speech recognition model training device based on the above-described speech recognition model training method. The meanings of the terms used are the same as in the above-described speech recognition model training method; for specific implementation details, please refer to the descriptions in the above method embodiments.

[0192] Please refer to Figure 5 , Figure 5 This is a schematic diagram of the structure of a training device for a speech recognition model provided in an embodiment of this application. The training device may include an acquisition module 501, a first training module 502, a mapping module 503, a data augmentation module 504, and a second training module 505.

[0193] Module 501 is used to acquire the current speech training sample;

[0194] The first training module 502 is used to input the current speech training sample into the speech recognition model for training, and obtain the training loss value corresponding to the current speech training sample.

[0195] The mapping module 503 is used to perform nonlinear mapping on the training loss value of the speech training sample to obtain the enhancement parameters of the speech training sample, wherein the enhancement parameters include operation parameters of at least one preset enhancement operation.

[0196] The data augmentation module 504 is used to perform data augmentation on the current speech training sample based on the augmentation parameters of the current speech training sample to obtain an augmented speech training sample;

[0197] The second training module 505 is used to train the speech recognition model based on the enhanced speech training samples until a preset stopping condition is met.

[0198] In an optional embodiment, the mapping module is used for:

[0199] Normalize the training loss value of the current speech training sample to obtain the normalized value of the current speech training sample.

[0200] Input the normalized value of the current speech training sample into a preset nonlinear mapping function to obtain the mapping parameters of the current speech training sample;

[0201] The enhancement parameters for the current speech training sample are determined based on the mapping parameters of the current speech training sample.

[0202] In an optional embodiment, the mapping module is used for:

[0203] Obtain the training loss value of historical speech training samples, where the training loss value of historical speech training samples is obtained by inputting historical speech training samples into the speech recognition model before the current speech training sample;

[0204] Obtain the maximum and minimum loss values ​​among the training loss values ​​of historical speech training samples and the training loss values ​​of the current speech training sample;

[0205] The training loss value of the current speech training sample is normalized based on the maximum and minimum loss values ​​to obtain the normalized value of the current speech training sample.

[0206] In an optional embodiment, the mapping module is used for:

[0207] Obtain the first parameter mapping relationship for each preset enhanced operation;

[0208] The mapping parameters of the current speech training sample are mapped based on the first parameter mapping relationship of each preset enhancement operation to obtain the operation parameters of each preset enhancement operation for the current speech training sample, and thus the enhancement parameters of the current speech training sample.

[0209] In an optional embodiment, the data enhancement module is configured to:

[0210] Obtain the probability of selecting each preset enhancement operation for the current speech training sample;

[0211] Preset augmentation operations with a selection probability greater than a preset threshold are identified as target augmentation operations, resulting in multiple target augmentation operations.

[0212] Data augmentation is performed on the current speech training samples based on various target augmentation operations to obtain augmented speech training samples.

[0213] In an optional embodiment, the data enhancement module is configured to:

[0214] Obtain the average first training loss of the current speech training sample in the current iteration and the average second training loss of the current speech training sample in the previous iteration.

[0215] The rate of change of loss is determined based on the average of the first training loss and the average of the second training loss.

[0216] The probability of selecting each preset enhancement operation for the current speech training sample is determined based on the loss change rate.

[0217] In an optional embodiment, the acquisition module is configured to:

[0218] Obtain the raw audio data;

[0219] Feature extraction is performed on the raw audio data to obtain speech features;

[0220] The speech features are processed to obtain the current speech training samples.

[0221] For details on the implementation of each of the above modules, please refer to the previous examples, which will not be repeated here.

[0222] To facilitate better implementation of the speech recognition model training method provided in this application embodiment, this application embodiment also provides a speech recognition model training device based on the above-described speech recognition model training method. The meanings of the terms used are the same as in the above-described speech recognition model training method; for specific implementation details, please refer to the descriptions in the above method embodiments.

[0223] Please refer to Figure 6 , Figure 6 This is a schematic diagram of the structure of a speech recognition device provided in an embodiment of this application. The speech recognition device may include a speech acquisition module 601, a model acquisition module 602, and a speech recognition module 603, wherein...

[0224] The voice acquisition module 601 is used to acquire the voice information to be recognized.

[0225] The model acquisition module 602 is used to acquire a speech recognition model, wherein the speech recognition model is any one of the first aspects.

[0226] The speech recognition module 603 is used to input the speech information to be recognized into the speech recognition model to obtain the speech recognition result.

[0227] This application also provides an electronic device, including a memory and a processor, wherein the processor executes steps in the training method of the speech recognition model or steps in the speech recognition method provided in this embodiment by calling a computer program stored in the memory.

[0228] Please refer to Figure 7 , Figure 7 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application.

[0229] The electronic device may include components such as a processor 101 with one or more processing cores, a memory 102 with one or more computer-readable storage media, a power supply 103, and an input unit 104. Those skilled in the art will understand that... Figure 7 The electronic device structure shown does not constitute a limitation on the electronic device and may include more or fewer components than shown, or combine certain components, or have different component arrangements. Wherein:

[0230] The processor 101 is the control center of the electronic device, connecting various parts of the device via various interfaces and lines. It executes software programs and / or modules stored in the memory 102, and calls data stored in the memory 102, to perform various functions and process data. Optionally, the processor 101 may include one or more processing cores; alternatively, the processor 101 may integrate an application processor and a modem processor, wherein the application processor mainly handles the operating system, user interface, and applications, while the modem processor mainly handles wireless communication. It is understood that the modem processor may not be integrated into the processor 101.

[0231] The memory 102 can be used to store software programs and modules. The processor 101 executes various functional applications and data processing by running the software programs and modules stored in the memory 102. The memory 102 may mainly include a program storage area and a data storage area. The program storage area may store the operating system, application programs required for at least one function (such as sound playback function, image playback function, etc.), etc.; the data storage area may store data created according to the use of the electronic device, etc. In addition, the memory 102 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device. Accordingly, the memory 102 may also include a memory controller to provide the processor 101 with access to the memory 102.

[0232] The electronic device also includes a power supply 103 that supplies power to the various components. Optionally, the power supply 103 can be logically connected to the processor 101 through a power management system, thereby enabling functions such as charging, discharging, and power consumption management through the power management system. The power supply 103 may also include one or more DC or AC power supplies, recharging systems, power fault detection circuits, power converters or inverters, power status indicators, and other arbitrary components.

[0233] The electronic device may also include an input unit 104, which can be used to receive input digital or character information and generate keyboard, mouse, joystick, optical or trackball signal inputs related to user settings and function control.

[0234] Although not shown, the electronic device may also include a display unit, an image acquisition component, etc., which will not be described in detail here. Specifically, in this embodiment, the processor 101 in the electronic device will load one or more executable codes corresponding to computer programs into the memory 102 according to the following instructions, and the processor 101 will execute the training method and / or steps of the speech recognition method of the speech recognition model provided in this application, such as:

[0235] Get the current speech training sample;

[0236] Input the current speech training sample into the speech recognition model for training, and obtain the training loss value corresponding to the current speech training sample;

[0237] The training loss value of the speech training sample is nonlinearly mapped to obtain the enhancement parameters of the speech training sample, wherein the enhancement parameters include operation parameters of at least one preset enhancement operation.

[0238] Based on the augmentation parameters of the current speech training sample, data augmentation is performed on the current speech training sample to obtain augmented speech training samples;

[0239] The speech recognition model is trained based on enhanced speech training samples until a preset stopping condition is met.

[0240] Alternatively, obtain the speech information to be recognized;

[0241] Obtain the speech recognition model;

[0242] The speech information to be recognized is input into the speech recognition model to obtain the speech recognition result.

[0243] It should be noted that the electronic device provided in this application embodiment belongs to the same concept as the training method and / or speech recognition method of the speech recognition model in the above embodiment. The specific implementation process can be found in the above related embodiments, and will not be repeated here.

[0244] This application also provides a computer-readable storage medium storing a computer program thereon. When the computer program stored thereon is executed on the processor of the electronic device provided in the embodiments of this application, the processor of the electronic device executes the steps in the training method and / or speech recognition method of the speech recognition model provided in this application. The storage medium may be a magnetic disk, optical disk, read-only memory (ROM), or random access memory (RAM), etc.

[0245] This application also provides a computer program product or computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform various optional implementations of the above-described speech recognition model training method and / or speech recognition method.

[0246] The training method, speech recognition method, and apparatus for a speech recognition model provided in this application have been described in detail above. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.

[0247] It should be noted that when the above embodiments of this application are applied to specific products or technologies, and user-related data is involved, user permission or consent is required, and the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions.

Claims

1. A method for training a speech recognition model, characterized in that, include: Get the current speech training sample; The current speech training sample is input into the speech recognition model for training, and the training loss value corresponding to the current speech training sample is obtained. The training loss value of the current speech training sample is nonlinearly mapped to obtain the enhancement parameters of the speech training sample, wherein the enhancement parameters include operation parameters of at least one preset enhancement operation. Based on the enhancement parameters of the current speech training sample, data augmentation is performed on the current speech training sample to obtain an enhanced speech training sample. This involves obtaining the selection probability of each preset enhancement operation on the current speech training sample, obtaining the first average training loss obtained in the current iteration and the second average training loss obtained in the previous iteration; determining the loss change rate based on the first and second average training losses; inputting the loss change rate into a preset nonlinear mapping function to obtain a change coefficient; determining the selection probability of each preset enhancement operation based on the second parameter mapping relationship and the change coefficient; a larger loss change rate corresponds to a larger change coefficient and a smaller operation selection probability; identifying each preset enhancement operation as a candidate enhancement operation; generating a random number within the range (0,1); determining whether the operation selection probability of the candidate enhancement operation is greater than the random number; if the operation selection probability of the candidate enhancement operation is greater than the random number, then identifying the candidate enhancement operation as a target enhancement operation, resulting in multiple target enhancement operations; and performing data augmentation on the current speech training sample based on each target enhancement operation to obtain an enhanced speech training sample. The speech recognition model is trained based on the enhanced speech training samples until a preset stopping condition is met.

2. The training method for the speech recognition model according to claim 1, characterized in that, The step of performing a non-linear mapping on the training loss value of the current speech training sample to obtain the enhancement parameters of the speech training sample includes: The training loss value of the current speech training sample is normalized to obtain the normalized value of the current speech training sample. The normalized value of the current speech training sample is input into a preset nonlinear mapping function to obtain the mapping parameters of the current speech training sample. The enhancement parameters of the current speech training sample are determined based on the mapping parameters of the current speech training sample.

3. The training method for the speech recognition model according to claim 2, characterized in that, The step of normalizing the training loss value of the current speech training sample to obtain the normalized value of the current speech training sample includes: Obtain the training loss value of historical speech training samples, wherein the training loss value of the historical speech training samples is obtained by inputting the historical speech training samples into the speech recognition model before the current speech training samples; Obtain the maximum and minimum loss values ​​among the training loss values ​​of the historical speech training samples and the training loss values ​​of the current speech training samples; The training loss value of the current speech training sample is normalized based on the maximum loss value and the minimum loss value to obtain the normalized value of the current speech training sample.

4. The training method for the speech recognition model according to claim 2, characterized in that, The step of determining the enhancement parameters of the current speech training sample based on the mapping parameters of the current speech training sample includes: Obtain the first parameter mapping relationship for each preset enhanced operation; The mapping parameters of the current speech training sample are mapped based on the first parameter mapping relationship of each preset enhancement operation to obtain the operation parameters of each preset enhancement operation for the current speech training sample, and thus the enhancement parameters of the current speech training sample are obtained.

5. The training method for the speech recognition model according to claim 1, characterized in that, The step of obtaining the current speech training sample includes: Obtain the raw audio data; Feature extraction is performed on the raw audio data to obtain speech features; The speech features are processed to obtain the current speech training sample.

6. A speech recognition method, characterized in that, The speech recognition method includes: Obtain the speech information to be recognized; A speech recognition model is obtained, wherein the speech recognition model is trained by the training method according to any one of claims 1-5; The speech information to be recognized is input into the speech recognition model to obtain the speech recognition result.

7. A training device for a speech recognition model, characterized in that, include: The acquisition module is used to acquire the current speech training samples; The first training module is used to input the current speech training sample into the speech recognition model for training, and obtain the training loss value corresponding to the current speech training sample. The mapping module is used to perform nonlinear mapping on the training loss value of the current speech training sample to obtain the enhancement parameters of the speech training sample, wherein the enhancement parameters include operation parameters of at least one preset enhancement operation. The data augmentation module is used to augment the current speech training sample based on the augmentation parameters of the current speech training sample to obtain an augmented speech training sample. Specifically, it acquires the selection probability of each preset augmentation operation for the current speech training sample, and acquires the first average training loss obtained in the current iteration and the second average training loss obtained in the previous iteration. Based on the first and second average training losses, it determines the loss change rate. The loss change rate is input into a preset nonlinear mapping function to obtain a change coefficient. The selection probability of each preset augmentation operation is determined according to the second parameter mapping relationship and the change coefficient; a larger loss change rate corresponds to a larger change coefficient and a smaller operation selection probability. Each preset augmentation operation is identified as a candidate augmentation operation. A random number is randomly generated within the range (0,1). The selection probability of the candidate augmentation operation is determined to be greater than the random number. If the selection probability of the candidate augmentation operation is greater than the random number, the candidate augmentation operation is identified as a target augmentation operation, resulting in multiple target augmentation operations. Based on each target augmentation operation, the current speech training sample is augmented to obtain an augmented speech training sample. The second training module is used to train the speech recognition model based on the enhanced speech training samples until a preset stopping condition is met.

8. A voice recognition device, characterized in that, include: The voice acquisition module is used to acquire the voice information to be recognized. The model acquisition module is used to acquire speech recognition models. Wherein, the speech recognition model is trained by the training method described in any one of claims 1-5; The speech recognition module is used to input the speech information to be recognized into the speech recognition model to obtain the speech recognition result.

9. An electronic device, characterized in that, The method includes a memory and a processor, the memory storing a computer program, and the processor running the computer program in the memory to perform the steps in the training method of the speech recognition model according to any one of claims 1 to 5 or the steps in the speech recognition method according to claim 6.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a plurality of instructions adapted for loading by a processor to execute steps in the training method of the speech recognition model according to any one of claims 1 to 5 or steps in the speech recognition method according to claim 6.

11. A computer program product, comprising a computer program or instructions, characterized in that, When the computer program or instructions are executed by the processor, they implement the steps in the training method of the speech recognition model according to any one of claims 1 to 5 or the steps in the speech recognition method according to claim 6.

Images