Speech recognition model training method and device, medium, equipment and program product

Abstract

The invention discloses a voice recognition model training method and device, a medium, equipment and a program product, and relates to the technical field of artificial intelligence, and the method comprises the steps: determining the voice feature length of each piece of sample voice data of a sample data set and the text label length of each text label; performing sample batching on the plurality of samples based on the sequence of the voice feature length and the text label length to obtain a plurality of batch data, the sample size of each batch of data being limited by a preset batch size and a preset video memory index, and the preset batch size being used for limiting the sample size upper limit of the batch data; the preset video memory index is an expected value required to be reached by a batch video memory index of the batch data, and the batch video memory index is used for indicating the video memory consumption degree of the batch data in the model training process and is determined by performing video memory consumption estimation on the batch data based on the voice feature length and the text label length; and training the initial recognition model based on the plurality of batch data to obtain a target speech recognition model. The training efficiency and the resource utilization rate can be remarkably improved.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, specifically to training methods, devices, media, equipment, and program products for speech recognition models. Background Technology

[0002] Due to the unique nature of speech data, compared to image and natural language processing tasks, the training process of speech recognition models involves two parts of data simultaneously: input audio and text labels. The lengths of these two parts of data are not strictly correlated due to differences in speech rate. Furthermore, due to the continuous nature of speech data, it is impossible to divide it into fixed lengths, and the data size varies greatly. Therefore, it is difficult to use a fixed batch size for model training, resulting in uneven data batching and consequently, high GPU memory usage.

[0003] To address the aforementioned issues, related technologies divide speech data into blocks based on the audio length and set a limit on the total length of each batch to ensure that the length of each block is similar. This means the batch size is constrained according to the length of the input audio, thereby improving the effective utilization of GPU resources. However, since the length of speech features cannot represent the speech rate and information content of the audio, some data with excessively fast speech rates can cause memory overflow. Therefore, the overall batch size needs to be appropriately reduced, but problems such as slow training speed and low memory utilization still exist. Summary of the Invention

[0004] This application provides a method, apparatus, medium, device, and program product for training a speech recognition model. The technical solution is as follows: On the one hand, this application provides a method for training a speech recognition model, the method comprising: Obtain a sample dataset, which includes multiple samples, each sample containing sample speech data and corresponding text labels for the sample speech data; Determine the speech feature length of each sample speech data and the text label length of each text label; Based on the sorting of the speech feature lengths and the text label lengths, the multiple samples are divided into batches to obtain multiple batches of data. The sample size of each batch of data is limited by a preset batch size and a preset memory index. The preset batch size is used to limit the upper limit of the sample size of the batch of data. The preset memory index is the expected value that the batch memory index of the batch of data needs to reach. The batch memory index is used to indicate the degree of memory consumption of the batch of data during model training. It is determined based on the speech feature lengths and text label lengths to estimate the memory consumption of the batch of data. The initial recognition model is iteratively trained based on the multiple batches of data to obtain the target speech recognition model.

[0005] On the other hand, this application provides a training apparatus for a speech recognition model, the apparatus comprising: Acquisition module: used to acquire sample dataset, which includes multiple samples, each sample including sample speech data and text labels corresponding to the sample speech data; Determination module: used to determine the length of the speech features of each sample speech data and the length of the text label of each text label; Sample batching module: used to batch the multiple samples based on the sorting of the speech feature length and the text label length to obtain multiple batches of data. The sample size of each batch of data is limited by a preset batch size and a preset memory index. The preset batch size is used to limit the upper limit of the sample size of the batch of data. The preset memory index is the expected value that the batch memory index of the batch of data needs to reach. The batch memory index is used to indicate the memory consumption of the batch of data during model training. It is determined based on the speech feature length and the text label length to estimate the memory consumption of the batch of data. Training module: used to iteratively train the initial recognition model based on the multiple batches of data to obtain the target speech recognition model.

[0006] On the other hand, this application provides a computer-readable storage medium storing at least one instruction or at least one program, which is loaded and executed by a processor to implement the training method of the speech recognition model as described above.

[0007] On the other hand, this application provides a computer device including a processor and a memory, wherein the memory stores at least one instruction or at least one program, the at least one instruction or at least one program being loaded and executed by the processor to implement the training method for the speech recognition model as described above.

[0008] On the other hand, this application provides a computer program product, which includes computer instructions that, when executed by a processor, implement the training method for the aforementioned speech recognition model.

[0009] The training method, apparatus, medium, device, and program product for the speech recognition model provided in this application have the following technical effects: The technical solution of this application divides multiple samples into batches based on the sorting of speech feature lengths and text label lengths. During the batching process, memory consumption is estimated based on the speech feature length and text label length to determine the batch memory index for each batch of data. At the same time, the memory consumption indicated by the batch memory index of each batch of data is similar to the preset memory index, so that each training batch in the training data has similar memory usage, improving the fit between batch data storage and memory resources, and maximizing the overall batch size of the data. This ensures the effective utilization of memory resources and computing core resources during training, improving training efficiency and speed. Furthermore, by simultaneously limiting the memory usage of batch data based on speech feature length, text label length, batch memory index, and preset memory index, and combining this with the preset batch size to limit the sample size of the batch data, the problem of memory overflow of some speech data and overload of batch data training update information can be effectively avoided, further improving the training effect.

[0010] Additional aspects and advantages of this application will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of this application. Attached Figure Description

[0011] To more clearly illustrate the technical solutions and advantages in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0012] Figure 1 This is a schematic diagram of an application environment provided in an embodiment of this application; Figure 2 This is a flowchart illustrating a training method for a speech recognition model provided in an embodiment of this application; Figure 3 This is a flowchart illustrating another method for training a speech recognition model provided in an embodiment of this application; Figure 4 This is a flowchart illustrating another method for training a speech recognition model provided in an embodiment of this application; Figure 5 This is a flowchart illustrating another method for training a speech recognition model provided in an embodiment of this application; Figure 6 This is a flowchart illustrating another method for training a speech recognition model provided in an embodiment of this application; Figure 7 This is a schematic diagram of the structural framework of a speech recognition model training device provided in an embodiment of this application; Figure 8 This is a schematic diagram of the hardware structure of a device for implementing a training method for a speech recognition model, provided in an embodiment of this application. Detailed Implementation

[0013] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application. Examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout.

[0014] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or server that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or devices.

[0015] Please see Figure 1 , Figure 1 This is a schematic diagram of an application environment provided in an embodiment of this application, such as... Figure 1 As shown, the application environment may include at least server 01. In this embodiment, server 01 may be an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN (Content Delivery Network), and big data and artificial intelligence platforms.

[0016] Specifically, the server 01 mentioned above may include a physical device, which may include a network communication submodule, a processor and a memory, etc., and may also include software running on the physical device, which may include applications, etc.

[0017] In this embodiment, server 01 can be used to acquire a sample dataset and divide the samples in the dataset into batches based on the speech feature length of the sample speech data and the text label length of the text labels, to obtain multiple batches of data for initial recognition model training. During the sample batching process, a preset batch size and a preset memory index are used to limit the sample size and memory index of a single batch of data. The batch memory index is used to indicate the memory consumption of the batch of data during model training and is determined based on the memory consumption prediction of the batch of data based on the speech feature length and the text label length. Specifically, server 01 can also be used to train an initial recognition model based on each batch of data to obtain a target speech recognition model.

[0018] In some embodiments, server 01 can be a distributed system including multiple nodes. Each node is configured with a sample dataset and corresponding multiple batches of data. The multiple nodes perform distributed synchronous training on the initial recognition model. Each node updates the initial recognition model based on gradient synchronization to finally obtain the target speech recognition model.

[0019] Furthermore, it is understandable that Figure 1 The illustration only depicts an application environment for a video representation method, which may include more or fewer nodes; this application does not impose any limitations on this. For example, it may also include a terminal, and the terminal and server 01 can be directly or indirectly connected via wired or wireless communication; this application does not impose any limitations on this either. Specifically, the terminal may include physical devices such as smartphones, desktop computers, tablets, laptops, digital assistants, augmented reality (AR) / virtual reality (VR) devices, smart voice interaction devices, smart home appliances, smart wearable devices, and in-vehicle terminal devices, and may also include software running on the physical device, such as applications.

[0020] It is understood that in the specific implementation of this application, sample voice data, text tags, voice data and other related data are involved. When the embodiments of this application are applied to specific products or technologies, permission or consent from users or related parties is required, and the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions.

[0021] The following, with reference to the accompanying drawings, describes a training method for a speech recognition model provided in this application, which can be applied to the server side. Figure 2This is a flowchart illustrating a training method for a speech recognition model provided in an embodiment of this application. This application provides method operation steps as shown in the embodiments or flowcharts, but based on conventional or non-inventive methods, more or fewer operation steps may be included. The order of steps listed in the embodiments is merely one possible execution order among many and does not represent the only possible execution order. In actual system or server product execution, the method can be executed sequentially according to the embodiments or accompanying drawings, or in parallel (e.g., in a parallel processor or multi-threaded processing environment). Please refer to... Figure 2 The training method for a speech recognition model provided in this application embodiment may include the following steps S201-S207: S201: Obtain the sample dataset, which includes multiple samples. Each sample includes sample speech data and the corresponding text labels.

[0022] Specifically, text labels are used to characterize the text information contained in the audio of the sample speech data, or to describe the audio content of the sample speech data, and can be set based on the application task requirements of the target speech recognition model. The length of the text label is positively correlated with the amount of information contained in the sample speech data, where the amount of information refers to the amount of content information that needs to be recognized from the sample speech data, such as the amount of text.

[0023] In some embodiments, reference is made to Figure 3 S201 may include S301-S305: S301: Obtain the initial sample set; S303: For each initial sample, fuse the speech feature length of the sample speech data and the text label length of the text label in the initial sample to obtain the initial sample's memory coefficient. S305: Remove initial samples from the initial sample set whose memory coefficient is higher than the preset memory index to obtain the sample dataset.

[0024] Specifically, the initial sample set includes multiple initial samples, which consist of sample speech data and text labels. For each initial sample in the initial sample set, a memory coefficient is calculated based on its speech feature length and text label length. This memory coefficient indicates the memory consumption of the initial sample during model training and is positively correlated with the memory consumption required by the initial sample.

[0025] Specifically, the length of the speech feature is positively correlated with the audio length of the sample speech data. For example, the length of the speech feature can be equal to the audio length of the sample speech data, or it can be the length of the speech vector obtained after feature representation of the sample speech data. The length of the text label is positively correlated with the text length of the text label. For example, the length of the text label can be equal to the text length of the text label, or it can be the length of the text vector obtained after feature representation of the text label.

[0026] In some implementations, the memory coefficient of the initial sample can be obtained by length fusion of the speech feature length and the text label length. Length fusion can be multiplication, addition, or weighted addition. For example, memory coefficient = speech feature length * text label length.

[0027] Specifically, the preset video memory index is set based on the video memory resource configuration of the device performing model training, and is positively correlated with the device's video memory resources. Understandably, the closer the video memory consumption corresponding to the preset index is to the device's maximum video memory capacity, the higher the video memory resource utilization during model training. If the video memory coefficient of the initial sample is higher than the preset index, it indicates that the initial sample occupies too much memory, or that the data length or amount of data is too high, which can easily cause video memory overflow for a single sample or the entire batch of data. By filtering out initial samples with video memory coefficients greater than the preset index, the video memory consumption of each sample in the dataset can be optimized, reducing the risk of video memory overflow during training, while avoiding the impact of abnormal data on the training process, fully utilizing graphics card resources, and improving the training speed of the model.

[0028] S203: Determine the length of the speech features of each sample speech data and the length of the text label of each text label.

[0029] Specifically, for each sample in the sample dataset, its speech feature length and text label length are obtained to facilitate speech length sorting and information content estimation of the speech data. Understandably, the larger the speech feature length, the longer the audio length of the sample speech data; the larger the text label length, the greater the information content of the sample speech data.

[0030] S205: Based on the sorting of speech feature lengths and the length of text labels, multiple samples are divided into batches to obtain multiple batches of data.

[0031] Specifically, batch data is a subset of the sample dataset, including two or more samples. Each batch of data generated from the same sample dataset is a distinct set. All samples in the sample dataset are divided into batches based on the length of the speech feature and the length of the text label, resulting in multiple batches. Each batch of data refers to a batch of data processed in parallel during the training process of the speech recognition model, when optimizing the neural network.

[0032] Specifically, the sample size of each batch of data is limited by a preset batch size and a preset memory limit. The sample size of the batch data refers to the batch size, which is the number of samples in each batch of data processed in parallel. The preset batch size can be set based on the upper limit of the number of samples in a single iteration during model training, or it can be an empirical value. The preset batch size is used to limit the upper limit of the sample size of the batch data.

[0033] Specifically, the batch memory consumption metric indicates the memory consumption of a batch of data during model training. It is determined by estimating the memory consumption of the batch data based on the length of the speech features and the length of the text labels. The determination of the batch memory consumption metric considers both the length of the speech features and the length of the text labels on the memory consumption, taking into account both audio length and data information content, thus making the memory consumption level of the batch data indicated by the batch memory consumption metric more accurate. The batch memory consumption metric is positively correlated with the memory consumption level of the batch data.

[0034] Specifically, the preset video memory metric is the expected value that the batch video memory metric of the batch data needs to achieve; that is, the batch video memory metric of each batch of data is close to the preset video memory metric. Understandably, the batch video memory metric of each batch of data increases with the increase of the sample size. Using the preset video memory metric as a constraint, the preset video memory metric serves as the expected value of the batch video memory metric, avoiding both excessively low and excessively high batch video memory metrics, while also avoiding excessively low or excessively high sample sizes. In a preferred embodiment, the preset video memory metric is used as the upper limit of the batch video memory metric for the batch data.

[0035] In some implementations, the batch memory consumption index is obtained by: determining the maximum speech feature length and maximum text label length of each sample in the batch data; fusing the maximum speech feature length and maximum text label length of the batch data to obtain the batch memory consumption coefficient, which is used to estimate the memory consumption of a single sample in the batch data during model training; and using the batch memory consumption coefficient as the memory consumption coefficient of a single sample in the batch data to calculate the memory consumption of the batch data, thus obtaining the batch memory consumption index. Fusing the maximum speech feature length and maximum text label length of the batch data refers to multiplying, summing, or weighted summing the two. For example, the batch memory consumption coefficient is the product of the maximum speech feature length and maximum text label length of the batch data, and the batch memory consumption index = sample size of the batch data * batch memory consumption coefficient.

[0036] In other implementations, the batch memory index is obtained by: for each sample in the batch data, fusing the speech feature length and text tag length of the sample to obtain the memory coefficient of the sample; fusing the memory coefficients of each sample in the batch data to obtain the batch memory index. Fusing the memory coefficients of each sample in the batch data can be achieved by summing the memory coefficients to obtain the batch memory index.

[0037] S207: Iteratively train the initial recognition model based on multiple batches of data to obtain the target speech recognition model.

[0038] Specifically, after dividing the data into multiple batches, the initial recognition model is iteratively trained. In each iteration, at least one batch of data can be used as training samples to update and optimize the model parameters until the training termination condition is met, thus obtaining the target speech recognition model. The training termination condition refers to the number of iterations reaching a preset number or the model loss falling below a preset loss.

[0039] Understandably, due to the unique nature of speech data, compared to image and natural language processing tasks, speech recognition model training involves both input audio and text labels. The lengths of these two data components are not strictly correlated due to differences in speech rate. During speech recognition model training, the memory required for the model's recognition process and the calculation of the loss function are related to the audio length and information content. For example, the memory required for loss functions like CTC (Connectionist Temporal Classification) and Transducer is related to the input audio length and information content. While constraining batch size based on the length of the input audio in related techniques can improve GPU resource utilization, it only considers the length of the input speech features when forming batches, neglecting the information content. This can lead to memory overflow issues caused by some fast-paced data during training. Reducing the overall batch size to accommodate this data would slow down the training process. The technical solution of this application divides multiple samples into batches based on the sorting of speech feature lengths and text label lengths. During the batching process, memory consumption is estimated based on the speech feature length and text label length to determine the batch memory index for each batch of data. At the same time, the memory consumption indicated by the batch memory index of each batch of data is similar to the preset memory index, so that each training batch in the training data has similar memory usage, improving the fit between batch data storage and memory resources, and maximizing the overall batch size of the data. This ensures the effective utilization of memory resources and computing core resources during training, improving training efficiency and speed. Furthermore, by simultaneously limiting the memory usage of batch data based on speech feature length, text label length, batch memory index, and preset memory index, and combining this with the preset batch size to limit the sample size of the batch data, the problem of memory overflow of some speech data and overload of batch data training update information can be effectively avoided, further improving the training effect.

[0040] Based on some or all of the above embodiments, in some embodiments, reference is made to Figure 4S205 may include S401-S403: S401: Based on the descending order of speech feature length, multiple samples are sequentially combined, and the sample size and estimated memory index of the combined samples after each new sample are added are dynamically detected. If the sample size meets the preset batch size limit or the estimated memory index meets the preset memory index limit, the samples of the current combination are divided into batch data. S403: Starting with the next sample in descending order, repeatedly perform the sequential combination and dynamic detection operations to divide the next batch of data, until the sample batches of each sample in the sample dataset are completed, resulting in multiple batches of data.

[0041] Specifically, before batching the sample data, the samples in the dataset are sorted in descending order of speech feature length. Then, the samples are sequentially combined in descending order. After each new sample is added, the sample size and estimated memory index of the current combination are checked to compare the real-time sample size with the preset batch size, and the real-time estimated memory index with the preset memory index. If the current sample size is less than the preset batch size and the current estimated memory index is less than or equal to the preset memory index, it indicates that the sample group does not meet the batch data generation conditions, and the next sample in descending order is added to the sample group. If the sample size is greater than or equal to the preset batch size or the estimated memory index is greater than the preset memory index, it is determined that the current sample group meets the batch data generation conditions, and it is divided into a batch of data. Next, starting with the next sample after the last sample added in the generated current batch of data, the remaining samples in the sample dataset are sequentially combined in descending order, and the above operation of dynamically checking the sample size and estimated memory index of the new sample group is repeated until the batch data generation conditions are met, so as to realize the construction of the next batch of data. This process continues until all samples in the complete dataset are divided into batches, resulting in N batches. It's important to note that after generating the (N-1)th batch, if the sample size of a group formed by the remaining samples is less than the preset batch size and the estimated memory usage is lower than the preset memory usage, it will still be grouped into a single batch to avoid wasting training data. Thus, by dividing the samples into batches in descending order of speech feature length and using an estimated memory usage metric combining speech feature length and text label length, the batch size is limited by the speech feature length and text label length, ensuring that the memory usage of each training batch is similar, thereby improving training efficiency and memory utilization.

[0042] Specifically, the estimated video memory index of the sample group formed by the combined samples is used to indicate the overall video memory consumption of the currently combined samples after adding the next sample in descending order. That is, if the estimated video memory index calculated after adding the next sample is greater than the preset video memory index, it indicates that the sample group formed by the currently combined samples has approached the upper limit of video memory consumption. Adding the next sample will lead to a higher video memory index than the preset video memory index, that is, there is a risk of video memory overflow. The current sample group without adding the next sample will be divided into batch data. If the estimated video memory index is still less than or equal to the preset video memory index after adding the next sample, it indicates that the video memory consumption of the current sample group after adding another sample has not exceeded the video memory consumption corresponding to the preset video memory index. Another sample can be added.

[0043] In some embodiments, reference is made to Figure 5 S401 can include S4011-S4017: S4011: Create batch set; S4012: Enumerate each sample in descending order of speech feature length to add each sample to the batch set in sequence; S4013: After adding a sample to the batch set, determine the current sample size of the batch set, and determine the estimated memory index of the batch set based on the speech feature length and text feature length of each sample in the batch set. S4014: If the current sample size reaches the preset batch size, or the estimated memory index of the batch set reaches the preset memory index, the current batch data is formed based on each sample in the batch set.

[0044] S4015: If the current sample size does not reach the preset batch size and the estimated memory index of the batch set does not reach the preset memory index, add the next sample in descending order among the multiple samples to the batch set. S4016: Repeat the steps of determining the sample size and estimated memory index of the batch set to obtain the updated sample size and the updated estimated memory index. S4017: If the updated sample size reaches the preset batch size, or the updated estimated memory index reaches the preset memory index, execute the step of forming the current batch data based on each sample in the batch set.

[0045] Specifically, during the sample batching process, an empty batch set is first created, and a preset batch size and preset memory coefficient are obtained. Then, each sample in the sample dataset, sorted in descending order based on the length of its speech features, is enumerated and added to the batch set one by one, starting with the sample with the longest speech feature length. After each sample is added, the number of samples in the current batch set (i.e., the number of elements) is determined, and the memory consumption of the current batch set is estimated to determine its estimated memory index. The estimated memory index indicates the estimated memory consumption after adding the next sample to the current batch set; that is, it estimates the memory consumption of the updated batch set after adding another sample. If the current sample size is less than the preset batch size and the estimated memory index is less than or equal to the preset memory index, the next sample is added to the batch set in descending order. If at least one of the following conditions is met: the current sample size reaches the preset batch size and the estimated memory index is greater than the preset memory index, the current samples in the batch set are divided into a batch and added to the batch dataset. That is, after each new sample is added, the sample size and estimated memory usage of the batch set are checked to dynamically track the size of the batch set and the estimated memory usage until the batch data generation conditions are met. After each sample in the current batch set is divided into batch data, the batch set is cleared. For the remaining undivided samples in the sample dataset, steps S4012-S4017 above are repeated, and this cycle is repeated until all samples are enumerated, completing the batch data division. It can be understood that for the last remaining sample, even if the sample size is less than the preset batch size and the estimated memory usage is less than the preset memory usage, it is still packaged into a batch data and added to the batch dataset. In this way, batch data division by creating an empty batch set and enumerating in descending order is beneficial for dynamically updating the memory consumption of sample groups during sample batching. The number of samples to be put into the current batch set is determined based on the dynamic sample size and the estimated memory usage, improving the efficiency of sample batching and the rationality of batch data division.

[0046] In some embodiments, the method for determining the estimated memory index of the batch set includes: determining the maximum speech feature length and the maximum text label length of each sample in the batch set; fusing the maximum speech feature length and the maximum text label length to obtain the memory coefficient of the batch set; using the memory coefficient of the batch set as the memory coefficient of a single sample in the batch set, estimating the memory consumption after the batch set adds the next sample, and obtaining the estimated memory index of the batch set.

[0047] Specifically, the maximum speech feature length L x max This refers to the length of the speech features of the longest audio sample data in a batch, and the maximum text label length L. y max This refers to the length of the longest text tag in the batch set. The memory coefficient L of the batch set is obtained by combining the two.max,cur Here, fusion refers to multiplying, adding, or weighting the maximum speech feature length and the maximum text label length; for example, L max,cur = L x max *L y max The memory consumption coefficient of the batch set is used to predict the memory consumption of a single sample in the batch set during model training. The memory consumption coefficient of the batch set is used as the memory consumption coefficient of each sample in the batch set and the memory consumption coefficient of the next sample in descending order. The total memory consumption of the batch set after adding the next sample is estimated, resulting in the predicted memory consumption index of the batch set. For example, the predicted memory consumption index M of the batch set... total = (N cur +1)* L max,cur Accordingly, the batch memory consumption index = batch data sample size * batch data memory coefficient. Thus, estimating the memory consumption of batch sets based on the maximum speech feature length and the maximum text label length can ensure efficient use of memory while avoiding the risk of memory overflow during parallel training of batch data.

[0048] In other embodiments, the method for determining the estimated memory index of the batch set includes: for each sample in the current batch set and the next sample corresponding to the batch set, fusing the speech feature length and text label length of the sample to obtain the memory index of the sample; fusing the memory index of each sample in the batch set and the memory index of the next sample to obtain the estimated memory index of the batch set.

[0049] Specifically, the next sample in a batch refers to the sample adjacent to the last sample added in the current batch, ordered by descending speech feature length. The sample's memory coefficient indicates the memory consumption during model training; it can be the product, sum, or weighted sum of the speech feature length and text label length. Specifically, fusing the memory coefficients of each sample in the batch and the next sample involves summing these coefficients to obtain the estimated memory index for the batch. Correspondingly, the batch memory index is the sum of the memory coefficients of each sample in the batch. Thus, by calculating the accurate memory coefficient for each sample and then fusing them, the accuracy of the memory consumption prediction for the batch after adding the next sample is improved, thereby enhancing the compatibility between batch data storage and device memory resources and optimizing memory resource utilization.

[0050] In one embodiment, sample batching based on memory consumption pre-estimation can be implemented using the following process: Given a sample dataset {X, Y, D}, where X represents the sample speech dataset, Y represents the text label set, D represents the speech feature length set, and the i-th sample in the sample dataset is {x i , yi , d i}, x i y represents the sample speech data or the speech features corresponding to the sample speech data. i Indicates text label, d i This represents the length of the speech features. After processing by the sample batching algorithm described above, a batch dataset B is returned. This set contains multiple batches of data obtained by dividing the sample dataset. A single batch of data is an element in batch dataset B, and each batch contains a certain number of training samples. A specific algorithm implementation example is as follows: Obtain the initial sample set {X', Y', D'} Set the preset video memory index M max and preset batch size B max Obtain the initial sample set length L', where L' represents the sample size in the initial sample set. Initialize the batch dataset B = ∅ and the sub-batch dataset B. cur = ∅, Initialize the current maximum speech feature length L of the batch set x max = 0 and maximum text tag length L y max = 0 Initialize the estimated video memory size M for batch sets total Each sample in the initial sample set {X', Y', D'} is sorted according to d i Arrange in descending order Set counter i = 0 WHILE i <L' DO Obtain the speech feature length L of the i-th sample in the sorted initial sample set. x i and text label length L y i Calculate the memory coefficient M of the i-th sample. cur = L x i ×L y i IF M cur >M max THEN, filter the i-th sample Update counter i = i + 1 CONTINUE END IF Generate a sample dataset {X, Y, D} with a length of L. Update Lx max = max(L x max , L x i ) Update L y max = max(L y max , L y i ) Get batch set B cur The sample size N cur Update M total = (N cur + 1) ×L x max ×L y max IF M total >M max OR N cur ≥ B max THEN B cur Put into batch dataset B Reset B cur = ∅ Reset L x max = 0 Reset L y max = 0 CONTINUE END IF The i-th data item {x} in the sorted sample dataset i , y i}Put into set B cur Update counter i = i + 1 END WHILE IF B cur ≠ ∅THEN B cur Put into set B END IF RETURN B In the implementation of the above algorithm, an empty batch set B is first established. cur = ∅, then set the preset video memory index M max and preset batch size B maxThen, the sample dataset after descending order is enumerated, and the number of samples to be included in the current batch is determined by dynamically updating the estimated memory index. If the estimated memory index of the current batch exceeds M... max Or the batch size (sample size) exceeds B max Then, the samples in the current batch set will be combined into batch data and placed into batch dataset B, as an element b in batch dataset B. This process is repeated until all samples have been enumerated. Furthermore, the algorithm includes a fallback strategy, including the aforementioned "recommending the i-th data item {x} in the sorted sample dataset..." i , y i}Put into set B cur Update counter i = i + 1, END WHILE, IF B cur ≠ ∅THEN and B cur "Put them into set B", and pack the remaining samples into the last batch of data and add them to batch dataset B to avoid wasting training data.

[0051] Understandably, the initial recognition model can be trained on a single node of the server. The sample dataset is the total sample set. Multiple batches of data generated are randomly shuffled and then used to iteratively train the model until the training termination condition is met.

[0052] In other cases, the initial recognition model training method employs distributed training across multiple nodes. If the batch size of each training node is not fixed, the training speed of each node will be inconsistent, leading to a situation where the GPU is idle during gradient synchronization between nodes, wasting GPU resources and slowing down training time. Related technologies address this issue by using gradient accumulation, training multiple steps at each node before performing gradient synchronization. This reduces the number of iterations in a single synchronization, thus decreasing the GPU duty cycle. However, since current GPUs have large amounts of video memory, while the size of a speech recognition model is relatively small, the equivalent batch size after gradient accumulation becomes excessively large, meaning the number of samples requiring synchronization is too large. When the batch size exceeds a certain range, it can lead to poor convergence due to factors such as reduced training steps, affecting the performance of the speech recognition model. To optimize the distributed training method, this application optimizes distributed training by using a strategy of simultaneous and identical distribution of global training data, reducing the synchronization time required between training nodes while waiting for data and accelerating the training speed of the speech recognition model. Specifically, the model training utilizes multiple nodes running in a distributed system. Each node has a sample dataset and a batch dataset generated from the sample dataset, comprising multiple batches of data. Each node's sample dataset belongs to the total set of samples used to train the initial recognition model. In a preferred embodiment, the sample datasets of each node are distinct sets. Each node executes the aforementioned steps S201-S205 to generate its corresponding batch dataset. The multiple batches of data within the batch dataset are sorted according to the sample batch order, i.e., based on the length of the speech features within the batch data. In some embodiments, refer to... Figure 6 S205 may include S501-S505: S501: Get the batch count for each of the multiple nodes.

[0053] Specifically, the batch number indicates the number of batch data items contained in the batch dataset, that is, the number of elements in the batch dataset. For example, if the sample dataset of node 1 generates A1 batch data items, then the number of batch data items in the batch dataset is A1, and the batch number is A1. Understandably, the batch number may be the same or different for different nodes.

[0054] S503: Using the maximum number of batches corresponding to each node as the target quantity, perform repeated interpolation processing on the set elements of each node's batch dataset to expand the number of batches of each batch dataset to the target quantity, thus obtaining the target batch dataset for each node.

[0055] Specifically, the maximum number of batches for each node refers to the number of batches in the dataset with the largest number of batches among all nodes used in training. Using this value as the target, the number of batches in the batch datasets of each node is unified to the same length, thereby improving the training synchronization of each node.

[0056] In some embodiments, the batch data in the batch dataset are sorted based on the length of the speech feature, preferably in descending order, and the batch data in each batch dataset are based on sequential numbering. S503 may include S5031-S5033: S5031: For each batch of data sets with a quantity less than the target quantity, determine the duplication factor corresponding to the batch data set; S5032: Determine which batches of data in each batch of a batch dataset need to be repeated based on the repetition factor; S5033: Add the batch data that needs to be repeated as an interpolation term to the batch dataset to obtain the target batch dataset.

[0057] Specifically, batch datasets with a batch size less than the target size are identified in each node, and their repetition factors are calculated. The repetition factor characterizes the ratio of the batch size to the target size. This factor determines whether the sequence number of each batch in the batch dataset is a repeating interpolation term, mapping the original sequence length of the batch dataset to the target sequence length. The sequence length represents the batch size, and the target sequence length represents the target size. After the mapping operation, the sequences that need to be repeated in each batch are identified, and the batch data indexed by these repeating sequences are then determined as the batch data to be repeated. In this way, the repetition factor is determined by the ratio of the batch size to the target size for each batch dataset, adaptively determining the repeating interpolation terms in each batch dataset. This achieves uniformity in the length of each batch dataset while also ensuring uniform memory consumption for batch data at the same index position across all batch datasets.

[0058] In some embodiments, the batches of data in the batch dataset are sorted based on the length of the speech feature, and the sequence number of each batch of data forms a batch sequence number sequence. S5032 may include: 1) Obtain the target sequence number; 2) Scale and round each number in the target sequence based on the repetition factor to map each number in the target sequence to the index range of the batch sequence, thus obtaining the interpolated sequence. 3) The batch data indexed by the repeated serial numbers in the interpolation sequence is identified as the batch data that needs to be repeated.

[0059] Specifically, the target sequence number includes a target number of sequences and has the same indexing rules as the batch sequence number, that is, the starting sequence number of the two sequences is the same and the sequence number growth rules are the same. For example, if the batch sequence number uses a natural number sequence starting with 1, then the target sequence number is also a natural number sequence starting with 1. For example, the batch sequence number is {1,2,3,4…12} and the target sequence number is {1,2,3,4…15}.

[0060] Specifically, the scaling and rounding operation described above refers to scaling and rounding each number in the target sequence using a repetition factor as a coefficient, transforming each number into an index in the batch sequence, resulting in an interpolated sequence. The interpolated sequence corresponds to the batch data indexed by the same number in the batch sequence. The scaling method can be multiplying each number in the target sequence by the repetition factor, and the rounding method can be rounding to the nearest integer, rounding up, or rounding down, etc.

[0061] Next, the batch dataset is repeatedly interpolated using the index of the interpolation sequence. That is, each batch of data is arranged and repeated using the index of the interpolation sequence to obtain the target batch dataset. In essence, the sequence formed by the original indices of each batch of data in the target batch dataset is the interpolation sequence. Thus, by sorting the speech feature lengths and scaling and rounding the indices, the index ranges of the target sequence and each batch sequence are unified. This achieves repeated interpolation based on audio length sorting, improving the uniformity of the memory consumption distribution of the target batch dataset across nodes, thereby making the iteration time of each node during training similar and optimizing the consistency of gradient update timestamps.

[0062] For example, the repetition factor γ = L set k / L set max L set k L is the number of batches in the batch dataset of node k. set max For the target quantity, the index p = [j × γ] in the interpolation sequence, where j is the j-th index in the target sequence, and [] represents the rounding operation. In one example, the batch sequence is {1,2,3,4…12}, the target sequence is {1,2,3,4…15}, and the repetition factor γ = 12 / 15 = 0.8. Multiplying the indexes 1 to 15 in the target sequence by γ, the resulting interpolation sequence after rounding is {1,2,2,3,4,5,6,6,7,8,9,9,10,11,12}. The batch data that needs to be repeated are the batch data with indices 2, 6, and 9. The number of repetitions is 1 for each of them. In the generated target batch dataset, the batch data with indices 2, 6, and 9 are repeated twice.

[0063] In one embodiment, the algorithm implementation example for the target batch dataset is as follows: Given K training nodes Obtain the batch size of each node's batch dataset and form a vector L. set = [L set 1, L set 2, …,L set K ] Obtain the global target size (i.e., the maximum number of training batches) L set max = max(L set ) FOR k = 1 to K DO Create a new target batch dataset B with the kth node. knew = ∅ Get the batch dataset B of the kth node. k FOR j = 1 to L set max DO Calculate the index p of the interpolation sequence = [(j ÷ L) set max ) × L set k Note: [] indicates the floor function. B k The p-th element is added to set B. knew middle END FOR The batch dataset B used by the k-th node k Updated to B knew END FOR In the above algorithm implementation, the maximum batch size of all nodes is first obtained as the target size. Then, the batch size of all nodes is expanded to the target size, that is, the batch size is unified to the same length. This ensures that, under the same index, the batch data length obtained by each node is similar. The batch data set B of each node... k As derived from the algorithm above, the batches of data within the set are sorted in descending order of sample length. This method maintains a relatively consistent distribution of training sample data lengths across all nodes during distributed training, thereby ensuring that the time difference required for each node's batch dataset in each iteration is minimized, reducing GPU waiting time and achieving simultaneous and identical distribution of global training data.

[0064] S505: At each node, the initial recognition model is trained in a distributed iterative manner based on the target batch dataset until the training termination condition is met, thus obtaining the target speech recognition model.

[0065] Specifically, in each iteration, for each node, at least one batch of data is extracted from the node's target batch dataset as the training sample for the current iteration. The batch data in each target batch dataset is sorted according to the same indexing rule, and the sorting sequence number of the batch data extracted by each node is consistent. After generating the target batch dataset for each node, the batch data in the target batch dataset is sorted according to a unified indexing and sorting rule. The sorting rule can be based on speech feature length, and the indexing rule can be a natural number sequence number. Understandably, in the distributed synchronous training process, in each iteration, each node extracts batch data with consistent sequence numbers from its respective target batch dataset. The data length and memory consumption of batch data with the same sequence number (i.e., the same sequence position) in each target batch dataset are similar, ensuring that the time consumed after inputting the batch data into the initial recognition model is relatively small. After each node completes the same number of iterations of training sample processing, gradient synchronization of the node data and optimization and updating of the model parameters are performed. This process is repeated until all batch data in the target batch dataset is extracted and trained, completing one round of model training. Thus, by sorting based on speech feature length and unifying the batch data sequence number of each node, the time synchronization of iterative training of each node during training is improved, the waiting time for gradient synchronization is significantly reduced, and training efficiency is improved.

[0066] In some embodiments, during distributed iterative training, before each round of iterative training, the target batch dataset is randomly sorted to obtain the training sort for the current round, and the training sort is synchronized to each node; during the iterative training of the current round, each node sequentially extracts batch data from the target batch dataset corresponding to the node according to the training sort for iterative training.

[0067] Specifically, each node completes one training round by training on batch data in the target batch dataset. Before executing the current round, a training sorting sequence is obtained. This training sorting sequence is obtained by randomly sorting the indices of all batch data in the target batch dataset, and the order of the indices in the training sorting sequence is the training order. This training sorting sequence is synchronized to each node. Each node, based on the order in the random sorting sequence, sequentially extracts batch data with corresponding indices from its own target batch dataset for each iteration until the current round of training is completed. Then, the indices of all batch data in the target batch dataset are randomly sorted again to obtain the training sorting sequence for the next round, and the next round of training is executed. This process continues until the training termination condition is met, resulting in the target speech recognition model. In this way, during the model training process, all nodes use a consistent randomized sequence to extract batch data. This ensures the randomness of the sample length extracted in different iterations during training, while also ensuring the consistency of memory consumption of the samples extracted by each node in the same iteration. This improves training speed while maintaining training effectiveness, avoiding the problem of decreased randomness in training data after sorting and batching.

[0068] In one embodiment, all nodes use the same disordered vector D to index batch data from the target batch dataset. The disordered vector D represents the aforementioned training sorting sequence. A specific algorithm implementation example is as follows: Create from 0 to L set max A vector D consisting of natural numbers in ascending order Randomly shuffle the elements in vector D to obtain a disordered vector D. Synchronize the disordered vector D to all training nodes. Obtain the target batch dataset B at the k-th node. knew FOR h = 1 to L set max DO Get the h-th element d in the disordered vector D h (Serial number d) h ) Obtain the target batch dataset B knew The dth in h Batch data b Using batch data b for neural network forward computation and backpropagation Perform gradient synchronization with other nodes and update network parameters END FOR In summary, for distributed training, after the batch datasets for all training nodes are prepared, a repeated interpolation method is used to extend the batch size of each node to the same number to address the inconsistency in batch size between different nodes through inter-node communication. Batch synchronization is performed across all nodes, using the same randomized index (training order) for each round of training. Different rounds employ different randomized batch data extraction orders. This strategy ensures that all training nodes are using data of similar length during the same training time, resulting in minimal differences in gradient synchronization timestamps across all nodes. This reduces GPU duty cycle, improves training speed, and avoids the negative impact on model performance caused by decreased data randomness.

[0069] In one embodiment, the training method provided in this application can be applied to offline speech recognition algorithms in scenarios such as cloud intelligent voice assistant technology solutions. It can train the speech recognition model in the algorithm, improve the utilization of computing resources during model training, shorten training time, and facilitate faster iteration and updating of the model.

[0070] This application embodiment also provides a speech recognition model training device 600, such as... Figure 7 As shown, Figure 7 This illustration shows a structural diagram of a speech recognition model training device provided in an embodiment of this application. The device may include the following modules: Acquisition module 610: used to acquire sample datasets, which include multiple samples, including sample speech data and text labels corresponding to the sample speech data; Module 620: Used to determine the length of the speech features of each sample speech data and the length of the text label of each text label; Sample batching module 630: This module is used to batch multiple samples based on the sorting of speech feature length and the text label length to obtain multiple batches of data. The sample size of each batch is limited by a preset batch size and a preset memory index. The preset batch size is used to limit the upper limit of the sample size of the batch data. The preset memory index is the expected value that the batch memory index needs to reach. The batch memory index is used to indicate the memory consumption of the batch data during model training. It is determined based on the memory consumption prediction of the batch data based on the speech feature length and the text label length. Training module 640: Used to iteratively train the initial recognition model based on multiple batches of data to obtain the target speech recognition model.

[0071] In some embodiments, the sample batching module 630 may be specifically used for: Multiple samples are sequentially combined based on the descending order of speech feature length, and the sample size and estimated memory index of the combined samples are dynamically detected after each new sample is added. If the sample size meets the preset batch size limit or the estimated memory index meets the preset memory index limit, the samples of the current combination are divided into batch data. Starting with the next sample in descending order, the sequential combination and dynamic detection operations are repeatedly performed to divide the next batch of data until the sample data is divided into multiple batches.

[0072] In some embodiments, the sample batching module 630 may include: Create submodule: Used to create batch sets; Batch submodule: used to enumerate each sample in descending order based on the length of the speech features, so as to add each sample to the batch set in sequence; After adding a new sample to the batch set, the current sample size of the batch set is determined, and the estimated memory index of the batch set is determined based on the speech feature length and text feature length of each sample in the batch set. The estimated memory index is used to indicate the estimated memory consumption after the current batch set adds the next sample. Additionally, if the current sample size reaches the preset batch size, or the estimated memory index of the batch set reaches the preset memory index, the current batch data is formed based on each sample in the batch set.

[0073] In some embodiments, the batching submodule is also used for: If the current sample size does not reach the preset batch size, and the estimated memory index of the batch set does not reach the preset memory index, add the next sample in descending order among the multiple samples to the batch set. Repeat the steps of determining the sample size and estimated memory index of the batch set to obtain the updated sample size and updated estimated memory index. If the updated sample size reaches the preset batch size, or the updated estimated memory index reaches the preset memory index, the step of forming the current batch data based on each sample in the batch set is executed.

[0074] In some embodiments, the batching submodule may be specifically used for: Determine the maximum speech feature length and maximum text label length for each sample in the batch set; By fusing the maximum speech feature length and the maximum text label length, the memory coefficient of the batch set is obtained. The memory coefficient of the batch set is used to predict the memory consumption of a single sample in the batch set during model training. Using the memory coefficient of the batch set as the memory coefficient of a single sample in the batch set, the memory consumption after the batch set adds the next sample is estimated, and the estimated memory index of the batch set is obtained.

[0075] In some embodiments, the batching submodule may be specifically used for: For each sample in the current batch set and the next sample in the corresponding batch set, the speech feature length and text label length of the sample are fused to obtain the sample's memory coefficient. The sample's memory coefficient is used to indicate the degree of memory consumption of the sample during model training. By combining the memory coefficients of each sample in the batch set with the memory coefficient of the next sample, the estimated memory index of the batch set is obtained.

[0076] In some embodiments, the acquisition module 610 may be specifically used for: Obtain the initial sample set, which includes multiple initial samples; For each initial sample, the speech feature length of the sample speech data and the text label length of the text label in the initial sample are fused to obtain the memory coefficient of the initial sample. Remove initial samples from the initial sample set whose memory coefficients are higher than the preset memory index to obtain the sample dataset.

[0077] In some embodiments, the training method employs distributed training across multiple nodes. Each node is configured with a sample dataset and a batch dataset generated from the sample dataset, comprising multiple batch datasets. The sample datasets of each node belong to the total sample set used for training the initial recognition model. The training module 640 may include: Batch Count Acquisition Submodule: Used to obtain the batch count of each of multiple nodes. The batch count indicates the number of batch data contained in the batch dataset. Interpolation submodule: Used to perform repeated interpolation of set elements on the batch dataset of each node with the maximum batch size corresponding to each node as the target value, so as to expand the batch size of each batch dataset to the target value and obtain the target batch dataset of each node. Training submodule: Used for each node to perform distributed iterative training of the initial recognition model based on the target batch dataset until the training termination condition is met, and the target speech recognition model is obtained; In each iteration, for each node, at least one batch of data is extracted from the node's target batch dataset as the training sample for the current iteration. Each target batch dataset is sorted according to the same index rule, and the sorting sequence number of the batch data extracted by each node is consistent.

[0078] In some embodiments, the interpolation submodule is specifically used for: For each batch of data whose quantity is less than the target quantity, determine the repetition factor corresponding to the batch data. The repetition factor is used to characterize the ratio of the batch quantity to the target quantity. Based on the repetition factor, determine the batch data that needs to be repeated in each batch of the batch dataset; Add the batch data that needs to be repeated as an interpolation term to the batch dataset to obtain the target batch dataset.

[0079] In some embodiments, the batches of data in the batch dataset are sorted based on the length of the speech features, and the sequence numbers of each batch form a batch sequence number. The batches of data that need to be repeated in each batch of the batch dataset are determined based on the repetition factor, including: Obtain the target sequence number, which includes the target number of sequences and follows the same indexing rules as the batch sequence number. Based on the repetition factor, each number in the target sequence is scaled and rounded to map each number in the target sequence to the index range of the batch sequence, thus obtaining the interpolated sequence. The batch of data indexed by the repeated numbers in the interpolation sequence is determined as the batch of data that needs to be repeated.

[0080] In some embodiments, the training submodule is specifically used for: during the distributed iterative training process, before executing each round of iterative training, randomly sorting the batch data of the target batch dataset to obtain the training sort for the current round, and synchronizing the training sort to each node; during the iterative training process of the current round, each node sequentially extracts batch data from the target batch dataset corresponding to the node according to the training sort for iterative training.

[0081] It should be noted that the apparatus provided in the above embodiments is only illustrated by the division of the above functional modules when implementing its functions. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the apparatus and method embodiments provided in the above embodiments belong to the same concept, and the specific implementation process can be found in the method embodiments, which will not be repeated here.

[0082] This application provides a computer device including a processor and a memory. The memory stores at least one instruction or at least one program, which is loaded and executed by the processor to implement a training method for a speech recognition model as provided in the above method embodiments.

[0083] Figure 8 This diagram illustrates a hardware structure of an apparatus for implementing a training method for a speech recognition model provided in an embodiment of this application. The apparatus may constitute or include the device or system provided in the embodiment of this application. Figure 8As shown, device 10 may include one or more processors 1002 (shown as 1002a, 1002b, ..., 1002n in the figure) 1002 (processor 1002 may include, but is not limited to, a microprocessor MCU or a programmable logic device FPGA, etc.), a memory 1004 for storing data, and a transmission device 1006 for communication functions. In addition, it may also include: a display, an input / output interface (I / O interface), a universal serial bus (USB) port (which may be included as one of the ports of the I / O interface), a network interface, a power supply, and / or a camera. Those skilled in the art will understand that... Figure 8 The structure shown is for illustrative purposes only and does not limit the structure of the electronic device described above. For example, device 10 may also include a... Figure 8 The more or fewer components shown, or having the same Figure 8 The different configurations shown.

[0084] It should be noted that the aforementioned one or more processors 1002 and / or other data processing circuits are generally referred to herein as "data processing circuits". These data processing circuits may be embodied, in whole or in part, in software, hardware, firmware, or any other combination thereof. Furthermore, the data processing circuits may be a single, independent processing module, or may be wholly or partially integrated into any other element within device 10 (or mobile device). As involved in the embodiments of this application, the data processing circuits serve as a processor control mechanism (e.g., selection of a variable resistor termination path connected to an interface).

[0085] The memory 1004 can be used to store software programs and modules of application software, such as the program instructions / data storage device corresponding to the method in the embodiments of this application. The processor 1002 executes various functional applications and data processing by running the software programs and modules stored in the memory 1004, thereby realizing the above-described training method for a speech recognition model. The memory 1004 may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 1004 may further include memory remotely located relative to the processor 1002, and these remote memories can be connected to the device 10 via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0086] The transmission device 1006 is used to receive or send data via a network. Specific examples of the network described above may include a wireless network provided by the communication provider of device 10. In one example, the transmission device 1006 includes a Network Interface Controller (NIC), which can connect to other network devices via a base station to communicate with the Internet. In another example, the transmission device 1006 may be a Radio Frequency (RF) module, used for wireless communication with the Internet.

[0087] The display may be, for example, a touchscreen liquid crystal display (LCD) that allows the user to interact with the user interface of device 10 (or mobile device).

[0088] This application embodiment also provides a computer-readable storage medium, which can be disposed in a server to store at least one instruction or at least one program related to implementing a training method for a speech recognition model in the method embodiment. The at least one instruction or the at least one program is loaded and executed by the processor to implement the training method for a speech recognition model provided in the above method embodiment.

[0089] Optionally, in this embodiment, the storage medium may be located at at least one of the multiple network servers in a computer network. Optionally, in this embodiment, the storage medium may include, but is not limited to, various media capable of storing program code, such as USB flash drives, read-only memory (ROM), random access memory (RAM), portable hard drives, magnetic disks, or optical disks.

[0090] This invention also provides a computer program product or computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform a training method for a speech recognition model provided in the various optional embodiments described above.

[0091] It should be noted that the order of the embodiments described above is merely for descriptive purposes and does not represent the superiority or inferiority of the embodiments. Furthermore, the above description focuses on specific embodiments of this application. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps described in the claims can be performed in a different order than that shown in the embodiments and still achieve the desired results. Additionally, the processes depicted in the drawings do not necessarily require a specific or sequential order to achieve the desired results. In some implementations, multitasking and parallel processing are also possible or may be advantageous.

[0092] The various embodiments in this application are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the device, equipment, and storage medium embodiments are basically similar to the method embodiments, so the descriptions are relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0093] Those skilled in the art will understand that all or part of the steps of the above embodiments can be implemented by hardware or by a program instructing related hardware. The program can be stored in a computer-readable storage medium, such as a read-only memory, a disk, or an optical disk.

[0094] The above description is only a preferred embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A method for training a speech recognition model, characterized in that, The method includes: Obtain a sample dataset, which includes multiple samples, each sample containing sample speech data and corresponding text labels for the sample speech data; Determine the speech feature length of each sample speech data and the text label length of each text label; Based on the sorting of the speech feature lengths and the text label lengths, the multiple samples are divided into batches to obtain multiple batches of data. The sample size of each batch of data is limited by a preset batch size and a preset memory index. The preset batch size is used to limit the upper limit of the sample size of the batch of data. The preset memory index is the expected value that the batch memory index of the batch of data needs to reach. The batch memory index is used to indicate the degree of memory consumption of the batch of data during model training. It is determined based on the speech feature lengths and text label lengths to estimate the memory consumption of the batch of data. The initial recognition model is iteratively trained based on the multiple batches of data to obtain the target speech recognition model.

2. The method according to claim 1, characterized in that, The process of sorting the multiple samples based on the speech feature length and the text label length to obtain multiple batches of data includes: The multiple samples are sequentially combined based on the descending order of the speech feature length, and the sample size and estimated memory index of the combined samples are dynamically detected after each new sample is added. If the sample size meets the preset batch size limit or the estimated memory index meets the preset memory index limit, the samples of the current combination are divided into batch data. Starting with the next sample in the descending order, the sequential combination and dynamic detection operations are repeatedly performed to divide the next batch of data until the sample batching of each sample in the sample dataset is completed, resulting in the multiple batches of data.

3. The method according to claim 2, characterized in that, The process involves sequentially combining the multiple samples based on the descending order of the speech feature length, and dynamically detecting the sample size and estimated memory usage of the combined samples after each new sample is added. If the sample size meets the preset batch size limit or the estimated memory usage meets the preset memory usage limit, the samples in the current combination are divided into batches of data, including: Create batch sets; The samples are enumerated based on the descending order of the speech feature lengths, and each sample is added to the batch set in sequence. After a new sample is added to the batch set, the current sample size of the batch set is determined, and the estimated memory index of the batch set is determined based on the speech feature length and text feature length of each sample in the batch set; the estimated memory index is used to indicate the estimated memory consumption after the next sample is added to the current batch set. If the current sample size reaches the preset batch size, or the estimated memory index of the batch set reaches the preset memory index, the current batch data is formed based on each sample in the batch set.

4. The method according to claim 3, characterized in that, The method further includes: If the current sample size does not reach the preset batch size, and the estimated memory index of the batch set does not reach the preset memory index, the next sample in the descending order among the multiple samples is added to the batch set. Repeat the steps of determining the sample size and estimated memory index of the batch set to obtain the updated sample size and updated estimated memory index. If the updated sample size reaches the preset batch size, or the updated estimated memory index reaches the preset memory index, the step of forming the current batch data based on each sample in the batch set is executed.

5. The method according to claim 3, characterized in that, The method for determining the estimated memory metrics of the batch set includes: Determine the maximum speech feature length and the maximum text label length for each sample in the batch set; By fusing the maximum speech feature length and the maximum text label length, the memory coefficient of the batch set is obtained. The memory coefficient of the batch set is used to estimate the memory consumption of a single sample in the batch set during model training. Using the memory coefficient of the batch set as the memory coefficient of a single sample in the batch set, the memory consumption level after the batch set adds the next sample is estimated, and the estimated memory index of the batch set is obtained.

6. The method according to claim 3, characterized in that, The method for determining the estimated memory metrics of the batch set includes: For each sample in the current batch set and the next sample corresponding to the batch set, the speech feature length and text label length of the sample are fused to obtain the memory coefficient of the sample. The memory coefficient of the sample is used to indicate the memory consumption of the sample during the model training process. By combining the memory coefficients of each sample in the batch set and the memory coefficient of the next sample, the estimated memory index of the batch set is obtained.

7. The method according to any one of claims 1-6, characterized in that, The obtained sample dataset includes: Obtain an initial sample set, which includes multiple initial samples; For each initial sample, the speech feature length of the sample speech data and the text label length of the text label in the initial sample are fused to obtain the memory coefficient of the initial sample; Remove the initial samples in the initial sample set whose memory coefficient is higher than the preset memory index to obtain the sample dataset.

8. The method according to any one of claims 1-6, characterized in that, The training method employs distributed training across multiple nodes. Each node is configured with a sample dataset and a batch dataset generated from the sample dataset, which includes the multiple batch datasets. The sample datasets of each node belong to the total set of samples used to train the initial recognition model. The step of iteratively training the initial recognition model based on the multiple batches of data to obtain the target speech recognition model includes: Obtain the batch number for each of the plurality of nodes, wherein the batch number is used to indicate the number of batch data contained in the batch dataset; Using the maximum number of batches corresponding to each node as the target quantity, the batch datasets of each node are subjected to repeated interpolation of set elements to expand the number of batches of each batch dataset to the target quantity, thereby obtaining the target batch datasets of each node. At each node, the initial recognition model is subjected to distributed iterative training based on the target batch dataset until the training termination condition is met, thereby obtaining the target speech recognition model. In each iteration, for each node, at least one batch of data is extracted from the target batch dataset of the node as the training sample for the current iteration. Each target batch dataset is sorted according to the same indexing rule, and the sorting sequence number of the batch data extracted by each node is consistent.

9. The method according to claim 8, characterized in that, The step of using the maximum batch size corresponding to each node as the target quantity, and performing repeated interpolation processing on the batch datasets of each node to expand the batch size of each batch dataset to the target quantity, resulting in the target batch datasets for each node, includes: For each batch dataset whose batch size is less than the target size, a repetition factor is determined for the batch dataset. The repetition factor is used to characterize the ratio of the batch size to the target size. Based on the repetition factor, determine the batches of data that need to be repeated in each batch of the batch dataset; The batch data that needs to be repeated is added as an interpolation term to the batch dataset to obtain the target batch dataset.

10. The method according to claim 9, characterized in that, The batches of data in the dataset are sorted based on the length of the speech features, and the sequence numbers of each batch form a batch number sequence. The step of determining which batches of data in the dataset need to be repeated based on the repetition factor includes: Obtain the target sequence number, which includes a target number of sequences and follows the same indexing rules as the batch sequence number. Based on the repetition factor, each number in the target sequence is scaled and rounded to map each number in the target sequence to the index range of the batch sequence, thus obtaining the interpolated sequence. The batch of data indexed by the repeated serial numbers in the interpolation sequence is determined as the batch of data that needs to be repeated.

11. The method according to claim 8, characterized in that, During the distributed iterative training process, before each round of iterative training, the target batch dataset is randomly sorted to obtain the training sort for the current round, and the training sort is synchronized to each node. During the iterative training of the current round, each node sequentially extracts batch data from the target batch dataset corresponding to the node according to the training sort for iterative training.

12. A training device for a speech recognition model, characterized in that, The device includes: Acquisition module: used to acquire sample dataset, which includes multiple samples, each sample including sample speech data and text labels corresponding to the sample speech data; Determination module: used to determine the length of the speech features of each sample speech data and the length of the text label of each text label; Sample batching module: used to batch the multiple samples based on the sorting of the speech feature length and the text label length to obtain multiple batches of data. The sample size of each batch of data is limited by a preset batch size and a preset memory index. The preset batch size is used to limit the upper limit of the sample size of the batch of data. The preset memory index is the expected value that the batch memory index of the batch of data needs to reach. The batch memory index is used to indicate the memory consumption of the batch of data during model training. It is determined based on the speech feature length and the text label length to estimate the memory consumption of the batch of data. Training module: used to iteratively train the initial recognition model based on the multiple batches of data to obtain the target speech recognition model.

13. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores at least one instruction or at least one program, which is loaded and executed by a processor to implement the training method of the speech recognition model as described in any one of claims 1 to 11.

14. A computer device, characterized in that, The computer device includes a processor and a memory, the memory storing at least one instruction or at least one program, the at least one instruction or at least one program being loaded and executed by the processor to implement the training method of the speech recognition model as described in any one of claims 1 to 11.

15. A computer program product, characterized in that, The computer program product includes computer instructions that, when executed by a processor, implement the training method for the speech recognition model as described in any one of claims 1 to 11.

Images