Model training method, device, terminal device and computer-readable storage medium

Abstract

The present application is applicable to the field of data processing technology, and provides a model training method, apparatus, terminal device and computer-readable storage medium, including: training a preset model according to a first sample set that meets a first audio dynamic range to obtain the preset model after the first training; if the first model accuracy of the preset model after the first training on the validation set is less than the target accuracy, then expanding the first audio dynamic range to obtain a second audio dynamic range; continuing to train the preset model after the first training according to a second sample set that meets the second audio dynamic range until a target model that meets the target accuracy is obtained. Through the above method, the convergence effect of the deep learning model can be effectively guaranteed, and the training difficulty of the deep learning model can be reduced.

Description

Technical Field

[0001] The present application belongs to the field of data processing technology, and in particular, relates to a model training method, apparatus, terminal device and computer-readable storage medium. Background Art

[0002] With the development of deep learning, its application fields are becoming more and more extensive. For example, in the field of audio processing, deep learning models can be used for speech synthesis, that is, acoustic features are input into the deep learning model, and the corresponding synthesized speech is output. Before application, the deep learning model needs to be trained first.

[0003] In the existing model training methods, audio samples are usually randomly selected and then used to train the deep learning model. For some high-quality (high sampling rate) audio samples, their dynamic details are very rich and the impact on the model parameters of the deep learning model is relatively sensitive, which will make it difficult for the deep learning model to converge, thereby increasing the difficulty of training the deep learning model. Summary of the invention

[0004] The embodiments of the present application provide a model training method, apparatus, terminal device and computer-readable storage medium, which can effectively ensure the convergence effect of the deep learning model and reduce the training difficulty of the deep learning model.

[0005] In a first aspect, an embodiment of the present application provides a model method, including:

[0006] Training a preset model according to a first sample set that conforms to a first audio dynamic range to obtain the preset model after the first training;

[0007] If the first model accuracy of the preset model after the first training on the validation set is less than the target accuracy, expanding the first audio dynamic range to obtain a second audio dynamic range;

[0008] The preset model after the first training is continuously trained according to the second sample set that conforms to the second audio dynamic range until a target model that meets the target accuracy is obtained.

[0009] In an embodiment of the present application, the preset model is first trained based on a sample set with a smaller audio dynamic range, and then the preset model is further trained based on a sample set with a larger audio dynamic range. Audio samples with a higher sampling rate correspond to a larger audio dynamic range, which contains richer dynamic details, while audio samples with a smaller audio dynamic range are equivalent to blurring the dynamic details. Through the above method, it is equivalent to letting the preset model learn simpler features first, and then learn more complex features, thereby improving the problem of difficult model convergence caused by randomly selecting training samples, and effectively reducing the difficulty of training deep learning models.

[0010] In a possible implementation manner of the first aspect, training a preset model according to a first sample set that conforms to a first audio dynamic range to obtain the preset model after the first training includes:

[0011] Calculating a third audio dynamic range of audio samples in a preset audio library;

[0012] Adding audio samples in the preset audio library that meet a preset condition to the first sample set, wherein the preset condition is that the third audio dynamic range is smaller than the first audio dynamic range.

[0013] The preset model is trained according to the first sample set to obtain the preset model after the first training.

[0014] In a possible implementation manner of the first aspect, adding the audio samples in the preset audio library that meet the preset condition to the first sample set includes:

[0015] If the preset audio library does not contain the audio sample satisfying the preset condition, compressing the audio sample in the preset audio library into the audio sample satisfying the preset condition;

[0016] The audio samples satisfying the preset condition are added to the first sample set.

[0017] In a possible implementation manner of the first aspect, adding the audio samples in the preset audio library that meet the preset condition to the first sample set includes:

[0018] Acquire audio samples that meet the preset conditions from the preset audio library to obtain a first set;

[0019] Calculating a training value of each of the audio samples in the first set;

[0020] The audio samples in the first set whose training value is greater than a preset value are added to the first sample set.

[0021] In a possible implementation manner of the first aspect, calculating the training value of each of the audio samples in the first set includes:

[0022] Obtain a second model accuracy of the current preset model on the validation set;

[0023] The training value of each of the audio samples in the first set is calculated according to the second model accuracy and the third audio dynamic range of each of the audio samples in the first set.

[0024] In a possible implementation manner of the first aspect, the step of training the preset model according to the first sample set to obtain the preset model after the first training includes:

[0025] Extracting acoustic features of audio samples in the first sample set and actual values ​​of audio sampling points corresponding to the acoustic features;

[0026] Input the acoustic features into the preset model and output the predicted values ​​of the audio sampling points;

[0027] Calculating a first loss value of the preset model according to the actual value and the predicted value of the audio sampling point corresponding to the acoustic feature;

[0028] If the first loss value is greater than a first threshold, updating the model parameters of the preset model according to the first loss value;

[0029] If the first loss value is less than a first threshold, the current preset model is determined as the preset model after the first training.

[0030] In a possible implementation manner of the first aspect, after training a preset model according to a first sample set that conforms to a first audio dynamic range to obtain the preset model after a first training, the method further includes:

[0031] If the first model accuracy of the preset model after the first training on the validation set is equal to or greater than the target accuracy, the preset model after the first training is determined as the target model.

[0032] In a second aspect, an embodiment of the present application provides a model device, including:

[0033] A first training unit, configured to train a preset model according to a first sample set that conforms to a first audio dynamic range, to obtain the preset model after a first training;

[0034] a range expansion unit, configured to expand the first audio dynamic range to obtain a second audio dynamic range if the first model accuracy of the preset model after the first training on the validation set is less than the target accuracy;

[0035] The second training unit is used to continue training the preset model after the first training according to a second sample set that conforms to the second audio dynamic range until a target model that meets the target accuracy is obtained.

[0036] In a third aspect, an embodiment of the present application provides a terminal device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements a model method as described in any one of the first aspects above when executing the computer program.

[0037] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium. An embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and wherein the computer program, when executed by a processor, implements a model method as described in any one of the first aspects above.

[0038] In a fifth aspect, an embodiment of the present application provides a computer program product, which, when executed on a terminal device, enables the terminal device to execute the model method described in any one of the above-mentioned first aspects.

[0039] It can be understood that the beneficial effects of the second to fifth aspects mentioned above can be found in the relevant description of the first aspect mentioned above, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0041] Figure 1 It is a flowchart of the model training method provided in the embodiment of the present application;

[0042] Figure 2 It is a schematic diagram of the training process provided in the embodiment of the present application;

[0043] Figure 3 It is a structural schematic diagram of a model training device provided in an embodiment of the present application;

[0044] Figure 4 It is a schematic diagram of the structure of the terminal device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0045] In the following description, specific details such as specific system structures, technologies, etc. are provided for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present application. However, it should be clear to those skilled in the art that the present application may also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to prevent unnecessary details from obstructing the description of the present application.

[0046] It should be understood that when used in the present specification and the appended claims, the term "comprising" indicates the presence of described features, wholes, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or combinations thereof.

[0047] It should also be understood that the term “and / or” used in the specification and appended claims refers to any and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0048] As used in the specification and appended claims of this application, the term "if" can be interpreted as "when" or "uponce" or "in response to determining" or "in response to detecting", depending on the context. Similarly, the phrase "if it is determined" or "if [described condition or event] is detected" can be interpreted as meaning "uponce it is determined" or "in response to determining" or "uponce [described condition or event] is detected" or "in response to detecting [described condition or event]", depending on the context.

[0049] In addition, in the description of the present application specification and the appended claims, the terms "first", "second", "third", etc. are only used to distinguish the descriptions and cannot be understood as indicating or implying relative importance.

[0050] References to "one embodiment" or "some embodiments" etc. described in the specification of this application mean that one or more embodiments of the present application include specific features, structures or characteristics described in conjunction with the embodiment. Therefore, the phrases "in one embodiment", "in some embodiments", "in some other embodiments", "in some other embodiments", etc. appearing in different places in this specification do not necessarily refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways.

[0051] The input of the vocoder is the acoustic features, and the output is the corresponding synthesized speech. The relationship between acoustic features and speech is a complex nonlinear mapping. Therefore, the neural network vocoder has become one of the mainstream vocoder technologies due to its powerful nonlinear function fitting ability. The neural network vocoder can obtain more natural synthesized speech. Commonly used neural network models include recurrent neural network (RNN), gated recurrent unit (GRU) network and long short-term memory (LSTM) network.

[0052] In the existing neural network vocoder training method, audio samples are collected in advance to generate data pairs of acoustic features and audio, and then the network model is trained based on loss functions such as cross entropy. The training method is to randomly select audio samples from the training data set (preset voice library) and adjust the network weights according to the loss. Since high-quality (such as sampling rate greater than 16k) audio has very rich dynamic details, and common models such as RNN are mainly based on recursive structures, a small weight adjustment may cause a huge change in the overall model. Therefore, in actual situations, it will be difficult for the neural network vocoder to converge, which increases the difficulty of model training.

[0053] In order to solve the above problems, the present application embodiment provides a model training method. Figure 1 , is a flow chart of the model method provided in the embodiment of the present application. As an example but not a limitation, the method may include the following steps:

[0054] S101, training a preset model according to a first sample set that meets a first audio dynamic range, to obtain the preset model after the first training.

[0055] The dynamic range is the ratio of the maximum value to the minimum value of a variable signal (such as sound or light). It can be expressed as a logarithm (decibel) with a base of 10 or a logarithm with a base of 2. The audio dynamic range in the embodiment of the present application refers to the ratio of the maximum value to the minimum value of the audio. The first audio dynamic range can be an audio dynamic range preset based on experience.

[0056] In one embodiment, S101 may include:

[0057] S1011: Calculate a third audio dynamic range of audio samples in a preset audio library.

[0058] The audio sample in the embodiment of the present application can be an audio segment, or an audio sampling point sequence obtained by sampling the audio segment. Specifically, the audio sample can be obtained according to the formula Calculate the third audio dynamic range of the audio sample. Wherein, z is the audio sampling point sequence corresponding to the audio sample.

[0059] S1012: Add audio samples in the preset audio library that meet preset conditions to the first sample set.

[0060] The preset condition is that the third audio dynamic range is smaller than the first audio dynamic range. Specifically, the third audio dynamic range of the audio samples in the preset audio library is compared with the first audio dynamic range; if the third audio dynamic range is smaller than the first audio dynamic range, the audio samples corresponding to the third audio dynamic range are added to the first sample set.

[0061] As can be seen from the above, in actual applications, there may be the following situation: there is no audio sample that meets the preset conditions in the preset audio library. To solve this problem, optionally, S1012 also includes:

[0062] If there is no audio sample satisfying the preset condition in the preset audio library, compress the audio samples in the preset audio library into audio samples satisfying the preset condition; and add the audio samples satisfying the preset condition to the first sample set.

[0063] Compressing audio samples, for example, reducing 16-bit audio data to 8 bits, or more broadly speaking, discretizing the amplitude of the sampled signal, i.e., quantizing. Commonly used quantization methods include uniform quantization and non-uniform quantization. Among them, uniform quantization refers to quantizing small signals or large signals within the coding range with equal quantization levels, so that the "signal to quantization noise ratio" of small signals is small, while the "signal to quantization noise ratio" of large signals is large, which is unfavorable for small signals. In order to improve the signal-to-noise ratio of small signals, the quantization levels can be further subdivided, and the signal-to-noise ratio of large signals is also improved. However, the result of doing so is that the digital rate is also increased, which will require a wider bandwidth channel for transmission. The problems existing in uniform quantization can be solved by non-uniform quantization. The basic idea is to compress large signals and amplify small signals to a greater extent. Since the amplitude of small signals is amplified to a greater extent (that is, more quantization levels are provided for small signals and fewer quantization levels are provided for large signals), the signal-to-noise ratio of small signals is greatly improved.

[0064] In the embodiment of the present application, the purpose of compressing the audio sample is to reduce the amplitude of the large signal and amplify the amplitude of the small signal, so as to reduce the data accuracy of the audio sample and blur the dynamic detail characteristics of the audio sample. Therefore, a non-uniform quantization method is optionally used. Existing non-uniform quantization methods include A-law compression and MU-law compression.

[0065] Preferably, the MU-law compression method is used, which can improve the signal-to-noise ratio without adding more data. The principle of this method is: Among them, t_in is the quantizer input, and t_out is the quantizer output. The larger the constant μ is, the higher the compression and expansion efficiency of small signals is.

[0066] In the embodiment of the present application, if there are audio samples meeting the preset conditions in the preset audio library, multiple audio samples meeting the preset conditions can be randomly selected and added to the first sample set.

[0067] In order to ensure the model training effect, it is optional to select audio samples based on their training value, including:

[0068] The audio samples satisfying the preset conditions are obtained from the preset audio library to obtain a first set; the training value of each of the audio samples in the first set is calculated; and the audio samples in the first set whose training value is greater than a preset value are added to the first sample set.

[0069] Furthermore, the calculation method of training value includes:

[0070] Obtain a second model accuracy of the current preset model on the validation set; and calculate the training value of each of the audio samples in the first set based on the second model accuracy and the third audio dynamic range of each of the audio samples in the first set.

[0071] Specifically, the formula Calculate the training value of the audio sample. Where x is the audio sample, p learn x is the probability of selecting sample x for model training (i.e., training value), Z is the preset normalization coefficient, R(x) is the audio dynamic range of sample x, and Loss(Val|M) is the performance evaluation of the current preset model M on the validation set Val (such as loss value, etc.). is a random distribution in the interval [-ε,ε], where ε is a value close to 0 based on experience. The purpose of is to serve as a compensation mechanism for possible limitations of the probability calculation model. For a sample x, if p learn x If it is greater than a preset value, the sample x is added to the first sample set.

[0072] Through the above method, audio samples are selected according to training value, which can avoid the problem of difficulty in training due to random selection, further reduce the difficulty of model training, and enhance the model training effect.

[0073] S1013: Train the preset model according to the first sample set to obtain the preset model after the first training.

[0074] Optionally, the training process in S1013 includes:

[0075] Extracting acoustic features of audio samples in the first sample set and actual values ​​of audio sampling points corresponding to the acoustic features; inputting the acoustic features into the preset model and outputting predicted values ​​of the audio sampling points; calculating a first loss value of the preset model according to the actual values ​​and the predicted values ​​of the audio sampling points corresponding to the acoustic features; if the first loss value is greater than a first threshold, updating the model parameters of the preset model according to the first loss value; if the first loss value is less than a first threshold, determining the current preset model as the preset model after the first training.

[0076] Optionally, the acoustic feature may be a Mel-frequency inverse spectrum. The preset model may be an RNN model, a GRU model, an LSTM model, etc. The loss function used to calculate the loss value may be a mean square error function or a cross entropy function, etc. The method for updating the model parameters using the loss value may be a steepest descent method or a gradient method, etc. The specific model training process is not specifically limited here.

[0077] The first threshold in this step refers to the threshold of the training accuracy that can be achieved by training the preset model based on the first sample set. As mentioned above, the sample set corresponding to a larger audio dynamic range has more quantization levels of large signals and more detailed features of audio samples. Accordingly, the model can learn some more complex feature relationships; the sample set corresponding to a smaller audio dynamic range has fewer quantization levels of large signals and fewer detailed features of audio samples. Accordingly, the model can only learn some simpler feature relationships. Therefore, different training accuracies are obtained by training the preset model with sample sets corresponding to different audio dynamic ranges. The larger the audio dynamic range, the higher the training accuracy of the preset model after training the corresponding sample set.

[0078] S102: If the first model accuracy of the preset model after the first training on the validation set is less than the target accuracy, the first audio dynamic range is expanded to obtain a second audio dynamic range.

[0079] If the first model accuracy of the preset model after the first training on the validation set is equal to or greater than the target accuracy, the preset model after the first training is determined as the target model.

[0080] The embodiment of the present application adopts a training mode of sample set training and verification set verification.

[0081] The increment of each expansion of the audio dynamic range can be pre-set as needed. The larger the increment, the larger the learning span of the model for the feature, and the model training times are relatively small; the smaller the increment, the smaller the learning span of the model for the feature, the model training times are relatively large, and the model is easier to converge.

[0082] Exemplarily, the first audio dynamic range is 2 8 , the second audio dynamic range is 2 10 .

[0083] S103: Continue training the preset model after the first training according to the second sample set that meets the second audio dynamic range, until a target model that meets the target accuracy is obtained.

[0084] The specific step of training the preset model according to the second sample set is the same as S101. For details, please refer to the description in S101, which will not be repeated here.

[0085] If the model accuracy of the second trained preset model obtained by continuing to train the preset model after the first training according to the second sample set on the validation set is still less than the target accuracy, repeat S102-S103 until the model accuracy of the trained preset model on the validation set meets the target accuracy.

[0086] In an embodiment of the present application, the preset model is first trained based on a sample set with a smaller audio dynamic range, and then the preset model is further trained based on a sample set with a larger audio dynamic range. Audio samples with a higher sampling rate correspond to a larger audio dynamic range, which contains richer dynamic details, while audio samples with a smaller audio dynamic range are equivalent to blurring the dynamic details. Through the above method, it is equivalent to letting the preset model learn simpler features first, and then learn more complex features, thereby improving the problem of difficult model convergence caused by randomly selecting training samples, and effectively reducing the difficulty of training deep learning models.

[0087] In addition, in the embodiment of the present application, the difficulty of model learning features is gradually increased, and when the model accuracy reaches the target accuracy, the training is stopped. Through the above method, when the target accuracy is low, the model can reduce the learning of some detail features, thereby reducing the difficulty of model training and improving the training efficiency of the model.

[0088] For example, see Figure 2 , is a schematic diagram of the training process provided in the embodiment of the present application. Figure 2 As shown, the training process includes the following steps:

[0089] 1. Get the results of the model (i.e. the preset model) on the validation set.

[0090] 2. Adjust the threshold of the dynamic range (i.e., the audio dynamic range) according to the result in step 1.

[0091] 3. According to the threshold value of the dynamic range, audio (ie, audio sample) that meets the threshold value of the dynamic range is selected from the speech data set (ie, the preset speech library).

[0092] Use these audios to train the model, as shown in steps 4-8.

[0093] 4. Extract the acoustic features of the audio.

[0094] 5. Resample the acoustic features to align them with the audio sampling points.

[0095] 6. Input the acoustic features into the model to obtain the predicted values ​​of the audio sampling points.

[0096] 7. Calculate the loss value of the model based on the actual value and predicted value of the audio sampling point.

[0097] 8. Calculate the gradient based on the loss value and adjust the model parameters (hidden layer node weights) of the model based on the gradient.

[0098] If the stopping condition is currently met, the current model is output; if the stopping condition is not met, return to step 1 and continue executing until the stopping condition is met.

[0099] The stopping condition in the embodiment of the present application may be that the number of iterations reaches a preset number, or the loss value of the model on the validation set is less than a second threshold (ie, the model accuracy reaches the target accuracy).

[0100] It should be understood that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0101] Corresponding to the model method described in the above embodiment, Figure 3 This is a structural block diagram of the model device provided in an embodiment of the present application. For the sake of convenience of explanation, only the parts related to the embodiment of the present application are shown.

[0102] Reference Figure 3 , the device comprises:

[0103] The first training unit 31 is used to train the preset model according to a first sample set that conforms to a first audio dynamic range, to obtain the preset model after the first training.

[0104] The range expansion unit 32 is used to expand the first audio dynamic range to obtain a second audio dynamic range if the first model accuracy of the preset model after the first training on the verification set is less than the target accuracy.

[0105] The second training unit 33 is used to continue training the preset model after the first training according to the second sample set that meets the second audio dynamic range, until a target model that meets the target accuracy is obtained.

[0106] Optionally, the first training unit 31 is further used for:

[0107] Calculating a third audio dynamic range of audio samples in a preset audio library;

[0108] Adding audio samples in the preset audio library that meet a preset condition to the first sample set, wherein the preset condition is that the third audio dynamic range is smaller than the first audio dynamic range.

[0109] The preset model is trained according to the first sample set to obtain the preset model shown after the first training.

[0110] Optionally, the first training unit 31 is further used for:

[0111] If there is no audio sample satisfying the preset condition in the preset audio library, compress the audio samples in the preset audio library into audio samples satisfying the preset condition; and add the audio samples satisfying the preset condition to the first sample set.

[0112] Optionally, the first training unit 31 is further used for:

[0113] Acquire audio samples that meet the preset conditions from the preset audio library to obtain a first set;

[0114] Calculating a training value of each of the audio samples in the first set;

[0115] The audio samples in the first set whose training value is greater than a preset value are added to the first sample set.

[0116] Optionally, the first training unit 31 is further used for:

[0117] Obtain a second model accuracy of the current preset model on the validation set;

[0118] The training value of each of the audio samples in the first set is calculated according to the second model accuracy and the third audio dynamic range of each of the audio samples in the first set.

[0119] Optionally, the first training unit 31 is further used for:

[0120] Extracting acoustic features of audio samples in the first sample set and actual values ​​of audio sampling points corresponding to the acoustic features;

[0121] Input the acoustic features into the preset model and output the predicted values ​​of the audio sampling points;

[0122] Calculating a first loss value of the preset model according to the actual value and the predicted value of the audio sampling point corresponding to the acoustic feature;

[0123] If the first loss value is greater than a first threshold, updating the model parameters of the preset model according to the first loss value;

[0124] If the first loss value is less than a first threshold, the current preset model is determined as the preset model after the first training.

[0125] Optionally, the device 3 further comprises:

[0126] The model determination unit 34 is used to determine the preset model after the first training as the target model if the first model accuracy of the preset model after the first training on the validation set is equal to or greater than the target accuracy.

[0127] It should be noted that the information interaction, execution process, etc. between the above-mentioned devices / units are based on the same concept as the method embodiment of the present application. Their specific functions and technical effects can be found in the method embodiment part and will not be repeated here.

[0128] in addition, Figure 3 The model device shown may be a software unit, a hardware unit, or a combination of software and hardware that is built into an existing terminal device, or may be integrated into the terminal device as an independent accessory, or may exist as an independent terminal device.

[0129] The technicians in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In practical applications, the above-mentioned function allocation can be completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated in a processing unit, or each unit can exist physically separately, or two or more units can be integrated in one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, which will not be repeated here.

[0130] Figure 4 Schematic diagram of the structure of the terminal device provided in the embodiment of the present application. Figure 4 As shown, the terminal device 4 of this embodiment includes: at least one processor 40 ( Figure 4 Only one is shown in the figure) a processor, a memory 41, and a computer program 42 stored in the memory 41 and executable on the at least one processor 40, wherein the processor 40 implements the steps in any of the above-mentioned model method embodiments when executing the computer program 42.

[0131] The terminal device may be a computing device such as a desktop computer, a notebook, a PDA, or a cloud server. The terminal device may include, but is not limited to, a processor and a memory. Those skilled in the art will understand that Figure 4 It is only an example of the terminal device 4 and does not constitute a limitation on the terminal device 4. It may include more or fewer components than shown in the figure, or a combination of certain components, or different components. For example, it may also include input and output devices, network access devices, etc.

[0132] The processor 40 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

[0133] In some embodiments, the memory 41 may be an internal storage unit of the terminal device 4, such as a hard disk or memory of the terminal device 4. In other embodiments, the memory 41 may also be an external storage device of the terminal device 4, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. equipped on the terminal device 4. Further, the memory 41 may also include both an internal storage unit and an external storage device of the terminal device 4. The memory 41 is used to store an operating system, an application program, a boot loader (BootLoader), data, and other programs, such as the program code of the computer program, etc. The memory 41 may also be used to temporarily store data that has been output or is to be output.

[0134] An embodiment of the present application further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps in the above-mentioned method embodiments can be implemented.

[0135] An embodiment of the present application provides a computer program product. When the computer program product is run on a terminal device, the terminal device can implement the steps in the above-mentioned method embodiments when executing the computer program product.

[0136] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the processes in the above-mentioned embodiment method, which can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may at least include: any entity or device that can carry the computer program code to the device / terminal device, a recording medium, a computer memory, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), an electric carrier signal, a telecommunication signal, and a software distribution medium. For example, a USB flash drive, a mobile hard disk, a magnetic disk or an optical disk. In some jurisdictions, according to legislation and patent practice, computer-readable media cannot be electric carrier signals and telecommunication signals.

[0137] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0138] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0139] In the embodiments provided in the present application, it should be understood that the disclosed devices / terminal equipment and methods can be implemented in other ways. For example, the device / terminal equipment embodiments described above are only schematic. For example, the division of the modules or units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0140] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0141] The embodiments described above are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, a person skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.

Claims

1. A model training method, characterized in that: include: Training a preset model according to a first sample set that conforms to a first audio dynamic range to obtain the preset model after the first training; If the first model accuracy of the preset model after the first training on the validation set is less than the target accuracy, expanding the first audio dynamic range to obtain a second audio dynamic range; The preset model after the first training is continuously trained according to the second sample set that conforms to the second audio dynamic range until a target model that meets the target accuracy is obtained.

2. The model training method according to claim 1, characterized in that: The step of training a preset model according to a first sample set that conforms to a first audio dynamic range to obtain the preset model after the first training includes: Calculating a third audio dynamic range of audio samples in a preset audio library; adding audio samples in the preset audio library that meet a preset condition to the first sample set, wherein the preset condition is that the third audio dynamic range is smaller than the first audio dynamic range; The preset model is trained according to the first sample set to obtain the preset model after the first training.

3. The model training method according to claim 2, characterized in that: The step of adding the audio samples in the preset audio library that meet the preset conditions to the first sample set includes: If the preset audio library does not contain the audio sample satisfying the preset condition, compressing the audio sample in the preset audio library into the audio sample satisfying the preset condition; The audio samples satisfying the preset condition are added to the first sample set.

4. The model training method according to claim 2, characterized in that: The step of adding the audio samples in the preset audio library that meet the preset conditions to the first sample set includes: Acquire audio samples that meet the preset conditions from the preset audio library to obtain a first set; Calculating the training value of each of the audio samples in the first set; wherein the training value is used to characterize the degree of influence of the audio sample on the training effect of the model; The audio samples in the first set whose training value is greater than a preset value are added to the first sample set.

5. The model training method according to claim 4, characterized in that: The calculating the training value of each of the audio samples in the first set includes: Obtain a second model accuracy of the current preset model on the validation set; The training value of each of the audio samples in the first set is calculated according to the second model accuracy and the third audio dynamic range of each of the audio samples in the first set.

6. The model training method according to claim 2, characterized in that: The step of training the preset model according to the first sample set to obtain the preset model after the first training includes: Extracting acoustic features of audio samples in the first sample set and actual values ​​of audio sampling points corresponding to the acoustic features; Input the acoustic features into the preset model and output the predicted values ​​of the audio sampling points; Calculating a first loss value of the preset model according to the actual value and the predicted value of the audio sampling point corresponding to the acoustic feature; If the first loss value is greater than a first threshold, updating the model parameters of the preset model according to the first loss value; If the first loss value is less than the first threshold, the current preset model is determined as the preset model after the first training.

7. The model training method according to claim 1, characterized in that: After training the preset model according to the first sample set that conforms to the first audio dynamic range to obtain the preset model after the first training, the method further includes: If the first model accuracy of the preset model after the first training on the validation set is equal to or greater than the target accuracy, the preset model after the first training is determined as the target model.

8. A model training device, characterized in that: include: A first training unit, configured to train a preset model according to a first sample set that conforms to a first audio dynamic range, to obtain the preset model after a first training; a range expansion unit, configured to expand the first audio dynamic range to obtain a second audio dynamic range if the first model accuracy of the preset model after the first training on the validation set is less than the target accuracy; The second training unit is used to continue training the preset model after the first training according to a second sample set that conforms to the second audio dynamic range until a target model that meets the target accuracy is obtained.

9. A terminal device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the method according to any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images