Voice classification model optimization training method and device, computer equipment and medium

Abstract

The application relates to the technical field of artificial intelligence, in particular to a speech classification model optimization training method and device, computer equipment and a medium. The method obtains a speech sample and a sampling frequency, determines a frequency range according to the sampling frequency, constructs a filter function according to first and second cutoff frequencies determined in the frequency range, obtains a time domain function through inverse Fourier transform, inputs a classification model after convolution calculation of the speech sample and the time domain function, obtains an output result, trains the classification model and the time domain function according to the output result and the category of the speech sample, constructs the filter function with the first and second cutoff frequencies, and then performs convolution with the time domain function corresponding to the filter function, so that the trained time domain convolution function can more easily capture information of a target frequency band, and the trained classification model can be provided with more accurate speech features, thereby improving the accuracy of speech classification model classification.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and in particular to an optimized training method, apparatus, computer equipment, and medium for a speech classification model. Background Technology

[0002] Currently, with the development of artificial intelligence technology, the synthesized speech obtained through speech synthesis technology is becoming increasingly realistic and has acoustic features similar to real speech. This allows users to receive more realistic voice feedback during voice interaction, thus improving the user experience.

[0003] However, synthetic speech technology also brings serious security risks. Therefore, it is necessary to classify the speech to be detected to determine whether it belongs to the category of synthetic speech, thereby ensuring the security of voice communication. Existing synthetic speech detection methods usually use manually designed features for speech feature extraction. When the speech features belong to known synthetic speech features, it can effectively detect whether the speech belongs to the synthetic speech category. However, when the speech features are deployed under known synthetic speech features, manually designed features may lead to insufficient extraction of speech feature information, resulting in low classification accuracy of synthetic speech classification models. Therefore, how to improve the classification accuracy of synthetic speech classification models has become an urgent problem to be solved. Summary of the Invention

[0004] In view of this, embodiments of the present invention provide an optimized training method, apparatus, computer equipment, and medium for a speech classification model to solve the problem of low classification accuracy of speech classification models.

[0005] In a first aspect, embodiments of the present invention provide an optimization training method for a speech classification model, the optimization training method comprising:

[0006] Acquire speech samples of known categories and their corresponding sampling frequencies, and use the interval formed by a preset multiple and a preset value of the sampling frequency as the frequency range, wherein the preset value is greater than or equal to zero and less than the preset multiple of the sampling frequency;

[0007] Two frequency values ​​are randomly selected within the frequency range, and the larger of the two frequency values ​​is taken as the first cutoff frequency, and the smaller of the two frequency values ​​is taken as the second cutoff frequency.

[0008] Based on the first cutoff frequency and the second cutoff frequency, a rectangular bandpass filter function is constructed, and the rectangular bandpass filter function is transformed into a time-domain function through inverse Fourier transform;

[0009] The speech samples and the time-domain function are convolved, and the convolution result is input into a preset classification model to obtain the output result of the classification model.

[0010] Based on the output of the classification model and the category corresponding to the speech sample, the cross-entropy loss is calculated. Using the cross-entropy loss as a basis, the first cutoff frequency, the second cutoff frequency, and the model parameters in the classification model are adjusted by gradient descent until the cross-entropy loss converges, thus obtaining the trained temporal function and the trained classification model.

[0011] Secondly, embodiments of the present invention provide an optimization training device for a speech classification model, the optimization training device comprising:

[0012] The range determination module is used to acquire speech samples of known categories and their corresponding sampling frequencies, and to take the interval formed by a preset multiple of the sampling frequency and a preset value as the frequency range, wherein the preset value is greater than or equal to zero and less than the preset multiple of the sampling frequency.

[0013] The random sampling module is used to randomly sample two frequency values ​​within the frequency range, using the larger of the two frequency values ​​as the first cutoff frequency and the smaller of the two frequency values ​​as the second cutoff frequency.

[0014] The function transformation module is used to construct a rectangular bandpass filter function based on the first cutoff frequency and the second cutoff frequency, and transform the rectangular bandpass filter function into a time-domain function through inverse Fourier transform;

[0015] The sample classification module is used to perform convolution calculation on the speech samples and the time domain function, and input the convolution calculation result into a preset classification model to obtain the output result of the classification model;

[0016] The model training module is used to calculate the cross-entropy loss based on the output of the classification model and the category corresponding to the speech sample. Based on the cross-entropy loss, the first cutoff frequency, the second cutoff frequency, and the model parameters in the classification model are adjusted using the gradient descent method until the cross-entropy loss converges, thereby obtaining the trained temporal function and the trained classification model.

[0017] Thirdly, embodiments of the present invention provide a computer device, the computer device including a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the optimization training method for the speech classification model as described in the first aspect.

[0018] Fourthly, embodiments of the present invention provide a computer-readable storage medium storing a computer program that, when executed by a processor, implements the optimization training method for the speech classification model as described in the first aspect.

[0019] The beneficial effects of the embodiments of the present invention compared with the prior art are as follows:

[0020] Acquire speech samples of known categories and their corresponding sampling frequencies. Define the frequency range as the interval formed by a preset multiple and a preset value of the sampling frequency. Randomly select two frequency values ​​within this range, using the larger of the two values ​​as the first cutoff frequency and the smaller as the second cutoff frequency. Construct a rectangular bandpass filter function based on the first and second cutoff frequencies. Transform the rectangular bandpass filter function into a time-domain function using an inverse Fourier transform. Perform convolution calculations on the speech samples and the time-domain function, and input the convolution result into a preset classification model to obtain the output of the classification model. Then, based on the output of the classification model and the speech samples... For each category, the cross-entropy loss is calculated. Based on the cross-entropy loss, the first cutoff frequency, the second cutoff frequency, and the model parameters in the classification model are adjusted using gradient descent until the cross-entropy loss converges, resulting in a trained temporal function and a trained classification model. A rectangular bandpass filter function is constructed using the first and second cutoff frequencies. Then, the corresponding temporal function of the rectangular bandpass filter function is convolved. This allows the trained temporal convolution function to act as a bandpass filter for speech samples, making it easier to capture information from the target frequency band and providing more accurate speech features for the subsequently trained classification model, thereby improving the accuracy of the speech classification model. Attached Figure Description

[0021] To more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0022] Figure 1 This is a schematic diagram of an application environment for an optimized training method for a speech classification model provided in Embodiment 1 of the present invention;

[0023] Figure 2 This is a flowchart illustrating an optimized training method for a speech classification model provided in Embodiment 1 of the present invention.

[0024] Figure 3 This is a flowchart illustrating an optimized training method for a speech classification model provided in Embodiment 2 of the present invention.

[0025] Figure 4 This is a schematic diagram of the structure of an optimized training device for a speech classification model provided in Embodiment 3 of the present invention;

[0026] Figure 5 This is a schematic diagram of the structure of a computer device provided in Embodiment 4 of the present invention. Detailed Implementation

[0027] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of the invention. However, those skilled in the art will understand that the invention can be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods are omitted so as not to obscure the description of the invention with unnecessary detail.

[0028] It should be understood that, when used in this specification and the appended claims, the term "comprising" indicates the presence of the described features, integrals, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or collections thereof.

[0029] It should also be understood that the term “and / or” as used in this specification and the appended claims refers to any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0030] As used in this specification and the appended claims, the term "if" may be interpreted, depending on the context, as "when," "once," "in response to determination," or "in response to detection." Similarly, the phrase "if determined" or "if [described condition or event] is detected" may be interpreted, depending on the context, as meaning "once determined," "in response to determination," "once [described condition or event] is detected," or "in response to detection of [described condition or event]."

[0031] Furthermore, in the description of this invention and the appended claims, the terms "first," "second," "third," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0032] References to "one embodiment" or "some embodiments" as described in this specification mean that one or more embodiments of the invention include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.

[0033] The embodiments of this invention can acquire and process relevant data based on artificial intelligence technology. Artificial intelligence (AI) refers to the theories, methods, technologies, and application systems that utilize digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to obtain optimal results.

[0034] Foundational technologies for artificial intelligence generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing, operating / interactive systems, and mechatronics. AI software technologies mainly encompass computer vision, robotics, biometrics, speech processing, natural language processing, and machine learning / deep learning.

[0035] It should be understood that the sequence number of each step in the following embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0036] To illustrate the technical solution of the present invention, specific embodiments are described below.

[0037] The first embodiment of this invention provides an optimized training method for a speech classification model, which can be applied to applications such as... Figure 1 In this application environment, the client and server communicate with each other. Clients include, but are not limited to, PDAs, desktop computers, laptops, ultra-mobile personal computers (UMPCs), netbooks, cloud terminal devices, and personal digital assistants (PDAs). The server can be implemented using a standalone server or a server cluster consisting of multiple servers.

[0038] See Figure 2This is a flowchart illustrating an optimization training method for a speech classification model provided in Embodiment 1 of the present invention. The above optimization training method can be applied to... Figure 1 The client-side computer connects to the server to obtain speech samples, their categories, and sampling frequencies. The client-side computer has a built-in classification model architecture that can be used to classify the speech samples. Figure 2 As shown, the optimized training method may include the following steps:

[0039] Step S201: Obtain speech samples of known categories and their corresponding sampling frequencies, and use the interval formed by the preset multiple and preset value of the sampling frequency as the frequency range.

[0040] The category can be determined according to the application scenario of the classification model. In this embodiment, the application scenario of the classification model is a synthetic speech detection scenario, so the category can include real speech category and synthetic speech category, etc. The speech sample can refer to the speech signal of known category. The sampling frequency can refer to the sampling interval time when the speech sample is collected. It can be used to characterize the quality of the speech sample. The higher the sampling frequency, the higher the quality of the speech sample. The preset value is a value that is greater than or equal to zero and less than a preset multiple of the sampling frequency.

[0041] Specifically, according to the Nyquist theory, a digital signal can be restored to the original signal only when the sampling frequency is more than twice the highest frequency of the original signal. Therefore, when the sampling frequency is known, the maximum value of the speech signal should be less than half of the sampling frequency. Thus, the preset multiple is set to 0.5. The implementer can adjust the value of the preset multiple within [0, 0.5] according to the actual situation.

[0042] At the same time, the minimum value of the voice signal should be greater than or equal to zero; therefore, the range of the preset value is set to be... Among them, f s It can represent the sampling frequency.

[0043] Using a preset multiple of the sampling frequency as the maximum value of the interval and a preset value as the minimum value of the interval, we obtain the interval [α, βf]. s The interval is defined as the frequency range, where α can represent a preset value, β can represent a preset multiple, and the value range of α can be [0, βf]. s ), where the value of β can be in the range of [0, 0.5].

[0044] The steps described above—obtaining known categories of speech samples and their corresponding sampling frequencies, and using the interval formed by a preset multiple and a preset value of the sampling frequency as the frequency range—determine the frequency range based on the sampling frequency and Nyquist theory. This ensures the reproducibility of the speech samples, avoids errors in subsequent speech sample classification due to incorrect reconstruction of the speech samples, provides a more accurate input for the classification model, and thus improves the classification accuracy of the classification model.

[0045] Step S202: Randomly select two frequency values ​​within the frequency range, and use the larger of the two frequency values ​​as the first cutoff frequency and the smaller of the two frequency values ​​as the second cutoff frequency.

[0046] Here, frequency value can refer to a value within a frequency range, random sampling can refer to sampling all values ​​within the frequency range with equal probability, the first cutoff frequency can refer to a relatively high cutoff frequency, and the second cutoff frequency can refer to a relatively low cutoff frequency.

[0047] Optionally, two frequency values ​​are randomly selected within the frequency range, with the larger of the two values ​​used as the first cutoff frequency and the smaller of the two values ​​used as the second cutoff frequency, including:

[0048] According to the preset frequency sub-interval length, K sampling points are determined from the frequency range, where K is an integer greater than zero;

[0049] Assign the same sampling probability to each sampling point to obtain the first sampling probability distribution. Then, perform initial sampling from the K sampling points according to the first sampling probability distribution to obtain the first random sample value.

[0050] The preset value is determined as the first boundary, and the preset multiple of the sampling frequency is determined as the second boundary. The distance between the first random sample value and the first and second boundaries is calculated respectively. The boundary corresponding to the larger value is determined as the target boundary, and the smaller value is determined as the target difference.

[0051] If the target boundary is the first boundary, add the target boundary value to the target difference value to obtain the target frequency; if the target boundary is the second boundary, subtract the target boundary value from the target difference value to obtain the target frequency.

[0052] A Gaussian distribution is constructed with the target frequency as the mean and the target difference as the variance. The Gaussian distribution is then multiplied by the first sampling probability distribution to obtain the second sampling probability distribution.

[0053] Based on the second sampling probability distribution, resampling is performed from K sampling points to obtain the second random sampling value. The first random sampling value and the second random sampling value are then determined as frequency values.

[0054] Here, the frequency sub-interval length can refer to the frequency difference between adjacent sampling points, the sampling point can refer to the point after the frequency range is discretized, the first random sampling value can refer to the frequency value corresponding to the sampling point obtained in the first sampling, and the second random sampling value can refer to the frequency value corresponding to the sampling point obtained in the second sampling.

[0055] Specifically, the length of the frequency sub-interval is L. In this embodiment, the frequency range is... L is set to Based on the length of the frequency sub-interval, 51 sampling points can be determined within the frequency range, including the maximum and minimum points of the frequency range, with each sampling point corresponding to a frequency value.

[0056] This embodiment discretizes the frequency range before sampling and adjusts the sampling probability distribution during subsequent sampling based on the initial sampling results, thereby avoiding overly similar sampling results and improving the feasibility and efficiency of sampling.

[0057] Optionally, after obtaining the first random sample value and the second random sample value, the method further includes:

[0058] The first random sample value and the second random sample value are mapped to the first Mel frequency and the second Mel frequency using preset mapping coefficients;

[0059] The first random sample value is updated at the first Mel frequency, and the second random sample value is updated at the second Mel frequency.

[0060] The mapping coefficients include the mapping relationship between the Mel frequency and the results of random sampling. The Mel frequency can refer to the frequency unit that conforms to human hearing. The first Mel frequency can refer to the Mel frequency mapped to the first random sample value through the mapping coefficients, and the second Mel frequency can refer to the Mel frequency mapped to the second random sample value through the mapping coefficients.

[0061] Specifically, the mapping coefficients are expressed as Where f can represent a random sample value, and m can represent the Mel frequency corresponding to the random sample value.

[0062] This embodiment uses Mel frequency to update random sample values, thereby extracting effective information in accordance with human hearing, improving the convergence speed of subsequent training, and thus improving training efficiency.

[0063] The steps described above, which involve randomly selecting two frequency values ​​within a frequency range, using the larger of the two values ​​as the first cutoff frequency and the smaller of the two values ​​as the second cutoff frequency, initialize the first and second cutoff frequencies. This process improves the speed of gradient descent of the loss function during subsequent training and aims to make the loss function converge to the global optimum, thus ensuring the efficiency and accuracy of subsequent training.

[0064] Step S203: Based on the first cutoff frequency and the second cutoff frequency, construct a rectangular bandpass filter function, and transform the rectangular bandpass filter function into a time-domain function through inverse Fourier transform.

[0065] The rectangular bandpass filter function can refer to an ideal bandpass filter function, which allows signals in a certain frequency band to pass through while suppressing signals, interference, and noise below or above that frequency band. The inverse Fourier transform can be used to transform the rectangular bandpass filter function in the frequency domain into the corresponding time domain function, which can refer to the function of speech samples changing over time.

[0066] Specifically, the rectangular bandpass filter function G can be expressed as:

[0067]

[0068] Where f can represent the frequency of the speech sample, f1 can represent the first cutoff frequency, f2 can represent the second cutoff frequency, and rect can refer to a rectangular function. Since a rectangular function can be used to characterize a low-pass filter, a band-pass filter can be represented as the difference between two low-pass filters.

[0069] The time-domain function g can be expressed as:

[0070] g[n,f1,f2]=2f1sinc(2πf1n)-2f2sinc(2πf2n)

[0071] Where n can represent the sampling time point, and the formula for calculating the sinc function is: x can refer to the variable in the sinc function.

[0072] Optionally, the rectangular bandpass filter function is constructed based on the first cutoff frequency and the second cutoff frequency, including:

[0073] Calculate the first ratio of the preset frequency variable to the first cutoff frequency, and take half of the first ratio as the first parameter of the preset rectangular function to obtain the first rectangular function;

[0074] Calculate the second ratio of the frequency variable to the second cutoff frequency, and use half of the second ratio as the second parameter of the rectangular function to obtain the second rectangular function;

[0075] Subtracting the first rectangular function from the second rectangular function yields the rectangular bandpass filter function.

[0076] The first rectangular function can represent a low-pass filter, the second rectangular function can represent another low-pass filter, and the difference between the two low-pass filters is the band-pass filter.

[0077] Specifically, the first rectangular function is represented as follows: The second rectangular function is expressed as follows: Here, f can refer to a frequency variable, which is variable data determined based on the signal received by the filter.

[0078] This embodiment constructs a rectangular bandpass filter by subtracting a lowpass filter, which improves the efficiency of bandpass filter construction.

[0079] Optionally, after transforming the rectangular bandpass filter function into a time-domain function using inverse Fourier transform, the following steps are also included:

[0080] Multiply the time-domain function by the preset window function to obtain the windowed time-domain function;

[0081] Accordingly, the convolution calculation of speech samples and time-domain functions includes:

[0082] The speech samples and the windowed time-domain function are convolved to obtain the convolution result.

[0083] Among them, window function can refer to Hamming window, flat-top window, Kaiser window, etc. Window function can be used to make time domain signal better meet periodic requirements and reduce frequency leakage. Windowed time domain function can refer to time domain function after windowing processing.

[0084] Specifically, in this embodiment, a Hamming window is used as the window function, and the formula for the Hamming window w[n] is as follows:

[0085]

[0086] Where 0≤n≤N, N is the length of the Hamming window, which is also the length of the time-domain function.

[0087] The windowing process can be represented as g w [n,f1,f2]=g[n,f1,f2]·w[n], where, g w [n,f1,f2] can refer to the windowed time-domain function.

[0088] This embodiment applies a windowing process to the time-domain function, thereby enabling the time-domain signal to better meet the periodicity requirement and reduce frequency leakage.

[0089] The steps described above involve constructing a rectangular bandpass filter function based on the first and second cutoff frequencies, and then transforming the rectangular bandpass filter function into a time-domain function using inverse Fourier transform. The bandpass function obtained through inverse Fourier transform facilitates subsequent convolution with speech samples to extract effective features from the speech samples, thereby improving the classification accuracy of the speech classification model.

[0090] Step S204: Perform convolution calculation on the speech samples and the time domain function, and input the convolution calculation result into the preset classification model to obtain the output result of the classification model.

[0091] Here, speech samples can refer to the time-domain representation of speech signals. Using time-domain representation is beneficial for extracting the time-domain features of speech samples. Convolution can refer to the convolution calculation of functions. Classification models can refer to logistic regression models, neural network models, decision tree models, etc. The output of the classification model can refer to the predicted category of the corresponding speech sample.

[0092] Specifically, the convolution calculation formula can be expressed as:

[0093] y[n] = x[n] * g[n,f1,f2]

[0094] Where n can refer to the sampling time point, y[n] can refer to the convolution calculation result, x[n] can refer to the speech sample, and g[n,f1,f2] can refer to the time domain function.

[0095] In this embodiment, the only learnable parameters during the convolution process are the first cutoff frequency f1 and the second cutoff frequency f2. The values ​​of the first cutoff frequency f1 and the second cutoff frequency f2 determine the filtering frequency band of the filter. Therefore, the above convolution method is beneficial to improve the attention to the influence of filter parameters on speech classification performance, thereby enabling the training process of the time domain function to converge faster and be more efficient. Moreover, compared with conventional filters, fewer parameters need to be learned. Conventional filters need to determine the number of parameters to be learned based on their length, and the number of parameters is usually greater than 2.

[0096] In addition, due to the symmetry of the time-domain function, only half of the time-domain function needs to be considered during convolution calculation, which reduces the computational cost. Furthermore, the time-domain function convolution process is interpretable and can extract speech features such as pitch and formants, while conventional filters tend to focus on low-frequency information. Therefore, the time-domain function convolution process is more likely to capture narrowband information.

[0097] Optionally, the classification model includes a feature extraction layer and a fully connected layer;

[0098] Inputting the convolution calculation results into a pre-defined classification model includes:

[0099] The convolution calculation result is input into the feature extraction layer for feature extraction, and the feature extraction result is obtained.

[0100] The convolution calculation result is multiplied with the feature extraction result, and the multiplication result is input into the fully connected layer for feature mapping to obtain the output of the detection model.

[0101] The feature extraction layer can include 1*1 convolution and 1*3 convolution. 1*1 convolution can be used to reduce the number of computational parameters, and 1*3 convolution can be used to extract high-dimensional features.

[0102] Specifically, multiplying the convolution calculation result with the feature extraction result can be used to ensure the feature extraction effect and avoid the degradation of the feature extraction effect due to excessively high feature extraction layers.

[0103] This embodiment improves the computational efficiency of feature extraction by designing the architecture of the feature extraction layer, while ensuring good feature extraction results.

[0104] The steps described above, which involve convolving speech samples with a temporal function and inputting the convolution result into a pre-defined classification model to obtain the output of the classification model, can simplify the computational load of the convolution process, improve the training efficiency of the temporal function, and extract more accurate speech features, thereby improving the classification accuracy of the subsequent classification model.

[0105] Step S205: Calculate the cross-entropy loss based on the output of the classification model and the category corresponding to the speech sample. Using the cross-entropy loss as a basis, adjust the first cutoff frequency, the second cutoff frequency, and the model parameters in the classification model using the gradient descent method until the cross-entropy loss converges, thus obtaining the trained temporal function and the trained classification model.

[0106] Here, cross-entropy loss can refer to binary cross-entropy loss, multi-class cross-entropy loss, etc., and gradient descent method can refer to stochastic gradient descent method, batch gradient descent method, etc. In this embodiment, the classification model adopts a neural network model, and the model parameters can refer to the neuron parameters in the classification model.

[0107] Specifically, in this embodiment, the cross-entropy loss adopts the binary cross-entropy loss, and the specific formula for the binary cross-entropy loss is as follows:

[0108] Loss=-(z·log(z′)+(1-z)·log(1-z′))

[0109] Here, Loss can refer to the binary cross-entropy loss, z can refer to the category label corresponding to the speech sample. When the speech sample is a synthesized speech category, z takes the value of 1, and when the speech sample is a real speech category, z takes the value of 0. z′ can refer to the probability that the speech sample output by the classification model belongs to the synthesized speech category, and the value range of z′ is [0, 1].

[0110] When the speech sample is a synthesized speech, the value of z is 1, and the binary cross-entropy loss is expressed as Loss = -log(z′). At this time, the closer z′ is to 0, the lower the probability that the speech sample output by the classification model belongs to the synthesized speech category, and the larger the value of the binary cross-entropy loss. The closer z′ is to 1, the higher the probability that the speech sample output by the classification model belongs to the synthesized speech category, and the smaller the value of the binary cross-entropy loss.

[0111] When the speech sample is a real speech category, the value of z is 0, and the binary cross-entropy loss is expressed as Loss = -log(1-z′). At this time, the closer z′ is to 0, the lower the probability that the speech sample output by the classification model belongs to the synthetic speech category, and the smaller the value of the binary cross-entropy loss. The closer z′ is to 1, the higher the probability that the speech sample output by the classification model belongs to the synthetic speech category, and the larger the value of the binary cross-entropy loss.

[0112] The above steps involve calculating the cross-entropy loss based on the output of the classification model and the corresponding category of the speech sample. Using the cross-entropy loss as a basis, gradient descent is used to adjust the first cutoff frequency, the second cutoff frequency, and the model parameters in the classification model until the cross-entropy loss converges, thus obtaining the trained temporal function and the trained classification model. The first cutoff frequency, the second cutoff frequency, and the model parameters in the classification model are then trained to obtain the trained temporal function and the trained classification model. The trained temporal function can obtain more accurate speech features, and the trained classification model can classify speech features more accurately, thereby improving the classification accuracy of the classification model.

[0113] In this embodiment, a rectangular bandpass filter function is constructed using the first cutoff frequency and the second cutoff frequency. Then, the corresponding time-domain function of the rectangular bandpass filter function is used for convolution. This enables the trained time-domain convolution function to achieve a bandpass filtering effect on speech samples, making it easier to capture information of the target frequency band and providing more accurate speech features for the subsequently trained classification model, thereby improving the classification accuracy of the speech classification model.

[0114] See Figure 3 This is a flowchart illustrating an optimization training method for a speech classification model provided in Embodiment 2 of the present invention. In this optimization training method, when adjusting the first cutoff frequency, the second cutoff frequency, and the model parameters in the classification model using the gradient descent method, the training can be performed by combining the time-domain function with the classification model, or by separating the time-domain function from the classification model. Combining can mean that the parameters of the time-domain function and the model parameters in the classification model are adjusted synchronously, while separating can mean that the parameters of the time-domain function and the model parameters in the classification model are adjusted asynchronously. The parameters of the time-domain function can refer to the first cutoff frequency and the second cutoff frequency.

[0115] When training by combining the time-domain function with the classification model, the training process can be found in Example 1, and will not be repeated here.

[0116] When training by separating the time-domain function from the classification model, the training process includes:

[0117] Step S301: Fix the values ​​of the model parameters, and use the gradient descent method to adjust the values ​​of the first cutoff frequency and the second cutoff frequency until the cross-entropy loss converges, and obtain the first converged value.

[0118] Step S302: After obtaining the first convergence value, fix the values ​​of the first cutoff frequency and the second cutoff frequency, and use the gradient descent method to adjust the values ​​of the model parameters until the cross-entropy loss converges, thus obtaining the second convergence value.

[0119] Step S303: When the second convergence value does not meet the preset conditions, return to the step of fixing the values ​​of the model parameters and adjusting the values ​​of the first and second cutoff frequencies using the gradient descent method until the second convergence value meets the preset conditions.

[0120] The first convergence value can refer to the convergence value when the cross-entropy loss converges for the first time, and the second convergence value can refer to the convergence value when the cross-entropy loss converges for the second time.

[0121] Specifically, first, the values ​​of the model parameters are fixed, that is, the values ​​of the model parameters are kept unchanged, and only the first cutoff frequency and the second cutoff frequency are adjusted. The partial derivatives of the cross-entropy loss with respect to the first cutoff frequency and the second cutoff frequency are calculated according to the gradient descent method. Then, the first adjustment direction of the first cutoff frequency and the second cutoff frequency is determined according to the partial derivatives. The values ​​of the first cutoff frequency and the second cutoff frequency are adjusted according to the first adjustment direction.

[0122] If adjusting the values ​​of the first and second cutoff frequencies fails to reduce the cross-entropy loss, it indicates that the cross-entropy loss has converged. Obtain the value of the cross-entropy loss at this point; this value is the first convergence value.

[0123] After obtaining the first convergence value, it indicates that the values ​​of the first and second cutoff frequencies are the optimal solutions under the current circumstances. At this point, the values ​​of the first and second cutoff frequencies are fixed, that is, the values ​​of the first and second cutoff frequencies are kept unchanged, and only the model parameters are adjusted. The partial derivatives of the cross-entropy loss with respect to the model parameters are calculated according to the gradient descent method, and the second adjustment direction of the model parameters is determined according to the partial derivatives. The values ​​of the model parameters are then adjusted according to the second adjustment direction.

[0124] When adjusting the model parameters fails to reduce the cross-entropy loss, it indicates that the cross-entropy loss has converged. Obtain the value of the cross-entropy loss at this point; this value is the second convergence value.

[0125] The second convergence value is compared with a preset convergence threshold, for example, the convergence threshold can be set to 5. When the second convergence value is greater than or equal to the convergence threshold, it means that the output of the classification model has not achieved the expected effect. It is determined that the second convergence value does not meet the preset condition, and the values ​​of the fixed model parameters are returned. The steps of adjusting the values ​​of the first and second cutoff frequencies using the gradient descent method are performed to retrain the time domain function and the classification model. When the second convergence value is less than the convergence threshold, it means that the output of the classification model is now relatively accurate. It is determined that the second convergence value meets the preset condition, and training is stopped.

[0126] In this embodiment, the parameters of the time-domain function and the model parameters in the classification model are adjusted asynchronously during the training process. This avoids the situation where the loss function is difficult to converge when multiple parameters are adjusted at the same time, improves the training efficiency of the training process, and makes it easier to adjust the parameters to the optimal rather than the local optimum, thereby improving the classification accuracy of the trained classification model.

[0127] Corresponding to the optimized training method of the speech classification model in the above embodiment, Figure 4 A structural block diagram of the speech classification model optimization training device provided in Embodiment 3 of the present invention is shown. This optimization training device is applied to a client, and the computer device corresponding to the client is connected to the server to obtain speech samples, the category of the corresponding speech samples, and the sampling frequency of the corresponding speech samples from the server. The computer device corresponding to the client has a classification model architecture deployed within it, and the classification model can be used to classify speech samples. For ease of explanation, only the parts relevant to the embodiments of the present invention are shown.

[0128] See Figure 4 The optimization training apparatus for this speech classification model includes:

[0129] The range determination module 41 is used to obtain speech samples of known categories and their corresponding sampling frequencies, and to take the interval formed by a preset multiple of the sampling frequency and a preset value as the frequency range. The preset value is greater than or equal to zero and less than a preset multiple of the sampling frequency.

[0130] The random sampling module 42 is used to randomly sample two frequency values ​​within a frequency range, using the larger of the two frequency values ​​as the first cutoff frequency and the smaller of the two frequency values ​​as the second cutoff frequency.

[0131] The function transformation module 43 is used to construct a rectangular bandpass filter function based on the first cutoff frequency and the second cutoff frequency, and transform the rectangular bandpass filter function into a time domain function through inverse Fourier transform;

[0132] The sample classification module 44 is used to perform convolution calculation on speech samples and time-domain functions, and input the convolution calculation result into a preset classification model to obtain the output result of the classification model;

[0133] The model training module 45 is used to calculate the cross-entropy loss based on the output of the classification model and the category corresponding to the speech sample. Based on the cross-entropy loss, the gradient descent method is used to adjust the first cutoff frequency, the second cutoff frequency and the model parameters in the classification model until the cross-entropy loss converges, thus obtaining the trained time-domain function and the trained classification model.

[0134] Optionally, the above-mentioned random sampling module 42 includes:

[0135] The sampling point determination unit is used to determine K sampling points from the frequency range according to the preset frequency sub-interval length, where K is an integer greater than zero;

[0136] The sampling unit is used to perform two random samplings from K sampling points to obtain the first random sampling value and the second random sampling value.

[0137] The numerical comparison unit is used to compare the first random sample value with the second random sample value. If the first random sample value is greater than the second random sample value, the first random sample value is determined to be the first cutoff frequency and the second random sample value is determined to be the second cutoff frequency.

[0138] The frequency determination unit is used to determine the second random sampling value as the first cutoff frequency and the first random sampling value as the second cutoff frequency if the first random sampling value is less than the second random sampling value.

[0139] The resampling unit is used to perform two random samplings in the K sampling points respectively if the first random sampling value is equal to the second random sampling value, update the first random sampling value and the second random sampling value, and perform a numerical comparison between the first random sampling value and the second random sampling value.

[0140] Optionally, the random sampling module 42 mentioned above also includes:

[0141] The frequency mapping unit is used to map the first random sample value and the second random sample value to the first Mel frequency and the second Mel frequency using preset mapping coefficients. The mapping coefficients include the mapping relationship between the Mel frequency and the random sampling result.

[0142] The sampling value update unit is used to update the first random sampling value at a first Mel frequency and update the second random sampling value at a second Mel frequency.

[0143] Optionally, the above-mentioned function transformation module 43 includes:

[0144] The first function determination unit is used to calculate the first ratio of the preset frequency variable to the first cutoff frequency, and take half of the first ratio as the first parameter of the preset rectangular function to obtain the first rectangular function.

[0145] The second function determination unit is used to calculate the second ratio of the frequency variable to the second cutoff frequency, and half of the second ratio is used as the second parameter of the rectangular function to obtain the second rectangular function.

[0146] The function subtraction unit is used to subtract the first rectangular function from the second rectangular function to obtain the rectangular bandpass filter function.

[0147] Optionally, the optimization training device for the above speech classification model also includes:

[0148] The function windowing module is used to multiply the time-domain function with a preset window function to obtain the windowed time-domain function;

[0149] Accordingly, the above sample classification module 44 includes:

[0150] The windowed convolution unit is used to perform convolution calculations on speech samples and windowed time-domain functions to obtain the convolution result.

[0151] Optionally, the classification model includes a feature extraction layer and a fully connected layer;

[0152] The above sample classification module 44 includes:

[0153] The feature extraction unit is used to input the convolution calculation results into the feature extraction layer for feature extraction, and obtain the feature extraction results.

[0154] The feature mapping unit is used to multiply the convolution calculation result with the feature extraction result, and input the multiplication result into the fully connected layer for feature mapping to obtain the output of the detection model.

[0155] Optionally, the above model training module 45 includes:

[0156] The first adjustment unit is used to fix the values ​​of the model parameters and adjust the values ​​of the first and second cutoff frequencies using the gradient descent method until the cross-entropy loss converges, thus obtaining the first convergent value.

[0157] The second adjustment unit is used to fix the values ​​of the first cutoff frequency and the second cutoff frequency after obtaining the first convergence value, and adjust the values ​​of the model parameters using the gradient descent method until the cross-entropy loss converges, thus obtaining the second convergence value.

[0158] The condition judgment unit is used to return the steps of executing the fixed model parameter values ​​and adjusting the values ​​of the first and second cutoff frequencies using the gradient descent method when the second convergence value does not meet the preset conditions, until the second convergence value meets the preset conditions.

[0159] It should be noted that the information interaction and execution process between the above modules and units are based on the same concept as the method embodiments of the present invention. For details on their specific functions and technical effects, please refer to the method embodiments section, which will not be repeated here.

[0160] Figure 5 This is a schematic diagram of the structure of a computer device provided in Embodiment 4 of the present invention. Figure 5 As shown, the computer device of this embodiment includes: at least one processor ( Figure 5 Only one is shown in the diagram), a memory, and a computer program stored in the memory and capable of running on at least one processor, which, when executing the computer program, implements the steps in the embodiments of the optimization training methods for any of the above-described speech classification models.

[0161] This computer device may include, but is not limited to, a processor and memory. Those skilled in the art will understand that... Figure 5 The examples of computer devices are merely examples and do not constitute a limitation on computer devices. Computer devices may include more or fewer components than shown in the illustration, or combinations of certain components, or different components, such as network interfaces, displays, and input devices.

[0162] The processor referred to can be a CPU, but it can also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor can be a microprocessor or any conventional processor.

[0163] Memory includes readable storage media, internal memory, etc., wherein internal memory can be the RAM of a computer device, providing an environment for the operation of the operating system and computer-readable instructions stored in the readable storage media. The readable storage media can be the hard drive of a computer device, or in other embodiments, it can be an external storage device of the computer device, such as a plug-in hard drive, Smart Media Card (SMC), Secure Digital (SD) card, or Flash Card. Furthermore, memory can include both internal storage units and external storage devices of a computer device. Memory is used to store the operating system, applications, bootloader, data, and other programs, such as program code for computer programs. Memory can also be used to temporarily store data that has been output or will be output.

[0164] Those skilled in the art will understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is used as an example. In practical applications, the functions described above can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this invention. The specific working process of the units and modules in the above device can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here. If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present invention can implement all or part of the processes in the methods of the above embodiments by instructing related hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the above method embodiments. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. A computer-readable medium can include at least: any entity or device capable of carrying computer program code, a recording medium, a computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media. Examples include USB flash drives, portable hard drives, magnetic disks, or optical disks. In some jurisdictions, according to legislation and patent practice, computer-readable media cannot be electrical carrier signals or telecommunication signals.

[0165] The present invention can implement all or part of the processes in the methods of the above embodiments, or it can be accomplished by a computer program product. When the computer program product is run on a computer device, the computer device executes the steps in the above method embodiments.

[0166] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0167] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0168] In the embodiments provided by this invention, it should be understood that the disclosed apparatus / computer devices and methods can be implemented in other ways. For example, the apparatus / computer device embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0169] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0170] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included within the protection scope of the present invention.

Claims

1. An optimized training method for a speech classification model, characterized in that, The optimized training method includes: Acquire speech samples of known categories and their corresponding sampling frequencies, and use the interval formed by a preset multiple of the sampling frequency and a preset value as the frequency range, wherein the preset value is greater than or equal to zero and less than the preset multiple of the sampling frequency; Two frequency values ​​are randomly selected within the frequency range, and the larger of the two frequency values ​​is taken as the first cutoff frequency, and the smaller of the two frequency values ​​is taken as the second cutoff frequency. Based on the first cutoff frequency and the second cutoff frequency, a rectangular bandpass filter function is constructed, and the rectangular bandpass filter function is transformed into a time-domain function through inverse Fourier transform; The speech samples and the time-domain function are convolved, and the convolution result is input into a preset classification model to obtain the output result of the classification model. Based on the output of the classification model and the category corresponding to the speech sample, the cross-entropy loss is calculated. Based on the cross-entropy loss, the first cutoff frequency, the second cutoff frequency and the model parameters in the classification model are adjusted by gradient descent until the cross-entropy loss converges, thus obtaining the trained temporal function and the trained classification model. The step of randomly selecting two frequency values ​​within the frequency range, using the larger of the two frequency values ​​as the first cutoff frequency, and using the smaller of the two frequency values ​​as the second cutoff frequency includes: According to the preset frequency sub-interval length, K sampling points are determined from the frequency range, where K is an integer greater than zero; Assign the same sampling probability to each sampling point to obtain a first sampling probability distribution. Then, perform an initial sampling from the K sampling points according to the first sampling probability distribution to obtain a first random sampling value. The preset value is determined as the first boundary, the preset multiple of the sampling frequency is determined as the second boundary, the distance between the first random sampled value and the first boundary and the second boundary are calculated respectively, the boundary corresponding to the larger value is determined as the target boundary, and the smaller value is determined as the target difference. If the target boundary is the first boundary, add the target boundary to the target difference to obtain the target frequency; if the target boundary is the second boundary, subtract the target boundary from the target difference to obtain the target frequency. A Gaussian distribution is constructed using the target frequency as the mean and the target difference as the variance. The Gaussian distribution is then multiplied by the first sampling probability distribution to obtain a second sampling probability distribution. Based on the second sampling probability distribution, resampling is performed from the K sampling points to obtain a second random sampling value, and the first random sampling value and the second random sampling value are determined to be the frequency value.

2. The optimized training method according to claim 1, characterized in that, After obtaining the first random sample value and the second random sample value, the process further includes: The first random sample value and the second random sample value are mapped to a first Mel frequency and a second Mel frequency using a preset mapping coefficient. The mapping coefficient includes the mapping relationship between the Mel frequency and the result of the random sampling. The first random sample value is updated at the first Mel frequency, and the second random sample value is updated at the second Mel frequency.

3. The optimized training method according to claim 1, characterized in that, The step of constructing the rectangular bandpass filter function based on the first cutoff frequency and the second cutoff frequency includes: Calculate the first ratio of the preset frequency variable to the first cutoff frequency, and use half of the first ratio as the first parameter of the preset rectangular function to obtain the first rectangular function; Calculate the second ratio of the frequency variable to the second cutoff frequency, and use half of the second ratio as the second parameter of the rectangular function to obtain the second rectangular function; Subtracting the first rectangular function from the second rectangular function yields the rectangular bandpass filter function.

4. The optimized training method according to claim 1, characterized in that, After transforming the rectangular bandpass filter function into a time-domain function using inverse Fourier transform, the method further includes: Multiplying the time-domain function by a preset window function yields a windowed time-domain function; Accordingly, the convolution calculation of the speech sample and the time-domain function includes: The speech sample and the windowed time-domain function are convolved to obtain the convolution result.

5. The optimized training method according to claim 1, characterized in that, The classification model includes a feature extraction layer and a fully connected layer; The step of inputting the convolution calculation result into the preset classification model includes: The convolution calculation result is input into the feature extraction layer for feature extraction to obtain the feature extraction result; The convolution calculation result is multiplied by the feature extraction result, and the multiplication result is input into the fully connected layer for feature mapping to obtain the output of the classification model.

6. The optimization training method according to any one of claims 1 to 5, characterized in that, The step of adjusting the first cutoff frequency, the second cutoff frequency, and the model parameters in the classification model using gradient descent includes: The values ​​of the model parameters are fixed, and the values ​​of the first cutoff frequency and the second cutoff frequency are adjusted using the gradient descent method until the cross-entropy loss converges, thus obtaining the first convergent value. After obtaining the first convergence value, the values ​​of the first cutoff frequency and the second cutoff frequency are fixed, and the values ​​of the model parameters are adjusted using the gradient descent method until the cross-entropy loss converges, thus obtaining the second convergence value. If the second convergence value does not meet the preset condition, return to the step of fixing the value of the model parameters and adjusting the values ​​of the first cutoff frequency and the second cutoff frequency using the gradient descent method until the second convergence value meets the preset condition.

7. An optimization training device for a speech classification model, characterized in that, The optimized training device includes: The range determination module is used to acquire speech samples of known categories and their corresponding sampling frequencies, and to take the interval formed by a preset multiple of the sampling frequency and a preset value as the frequency range, wherein the preset value is greater than or equal to zero and less than the preset multiple of the sampling frequency. The random sampling module is used to randomly sample two frequency values ​​within the frequency range, using the larger of the two frequency values ​​as the first cutoff frequency and the smaller of the two frequency values ​​as the second cutoff frequency. The function transformation module is used to construct a rectangular bandpass filter function based on the first cutoff frequency and the second cutoff frequency, and transform the rectangular bandpass filter function into a time-domain function through inverse Fourier transform; The sample classification module is used to perform convolution calculation on the speech samples and the time domain function, and input the convolution calculation result into a preset classification model to obtain the output result of the classification model; The model training module is used to calculate the cross-entropy loss based on the output of the classification model and the category corresponding to the speech sample. Based on the cross-entropy loss, the gradient descent method is used to adjust the first cutoff frequency, the second cutoff frequency and the model parameters in the classification model until the cross-entropy loss converges, thereby obtaining the trained time-domain function and the trained classification model. The step of randomly selecting two frequency values ​​within the frequency range, using the larger of the two frequency values ​​as the first cutoff frequency, and using the smaller of the two frequency values ​​as the second cutoff frequency includes: According to the preset frequency sub-interval length, K sampling points are determined from the frequency range, where K is an integer greater than zero; Assign the same sampling probability to each sampling point to obtain a first sampling probability distribution. Then, perform an initial sampling from the K sampling points according to the first sampling probability distribution to obtain a first random sampling value. The preset value is determined as the first boundary, the preset multiple of the sampling frequency is determined as the second boundary, the distance between the first random sampled value and the first boundary and the second boundary are calculated respectively, the boundary corresponding to the larger value is determined as the target boundary, and the smaller value is determined as the target difference. If the target boundary is the first boundary, add the target boundary to the target difference to obtain the target frequency; if the target boundary is the second boundary, subtract the target boundary from the target difference to obtain the target frequency. A Gaussian distribution is constructed using the target frequency as the mean and the target difference as the variance. The Gaussian distribution is then multiplied by the first sampling probability distribution to obtain a second sampling probability distribution. Based on the second sampling probability distribution, resampling is performed from the K sampling points to obtain a second random sampling value, and the first random sampling value and the second random sampling value are determined to be the frequency value.

8. A computer device, characterized in that, The computer device includes a processor, a memory, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the optimized training method for the speech classification model as described in any one of claims 1 to 6.

9. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the optimized training method for the speech classification model as described in any one of claims 1 to 6.

Images