Speech processing method, computer program product, device and storage medium

Abstract

The present application relates to a speech processing method, a computer program product, a device and a storage medium. The method comprises: determining a key parameter combination of a speech processing model, setting a parameter search space for each key parameter in the key parameter combination; selecting multiple initial points based on the parameter search space of each key parameter, constructing a hybrid kernel function proxy model based on the initial point evaluation result and the speech processing model; determining an acquisition function based on the hybrid kernel function proxy model, and selecting a target point; training the hybrid kernel function proxy model based on the parameter value corresponding to the key parameter combination of the target point, outputting the target parameter value of the key parameter combination when the training termination condition is met, training the speech processing model according to the loss of the speech sample and the target parameter value of the key parameter combination, obtaining a target speech processing model, and processing the target speech using the target speech processing model. The reliability of speech processing can be improved by using this method.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a speech processing method, a computer program product, a device and a storage medium. Background Art

[0002] In recent years, deep learning models have made significant progress in many fields. Whether it is image classification, speech processing or natural language processing, deep learning technology has promoted the rapid development of artificial intelligence. However, as the scale and complexity of the model increase, the demand for optimizing model performance and training efficiency is also increasing.

[0003] During the training of speech processing models, a large number of parameters need to be adjusted to optimize training performance. However, the common tuning process usually relies on manual operations, which is time-consuming and labor-intensive, and it is difficult to accurately find the optimal parameter combination, resulting in low training efficiency and unstable model performance. Therefore, further optimization is needed. Summary of the invention

[0004] Based on this, it is necessary to provide a speech processing method, computer program product, device and storage medium that can improve the accuracy of speech processing in response to the above technical problems.

[0005] In order to solve the above technical problems, in a first aspect, a speech processing method is provided, the method comprising:

[0006] Processing multiple speech samples in the speech training set to obtain a score for each speech sample, and calculating the loss of the speech sample based on the score of each speech sample;

[0007] Determine a key parameter combination of a speech processing model, the key parameter combination including a plurality of key parameters, and set a parameter search space for each key parameter in the key parameter combination;

[0008] Based on the parameter search space of each key parameter, multiple initial points are selected, the initial points are evaluated, and a hybrid kernel function proxy model is constructed based on the initial point evaluation results and the speech processing model. The kernel function of the hybrid kernel function proxy model is a hybrid kernel function constructed based on the radial basis function kernel and the Mattern kernel.

[0009] Determine the acquisition function based on the hybrid kernel function proxy model, and select the target point based on the acquisition function;

[0010] The hybrid kernel function proxy model is trained based on the parameter values ​​corresponding to the key parameter combination of the target point. When the hybrid kernel function proxy model training termination condition is met, the hybrid kernel function proxy model outputs the target parameter value of the key parameter combination.

[0011] The speech processing model is trained according to the loss of the speech sample and the target parameter value of the key parameter combination to obtain the target speech processing model, and the target speech is processed using the target speech processing model.

[0012] In one embodiment, determining a key parameter combination of a speech processing model includes:

[0013] Obtain a key parameter combination to be selected, the key parameter combination to be selected includes multiple key parameters to be selected, each key parameter to be selected is set to multiple target discrete value points, and multiple sampling combinations are generated based on the multiple target discrete value points;

[0014] Testing multiple sampling combinations to obtain performance indicators of the sampling combinations;

[0015] Analyze the performance indicators of the sampling combination and determine the sensitivity index of each key parameter to be selected;

[0016] Based on the sensitivity index of each key parameter to be selected, a key parameter combination of the speech processing model is determined from the key parameter set to be selected.

[0017] In one embodiment, setting a parameter search space for each key parameter in the key parameter combination includes:

[0018] The Bayesian optimization method is used to search for the target discrete value points of each key parameter in the key parameter combination to find the parameter search space corresponding to each key parameter.

[0019] In one embodiment, the method further includes: constructing a hybrid kernel function, and constructing the hybrid kernel function includes:

[0020] Determine a first weight value corresponding to the radial basis function kernel and a second weight value corresponding to the Mattern kernel;

[0021] Constructing a hybrid kernel function formula based on the radial basis function kernel, the first weight value, the Mattern kernel and the second weight value;

[0022] The distance values ​​between the multiple initial points are calculated, and the distance values ​​between the multiple initial points are input into the mixed covariance matrix to obtain the mixed covariance matrix value.

[0023] In one embodiment, the hybrid kernel function formula is as follows:

[0024] ;

[0025] ;

[0026] ;

[0027] in, represents the mixed kernel function, represents the radial basis function, represents the Mattern kernel function, Indicates the initial point With the initial point The Euclidean distance between represents the variance of the output, represents the length scale, is the first weight value, is the second weight value.

[0028] In one embodiment, constructing a hybrid kernel function proxy model based on the initial point evaluation result and the speech processing model includes:

[0029] Select prediction points based on adaptive sampling strategy;

[0030] Calculate the distance between the prediction point and the initial point, and calculate the prediction result of the prediction point based on the distance between the prediction point and the initial point, the prediction mean formula, and the prediction variance formula;

[0031] The predicted mean formula is as follows:

[0032] ;

[0033] The prediction variance formula is as follows:

[0034] ;

[0035] in, Indicates the predicted point The predicted mean of Indicates the predicted point The prediction variance of represents the covariance vector between the predicted point and the initial point, represents the inverse of the covariance matrix of the key parameters, is the evaluation result vector of the initial point, Indicates the predicted point The autocovariance of Represents the covariance between the forecast point and the initial point minus the covariance adjustment term.

[0036] In one embodiment, determining the acquisition function based on the hybrid kernel function proxy model includes: determining the acquisition function type, and determining the acquisition function according to the acquisition function type and the hybrid kernel function proxy model, where the acquisition function is as follows:

[0037] ;

[0038] ;

[0039] in, Represents the predicted point and the expectation of improving the optimal point after sampling, Indicates the predicted point The predicted mean of Indicates the predicted point The predicted standard deviation of represents the known optimal evaluation result, represents the cumulative distribution function of the standard normal distribution, represents the probability density function of the standard normal distribution, represents the amount of improvement after normalization.

[0040] In one embodiment, selecting a target point based on an acquisition function includes:

[0041] The predicted value of the prediction point is calculated using the acquisition function. The predicted value is the expectation of the improvement of the optimal point after sampling the prediction point.

[0042] Determine whether the predicted value of the predicted point is greater than a preset threshold;

[0043] The prediction point whose predicted value is greater than the preset threshold is taken as the target point.

[0044] In one embodiment, selecting prediction points based on the adaptive sampling strategy includes:

[0045] Divide the parameter search space of the key parameters to obtain multiple sub-parameter search spaces;

[0046] determining a level of the sub-parameter search space according to the number of prediction points determined as target points in the sub-parameter search space;

[0047] When the level in any sub-parameter search space is lower than a preset level, the prediction point sampling in the sub-parameter search space whose level is lower than the preset level is reduced.

[0048] In one embodiment, training the hybrid kernel function proxy model based on parameter values ​​corresponding to the key parameter combination of the target point includes:

[0049] The parameter values ​​corresponding to the key parameter combinations of the target points are input into the proxy model for training to obtain the target point evaluation results;

[0050] Based on the target point evaluation result, the data set is updated, and the hybrid kernel function proxy model is trained using the updated data set to obtain an updated hybrid kernel function proxy model;

[0051] Construct an updated mixed covariance matrix, and calculate the prediction mean and prediction variance corresponding to the updated mixed kernel function proxy model based on the updated mixed covariance matrix and the updated data set;

[0052] The prediction result of the new prediction point is calculated using the prediction mean and prediction variance corresponding to the updated hybrid kernel function proxy model.

[0053] In one embodiment, when the hybrid kernel function proxy model training termination condition is met, the hybrid kernel function proxy model outputs a target parameter value of a key parameter combination including:

[0054] In response to the fact that the number of iterations of the hybrid kernel function proxy model is greater than the iteration threshold or that the predicted value of the prediction point is less than the preset value within a certain number of iterations of the hybrid kernel function proxy model, it is considered that the termination condition of the hybrid kernel function proxy model training is met, and the hybrid kernel function proxy model outputs the target parameter value of the key parameter combination in the iteration process.

[0055] In one embodiment, analyzing the performance indicators of the sampling combination to determine the sensitivity index of the key parameter to be selected includes:

[0056] Calculate the first sensitivity index and the second sensitivity index of the key parameter to be selected based on the performance index of the sampling combination, the first sensitivity index calculation formula, and the second sensitivity index calculation formula;

[0057] The first sensitivity index calculation formula is as follows:

[0058] ;

[0059] The second sensitivity index calculation formula is as follows:

[0060] ;

[0061] in, Indicates that due to input parameters Output caused by changes variance, Representation output The total variance of represents the batch size, Indicates exclusion of input parameters Output caused by changes in other parameters other than variance, Indicates the performance impact.

[0062] In order to solve the above technical problem, in a second aspect, a computer program product is provided, including a computer program, which implements the steps of the method described in the first aspect when executed by a processor.

[0063] In order to solve the above technical problem, in the third aspect, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the following steps when executing the computer program: the processor implements the steps of the method described in the first aspect when executing the computer program.

[0064] In order to solve the above technical problem, in a fourth aspect, the present application provides a computer-readable storage medium on which a computer program is stored, and when the computer program is executed by a processor, the steps of the method described in the first aspect are implemented.

[0065] Different from the prior art, the present application processes multiple speech samples in a speech training set to obtain a score for each speech sample, and calculates the loss of the speech sample based on the score of each speech sample; determines a key parameter combination of a speech processing model, the key parameter combination includes multiple key parameters, and sets a parameter search space for each key parameter in the key parameter combination; selects multiple initial points based on the parameter search space of each key parameter, evaluates the initial points, and constructs a hybrid kernel function proxy model based on the initial point evaluation results and the speech processing model, wherein the kernel function of the hybrid kernel function proxy model is a hybrid kernel function constructed based on a radial basis function kernel and a Mattern kernel; determines an acquisition function based on the hybrid kernel function proxy model, and selects a target point based on the acquisition function; trains the hybrid kernel function proxy model based on the parameter values ​​corresponding to the key parameter combination of the target point, and when the termination condition of the hybrid kernel function proxy model training is met, the hybrid kernel function proxy model outputs the target parameter value of the key parameter combination; trains the speech processing model according to the loss of the speech sample and the target parameter value of the key parameter combination to obtain a target speech processing model, and processes the target speech using the target speech processing model. In this way, the hybrid kernel function constructed by applying the radial basis function kernel and the Mattern kernel can improve the smoothness and robustness of the hybrid kernel function proxy model, thereby improving the accuracy of parameter optimization; the application of the target point selection based on the acquisition function and the adaptive sampling strategy can adaptively explore the parameter search space, automatically find the optimal parameter combination, significantly improve the training performance of the speech processing model, and improve the accuracy of speech processing. At the same time, it reduces resource consumption and human intervention. BRIEF DESCRIPTION OF THE DRAWINGS

[0066] Figure 1 is a flow chart of a speech processing method in one embodiment;

[0067] Figure 2 is a structural block diagram of a speech processing device in one embodiment;

[0068] Figure 3 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0069] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0070] In deep learning applications, the training phase is crucial. It continuously adjusts model parameters to enable the model to have good generalization capabilities on real data. Accurately evaluating and comprehensively optimizing the training performance of deep learning models has become a current research hotspot and a key requirement in practical applications. In this process, hardware performance evaluation is particularly important. Through evaluation, the training efficiency of different hardware architectures can be objectively measured and compared, thereby providing a strong basis for selecting a suitable hardware platform.

[0071] As an important tool for evaluating the training performance of AI models on different hardware platforms, the MLPerf benchmark is widely used in the evaluation of training performance. However, in practical applications, the training speed and resource usage of deep learning models are often closely related to the parameter settings of the model.

[0072] Traditional training methods usually require manual adjustment of a large number of parameters, which is time-consuming, labor-intensive, and error-prone. Manual parameter adjustment often makes it difficult to find the optimal parameter combination, resulting in unstable model performance. In addition, the training effect of the model may fluctuate due to improper parameter selection, and consistent effects cannot be ensured, and the performance of the model may not be fully utilized. In addition, due to the need for multiple adjustments and experiments, the model training process consumes a lot of time and computing resources, resulting in low training efficiency.

[0073] In view of the above technical problems, in one embodiment, Figure 1 As shown, the present application provides a speech processing method, which specifically includes the following steps:

[0074] Step 101 , a plurality of speech samples in a speech training set are processed to obtain a score of each speech sample, and the loss of the speech sample is calculated based on the score of each speech sample.

[0075] You can obtain speech samples from the Internet, generate a speech training set, and process the speech samples in the speech training set. The processing here can be denoising, framing, and feature extraction. After processing, you can get the language score and acoustic score of each speech sample in multiple speech samples. Speech processing models can include CNN (convolutional neural network) model, RNN (recurrent neural network), RNNT (recurrent neural network transformer) model, ResNet (deep residual network) model, BERT (pre-trained language) model and other speech processing models.

[0076] Corresponding weights are set for the acoustic score and language score corresponding to each speech sample, and the acoustic score and language score of each speech sample are weighted to obtain the final score of each speech sample. The loss of each speech sample is calculated using the loss function based on the final score of each speech sample. The speech processing model can be trained according to the loss of each speech sample subsequently, which is conducive to improving the model processing accuracy.

[0077] Step 102, determining a key parameter combination of a speech processing model, the key parameter combination including a plurality of key parameters, and setting a parameter search space for each key parameter in the key parameter combination.

[0078] Specifically, a key parameter combination to be selected is obtained, the key parameter combination to be selected includes multiple key parameters to be selected, each key parameter to be selected is set to multiple target discrete value points, and multiple sampling combinations are generated based on the multiple target discrete value points; the multiple sampling combinations are tested to obtain performance indicators of the sampling combinations; the performance indicators of the sampling combinations are analyzed to determine the sensitivity index of each key parameter to be selected; and the key parameter combination of the speech processing model is determined from the key parameter set to be selected based on the sensitivity index of each key parameter to be selected.

[0079] The key parameter combination to be selected may be a combination of several types of key parameters that have an impact on the performance of the speech processing model.

[0080] The key parameter combination can be determined from the key parameter combination to be selected according to the type of training model and the sensitivity index corresponding to the multiple key parameters to be selected. Taking the training model as RNNT as an example, the RNNT model integrates the language model and the acoustic model to improve the recognition effect through joint optimization. The parameters that affect the performance of the RNNT model include:

[0081] (1) EVAL_BATCHSIZE (batch size used in the model evaluation phase). The batch size determines the number of data samples processed in one forward propagation. A larger batch size can usually improve computing efficiency, but it also requires more video memory or memory resources. If EVAL_BATCHSIZE is set too small, GPU resources are not fully utilized, resulting in low computing efficiency. If it is set too large, insufficient memory may occur, causing the program to crash. Through Bayesian optimization, the optimal batch size can be automatically found to maximize training efficiency.

[0082] (2) CG_UNROLL_FACTOR (number of times the loop is unrolled in the model computation graph). Unrolling a computation graph refers to explicitly unrolling the loop operation in the computation graph to improve computational efficiency or reduce dependencies. A higher unrolling factor can reduce the number of loop executions, thereby speeding up the computation, but may increase the complexity and memory requirements of the computation graph. In the RNNT model, a higher CG_UNROLL_FACTOR will speed up the forward and backward propagation of the model, but may cause the computation graph structure to be too complex and consume too much memory. If the unrolling factor is set too small, the loop is executed frequently, which may lead to a slow training process. Therefore, by optimizing this parameter, the optimal balance between computational speed and resource usage can be found.

[0083] (3) DATA_CPU_THREADS (specifies the number of CPU threads used for data processing and loading). During MLPerfRNNT training, data preprocessing and loading is a possible bottleneck. Increasing the number of CPU threads can improve the efficiency of the data input pipeline, thereby speeding up training. In RNNT model training, if the number of CPU threads is set too low, data processing is slow, which may lead to a waste of GPU resources. Setting too high a number of threads speeds up data loading, but it will take up more CPU resources. By optimizing this parameter, data processing can be avoided from becoming a bottleneck in training, while CPU resources can be used reasonably.

[0084] (4) BATCH_SPLIT_FACTOR (controls the factor that splits the batch into smaller parts), which is particularly useful when you need to distribute training across multiple devices (such as multiple GPUs). By splitting large batches into smaller parts, you can better utilize the computing power of multiple devices, balance the load, reduce communication overhead, and thus improve overall training efficiency. In a multi-GPU training environment, if BATCH_SPLIT_FACTOR is set too small, it may cause some GPUs to have a large load, increase communication latency, and reduce training efficiency. Through reasonable batch splitting, the resources of multiple GPUs can be used more evenly, improving the efficiency of multi-device training.

[0085] (5) AMP_LVLS (Automatic Mixed Precision Levels): During the training and inference process of deep learning models, the use of mixed precision can speed up calculations and reduce video memory usage.

[0086] The above parameters that have an impact on the performance of the RNNT model are taken as key parameters to be selected, and each key parameter to be selected is set to several representative discrete points, that is, target discrete value points. The parameter value range of these target discrete value points can be set according to experience to facilitate faster identification of key parameters that affect performance. The parameters of this application are hyperparameters, which are control variables in the model training process and are used to control the complexity and training process of the model. They are not learned through training data and are not directly included in the model.

[0087] Exemplarily, the parameter value range of the target discrete value point can be as follows:

[0088] EVAL_BATCHSIZE (batch size used during model evaluation): {16, 128};

[0089] CG_UNROLL_FACTOR (number of loop unrolling in the model computation graph): {1, 5};

[0090] DATA_CPU_THREADS (specifies the number of CPU threads used for data processing and loading): {1, 8};

[0091] BATCH_SPLIT_FACTOR (controls the factor for splitting the batch into smaller parts): {1, 8};

[0092] AMP_LVLS (Automatic Mixed Precision Level): {0, 1, 2, 3};

[0093] A number of sampling combinations can be generated according to the parameter value range of the key parameters to be selected by using random number sequences such as the Sobol sequence and random number table method, so that the value range of the key parameters to be selected is evenly covered. In this application, the Sobol sequence sampling method is introduced for uniform exploration of the initial parameter space to ensure comprehensive coverage of the key parameters.

[0094] Table 1 gives a specific example of generating multiple sampling combinations based on the parameter value range of the above target discrete value points:

[0095]

[0096] Table 1

[0097] It can be seen from Table 1 that each sampling combination is a sampling combination composed of the discrete point values ​​of each key parameter to be selected from the parameter value range of the target discrete value point. Multiple sampling combinations are tested and the performance index of each sampling combination is recorded. Table 2 shows the performance index obtained by testing the sampling combinations in Table 1:

[0098]

[0099] Table 2

[0100] Analyzing the performance index of the sampling combination to determine the sensitivity index of each key parameter to be selected may include, specifically, calculating a first sensitivity index and a second sensitivity index of the key parameter to be selected based on the performance index of the sampling combination, a first sensitivity index calculation formula, and a second sensitivity index calculation formula;

[0101] The first sensitivity index calculation formula is as follows:

[0102] ;

[0103] The second sensitivity index calculation formula is as follows:

[0104] ;

[0105] in, Indicates that due to input parameters Output caused by changes variance, Representation output The total variance of represents the batch size, Indicates exclusion of input parameters Output caused by changes in other parameters other than variance, Indicates the performance impact.

[0106] Sobol sensitivity analysis is a global sensitivity analysis method that quantifies the contribution of each input parameter and its interaction to the output result by decomposing the output variance. The results include the first-order sensitivity index S1 (first sensitivity index) and the total sensitivity index ST (second sensitivity index).

[0107] In deep learning models, batch size has a higher S1 value, which means that batch size alone has a significant impact on model inference performance (such as speed or latency), regardless of changes in other parameters. The total sensitivity index (ST) represents the total impact of a single parameter and its interaction with other parameters on performance indicators. The higher the ST index, the greater the impact of the parameter and its interaction with other parameters on model performance. For example, if the ST value of the number of infer streams is much higher than the S1 value, this indicates that the number of infer streams not only has a significant impact on its own, but also has very important interactions with other parameters.

[0108] Exemplarily, the calculated sensitivity index of each key parameter to be selected may be as follows:

[0109] EVAL_BATCHSIZE (batch size used during model evaluation): S1=0.38, ST=0.52;

[0110] CG_UNROLL_FACTOR (number of loop unrollings in the model computation graph): S1=0.87, ST=0.3;

[0111] DATA_CPU_THREADS (specifies the number of CPU threads used for data processing and loading): S1=0.53, ST=0.45;

[0112] BATCH_SPLIT_FACTOR (controls the factor for splitting the batch into smaller parts): S1=0.31, ST=0.69;

[0113] AMP_LVLS (Automatic Mixed Precision Level): S1=0.26, ST=0.03;

[0114] According to the values ​​of the first sensitivity index and the second sensitivity index, the key parameters to be selected are determined as EVAL_BATCHSIZE (batch size used in the model evaluation phase), CG_UNROLL_FACTOR (number of loop unrolling in the model calculation graph), DATA_CPU_THREADS (specify the number of CPU threads used for data processing and loading) and BATCH_SPLIT_FACTOR (control the factor for splitting the batch into smaller parts), and the key parameter combination is generated based on these key parameters. Since the values ​​of the first sensitivity index and the second sensitivity index of AMP_LVLS (automatic mixed precision level) are low and have little impact on performance, they can be discarded as the main optimization object, that is, they are not added to the key parameter combination as key parameters.

[0115] In this application, the performance index based on the sampling combination and the first sensitivity index calculation formula and the second sensitivity index calculation formula are set to calculate the first sensitivity index and the second sensitivity index of the key parameter to be selected. According to the numerical values ​​of the first sensitivity index and the second sensitivity index of each key parameter to be selected, the key parameters to be selected that have a great impact on the performance of the speech processing model are selected from the key parameters to be selected as key parameters to form a key parameter combination. Subsequently, the parameters that have a significant impact on the performance of the speech processing model can be optimized intensively to further improve the overall performance.

[0116] After determining the key parameter combination of the speech processing model, a parameter search space can be set for each key parameter in the key parameter combination. The parameter search space here is the selection range of each key parameter in this application. The parameter search space of each key parameter can be set based on experience. The parameter value of each key parameter is sampled within the range to obtain the sampling point for evaluation. Optionally, a fine search can be performed on the discrete points of the key parameters through Bayesian optimization to find the optimal configuration combination. Assuming that the parameters of an RNNT model are optimized, the specific parameter search space of its key parameters can be as follows: (EVAL_BATCHSIZE: [16,256]; CG_UNROLL_FACTOR: [1,5]; DATA_CPU_THREADS: [1,16]; BATCH_SPLIT_FACTOR: [1,8];).

[0117] In this application, the parameter space is adaptively explored and the optimal parameter combination is automatically found, which significantly improves the training performance of the deep learning model while reducing resource consumption and human intervention.

[0118] Step 103, select multiple initial points based on the parameter search space of each key parameter, evaluate the initial points, and build a hybrid kernel function proxy model based on the initial point evaluation results and the speech processing model. The kernel function of the hybrid kernel function proxy model is a hybrid kernel function built based on the radial basis function kernel and the Mattern kernel.

[0119] In Bayesian optimization, by random sampling, the value of each key parameter in the key parameter combination is selected from the parameter search space of each key parameter as the parameter value corresponding to the key parameter combination of the initial point, and then the initial point is evaluated. The evaluation results of these initial points will be used to construct a Gaussian process proxy model, that is, the hybrid kernel function proxy model of this application. In this application, an adaptive sampling strategy is added to dynamically adjust the subsequent sampling point selection according to the initial evaluation results. The parameter combination selection of the initial point is as follows:

[0120] Initial point 1: EVAL_BATCHSIZE=64, CG_UNROLL_FACTOR=3, DATA_CPU_THREADS=8, BATCH_SPLIT_FACTOR=4, the evaluation result is y_1 = 45;

[0121] Initial point 2: EVAL_BATCHSIZE=128, CG_UNROLL_FACTOR=2, DATA_CPU_THREADS=4, BATCH_SPLIT_FACTOR=2, the evaluation result is y_2 = 35;

[0122] Initial point 3: EVAL_BATCHSIZE=32, CG_UNROLL_FACTOR=4, DATA_CPU_THREADS=12, BATCH_SPLIT_FACTOR=1, the evaluation result is y_3 = 42;

[0123] Each of the above initial points corresponds to a specific key parameter combination, and then its performance is evaluated in the MLPerf training test. Here, the training time (min) is used as the main reference indicator of performance. The shorter the training time, the better the performance. The evaluation result is the result of the test according to the corresponding parameters.

[0124] Based on the evaluation results of the initial sampling points, a proxy model is constructed. The model predicts the training performance of each parameter combination. By introducing an adaptive sampling strategy, the proxy model dynamically adjusts the selection of subsequent sampling points based on the points that have been evaluated, giving priority to further sampling in areas with greater performance potential.

[0125] Use Gaussian Process as a proxy model, input parameters such as BATCHSIZE (batch size used during model evaluation phase), CG_UNROLL_FACTOR (number of loop unrolling in the model calculation graph), DATA_CPU_THREADS (specifies the number of CPU threads for data processing and loading) and BATCH_SPLIT_FACTOR (controls the factor for splitting the batch into smaller parts), and output the predicted training performance (training time).

[0126] The Gaussian process proxy model of the present application is a speech processing model (mixed kernel function proxy model) in a Gaussian process, specifically an RNNT model in a Gaussian process, in which the optimal value distribution of the objective function at any point obeys a Gaussian distribution determined by a mean function and a mixed covariance matrix, and the mixed covariance matrix is ​​constructed based on a radial basis function kernel and a Mattern kernel. The radial basis function kernel has higher smoothness and is suitable for data with strong global smoothness. The Mattern kernel provides more flexibility and can adapt to different smoothness requirements. The combination of the two can achieve a balance between smoothness and robustness.

[0127] In a specific implementation, the hybrid kernel function proxy model is constructed as follows:

[0128] S1: Construct the mixed covariance matrix:

[0129] First, a hybrid kernel function formula is constructed by determining the first weight value corresponding to the radial basis function kernel and the second weight value corresponding to the Mattern kernel; a hybrid kernel function formula is constructed based on the radial basis function kernel, the first weight value, the Mattern kernel and the second weight value; and a hybrid covariance matrix value is obtained.

[0130] In order to improve the construction of the proxy model, the present application sets a combination of radial basis function kernel and Mattern kernel to construct a hybrid kernel function, and then constructs a hybrid kernel function proxy model, and then calculates the covariance matrix K according to the hybrid kernel function. The radial basis function kernel (RBF kernel for short), also known as the Gaussian kernel, is a commonly used kernel function used in support vector machines (SVM) and other machine learning algorithms. It is a nonlinear kernel function that calculates similarity based on the distance between data points. It is suitable for data with strong global smoothness. The Matren kernel is widely used in kernel functions in technical fields such as spatial data analysis and regression modeling. By controlling the smoothness and correlation range of the kernel function, it can not only filter the noise signal, but also extract the spatial data features to obtain the optimal fitting result, and has good flexibility and practicality. In the present application, the method of setting the radial basis function kernel and the Mattern kernel in combination to construct a hybrid kernel function can make the proxy model achieve a balance between smoothness and robustness.

[0131] The mixed kernel function formula is as follows:

[0132] ;

[0133] ;

[0134] ;

[0135] in, represents the mixed kernel function, represents the radial basis function, represents the Mattern kernel function, Indicates the initial point With the initial point The Euclidean distance between represents the variance of the output, represents the length scale, is the first weight value, is the second weight value.

[0136] In one embodiment, determining the first weight value and the second weight value may include: calculating the first weight value based on the first and second sensitivity indexes corresponding to each key parameter and the first expected factor corresponding to the radial basis function kernel; calculating the second weight value based on the first and second sensitivity indexes corresponding to each key parameter and the second expected factor corresponding to the Mattern kernel;

[0137] The calculation formula of the first weight value can be shown as follows:

[0138] ;

[0139] The calculation formula of the second weight value can be as follows:

[0140] ;

[0141] in, represents the first sensitivity index of each of the n key parameters and

[0142] The sum of the products of the second sensitivity index, A represents the first expected factor, and B represents the second expected factor.

[0143] The expected factor here can be set according to the overall effect of the expected data. For example, if the data is expected to be smoother as a whole, the value of the first expected factor is set higher than the value of the second expected factor. The above implementation method is used to achieve a balance between smoothness and robustness of the proxy model.

[0144] Calculate the distance values ​​between multiple initial points, input the distance values ​​between multiple initial points into the mixed covariance matrix, obtain the elements of the mixed covariance matrix K, and then input the elements of the mixed covariance matrix K into the mixed covariance matrix to obtain the mixed covariance matrix value, that is, the specific value of K.

[0145] Exemplarily, the number of the multiple initial points is 3 as an example for description, and the known initial points include the selected parameter combination of initial point 1, initial point 2 and initial point 3;

[0146] Calculate the distance between initial point 1 and initial point 2 ,in, Indicates the parameter value of the key parameter of the initial point 1, Indicates the parameter value of the key parameter of the initial point 2. The specific value is the value of the initial point 1 and the initial point 2 obtained by random sampling;

[0147] ;

[0148] Calculate the distance between initial point 1 and initial point 3 ,in, Indicates the parameter value of the key parameter of the initial point 1, Indicates the parameter value of the key parameter of the initial point 3. The specific value is the value of the initial point 1 and the initial point 3 obtained by random sampling;

[0149] ;

[0150] Calculate the distance between initial point 2 and initial point 3 ,in, Indicates the parameter value of the key parameter of the initial point 2, Indicates the parameter value of the key parameter of the initial point 3. The specific value is the value of the initial point 2 and the initial point 3 obtained by random sampling;

[0151] ;

[0152] The formula for the mixture covariance matrix is ​​as follows:

[0153]

[0154] The elements of the mixed covariance matrix are calculated using the distance values ​​between multiple initial points, as shown below:

[0155] ;

[0156] ;

[0157] ;

[0158] ;

[0159] Therefore, the covariance matrix K is:

[0160] ;

[0161] S2: Calculate the predicted distribution of the predicted points:

[0162] Suppose that the evaluation result of a new sampling point (prediction point) needs to be predicted. The parameter value of each key parameter of the prediction point is randomly selected from the parameter search space of each key parameter. Suppose the parameter combination of the prediction point is: (EVAL_BATCHSIZE=80, CG_UNROLL_FACTOR=3, DATA_CPU_THREADS=10, BATCH_SPLIT_FACTOR=3).

[0163] Calculate the distance between the predicted point and the initial point, and calculate the mixed covariance vector , taking the initial points including initial point 1, initial point 2, and initial point 3 as examples, calculate the distance between each initial point and the predicted point:

[0164] Calculate the distance between the initial point 1 and the predicted point ,in, Indicates the parameter value of the key parameter of the initial point 1, Indicates the parameter value of the key parameter of the prediction point. The specific value is the value of the initial point 1 and the prediction point obtained by random sampling;

[0165] ;

[0166] Calculate the distance between the initial point 2 and the predicted point ,in, Indicates the parameter value of the key parameter of the initial point 2, Indicates the parameter value of the key parameter of the prediction point. The specific value is the value of the initial point 2 and the prediction point obtained by random sampling;

[0167] ;

[0168] Calculate the distance between the initial point 3 and the predicted point ,in, Indicates the parameter value of the key parameter of the initial point 3, The parameter value of the key parameter of the prediction point is represented by the value of the initial point 3 and the initial prediction point obtained by random sampling;

[0169] ;

[0170] Calculate the prediction result of the prediction point based on the distance value between the prediction point and the initial point, the prediction mean formula and the prediction variance formula;

[0171] The predicted mean formula is as follows:

[0172] ;

[0173] The prediction variance formula is as follows:

[0174] ;

[0175] in, Indicates the predicted point The predicted mean of , which is the estimate of the output value of this point in the Gaussian process model; Indicates the predicted point The prediction variance represents the uncertainty of the predicted value; represents the covariance vector between the predicted point and the initial point, represents the inverse of the covariance matrix of the key parameters, is the evaluation result vector of the initial point, Indicates the predicted point The autocovariance of Represents the covariance between the forecast point and the initial point minus the covariance adjustment term.

[0176] Substitute the distance value between the predicted point and the initial point into the prediction mean formula and the prediction variance formula to obtain the predicted mean result (prediction result) of the predicted point and the predicted standard deviation of the predicted point. The predicted standard deviation is the result of taking the square root of the prediction variance. Substitute the distance value between the predicted point and the initial point 1, initial point 2, and initial point 3 into the prediction mean formula and the prediction variance formula to obtain the predicted mean result of the predicted point. , the prediction standard deviation of the prediction point .

[0177] Step 104: determine an acquisition function based on the hybrid kernel function proxy model, and select target points based on the acquisition function.

[0178] In this application, an adaptive acquisition function is used to select new prediction points. Determining the acquisition function based on the hybrid kernel function proxy model includes: determining the acquisition function type. Commonly used acquisition functions include: Expected Improvement (EI), Upper Confidence Bound (UCB), Probability of Improvement (PI). In this example, combined with the adaptive sampling strategy, Expected Improvement (EI) is selected as the acquisition function. The adaptive sampling strategy ensures that the acquisition function not only considers the selection of the optimal point when selecting points, but also dynamically adjusts the density of sampling points to improve the exploration efficiency of high potential areas.

[0179] Dynamically adjusting the density of sampling points is mainly reflected in the use of adaptive sampling strategies. By selecting an appropriate acquisition function (such as predicted expected improvement EI), the algorithm can prioritize areas with greater performance potential for further sampling in each iteration. When the model identifies an area where there may be an opportunity for performance improvement, it increases the sampling density of that area, thereby improving the model's exploration efficiency in high-potential areas.

[0180] The acquisition function is determined based on the acquisition function type and the hybrid kernel function surrogate model. Optionally, the next point to be evaluated can be selected by optimizing the acquisition function (such as expected improvement EI or upper confidence limit UCB). Assuming that expected improvement EI is used, the acquisition function is as follows:

[0181] ;

[0182] ;

[0183] in, Represents the predicted point and the expectation of improving the optimal point after sampling, Indicates the predicted point The predicted mean of Indicates the predicted point The predicted standard deviation of Represents the known optimal evaluation result (in this application is 35), represents the cumulative distribution function of the standard normal distribution, Represents the probability density function of the standard normal distribution. It represents the improvement after standardization, which measures the The degree of improvement of the predicted value relative to the currently known optimal value can be used to calculate the expected improvement after normalization. The value of .

[0184] The predicted value of the prediction point is calculated using the acquisition function. The predicted value is the expected improvement of the optimal point after sampling the prediction point. It is determined whether the predicted value of the prediction point is greater than the preset threshold. The prediction point whose predicted value is greater than the preset threshold is taken as the target point. If the value is greater than the preset threshold, the point is selected as the next target point to be evaluated. The selection of the optimal point is reflected in the maximization of the acquisition function. When used as a collection function, each candidate point (prediction point) is calculated in each iteration. value, select the value with the largest The point with the best value is selected as the next sampling point (target point). This selection strategy ensures that the search is carried out in the direction of optimal performance as much as possible in each step, which increases the probability of finding the global optimal solution.

[0185] First, calculate the elements in the acquisition function and convert the predicted mean result of the predicted point into , the prediction standard deviation of the prediction point The value of the standardized improvement Z is 12.4. as well as For Z=12.4, we can find the value of the standard normal distribution table or calculate it: (Since the Z value is very small, is very close to 0), similarly, for Z = 12.4, we have (Since the Z value is very small, very close to 0).

[0186] Substituting the elements in the above acquisition function into the acquisition function formula, we get:

[0187] ;

[0188] In the above calculation, due to Much smaller than The value of Very small, resulting in expected improvement Close to zero. This means that under the current model, the point target point does not provide enough improvement potential and may not be a good choice. Therefore, it is improved and a threshold is introduced.

[0189] In order to avoid collecting inefficient points, a Thresholds, e.g.

[0190] When the calculated When the value exceeds this threshold, the point is considered to have improvement potential and is worthy of further evaluation. , does not exceed the threshold, so the point is considered to have insufficient improvement potential and is not suitable for selection as the next evaluation point (target point).

[0191] With the adaptive sampling strategy, the acquisition function not only adjusts the density of sampling points based on the current expected improvement, but also dynamically adjusts the density of sampling points. If the points in the sampling area gradually become inefficient, the adaptive strategy will reduce the sampling density in this area and explore more in potentially efficient areas.

[0192] For each prediction point, the mean is predicted by the Gaussian process model. and standard deviation , then calculate For example, in the given example, the calculated The value is 0 (i.e., lower than the set threshold of 0.01), indicating that the improvement potential of this point is not great. When the value is greater than the threshold (for example, 0.01), it means that the area where the point is located has great potential for improvement, so the area will be considered as an efficient area and further sampling will be performed in this area.

[0193] To find the optimum , will calculate different prediction points in the entire search space , then select The largest point is the next point to be evaluated.

[0194] Suppose in the search space, another point is found The predicted mean of is 33 and the predicted standard deviation is 1. Calculate The corresponding Z=-2, Z=-2 can be obtained by looking up the standard normal distribution table or by calculation: , similarly, for Z=-2, we have , substitute the elements in these acquisition functions into the acquisition function formula, and we get:

[0195] ;

[0196] Although The value is small, but it is better than the previous point Bigger, so Compared to The corresponding prediction point is a better prediction point and can be used as the target point.

[0197] By comparing all possible prediction points, choose The largest point is taken as the next parameter combination that needs to be actually evaluated (the parameter combination of the target point), and its true evaluation result is obtained.

[0198] In one embodiment, selecting prediction points based on an adaptive sampling strategy includes: dividing the parameter search space of the key parameters to obtain multiple sub-parameter search spaces; determining the level of the sub-parameter search space according to the number of prediction points determined as target points in the sub-parameter search space; when the level of any sub-parameter search space is lower than a preset level, reducing the sampling of prediction points in the sub-parameter search space with a level lower than the preset level.

[0199] Optionally, the parameter search space of the key parameters can be divided into sub-parameter search spaces of the same size as the range of the parameter search space of the key parameters (the integration of the parameter search spaces of multiple key parameters) and the number of sub-parameter search spaces that need to be divided. The ratio of the number of prediction points of calculated prediction values ​​in each sub-parameter search space to the number of prediction points of determined target points in the sub-parameter search space can be obtained, and the level of each sub-parameter search space is set according to the ratio of each sub-parameter search space. Generally, the higher the ratio, the higher the level of the sub-parameter search space, and vice versa, the lower the ratio, the lower the level of the sub-parameter search space. The set level of the sub-parameter search space can be a character, an operation symbol, a graphic symbol, etc., and the specific form of the set level of the sub-parameter search space is not limited in the present application.

[0200] When the level of any sub-parameter search space is lower than the preset level, it is considered that the efficiency of selecting the prediction point (sampling point) of the sub-parameter search space as the target point is reduced, and the prediction point sampling in the sub-parameter search space with a level lower than the preset level is reduced, and the prediction point sampling in the sub-parameter search space with a level higher than the preset level is increased to achieve dynamic adjustment of the sampling density. For example, when multiple prediction points in the sub-parameter search space A The values ​​are small, and the prediction points of the sub-parameter search space B A larger value indicates that sub-parameter search space B may contain a better solution, so the sampling density in sub-parameter search space A will be dynamically reduced, and the sampling density in sub-parameter search space B will be increased. The preset level here can be set according to the proportion. When the actual proportion of a sub-parameter search space is lower than the proportion corresponding to the preset level, it is considered that the efficiency of the prediction point of the sub-parameter search space being selected as the target point is reduced.

[0201] In this way, by setting an adaptive sampling strategy and dynamically adjusting the density of sampling points, we can focus on sampling in search spaces with higher potential, speed up convergence, and avoid ineffective sampling waste in search spaces with low potential. This method not only simplifies the overall optimization process and reduces dependence on manual intervention, but also enhances the adaptability of the model on different hardware platforms and data sets.

[0202] Step 105 , training the hybrid kernel function proxy model based on the parameter values ​​corresponding to the key parameter combination of the target point, and when the hybrid kernel function proxy model training termination condition is met, the hybrid kernel function proxy model outputs the target parameter value of the key parameter combination.

[0203] Specifically, the parameter values ​​corresponding to the key parameter combinations of the target points are input into the proxy model for training to obtain the target point evaluation results; the data set is updated based on the target point evaluation results, and the mixed kernel function proxy model is trained using the updated data set to obtain an updated mixed kernel function proxy model; an updated mixed covariance matrix is ​​constructed, and the prediction mean and prediction variance corresponding to the updated mixed kernel function proxy model are calculated based on the updated mixed covariance matrix and the updated data set; the prediction result of the new prediction point is calculated using the prediction mean and prediction variance corresponding to the updated mixed kernel function proxy model.

[0204] The hybrid kernel function proxy model is trained according to the parameter values ​​corresponding to the key parameter combination of the selected target point. By evaluating the performance of the parameter value of the target point, the hybrid kernel function proxy model is further improved to make its prediction more accurate. After determining the key parameter combination (target point) selected by optimizing the acquisition function, it is necessary to actually evaluate this new point and use the evaluation results to update the hybrid kernel function proxy model. The detailed process of this step is as follows:

[0205] Assume that the target point is selected by optimizing the expected improvement (EI) acquisition function , the parameter combination of the target point is: (EVAL_BATCHSIZE=100, CG_UNROLL_FACTOR=3, DATA_CPU_THREADS=10, BATCH_SPLIT_FACTOR=3).

[0206] Use the actual MLPerf RNNT to train with this parameter combination and record the corresponding evaluation results , assuming The value of is 30. This evaluation result is added to the existing evaluation result set to obtain an updated data set. For example, the existing evaluation result set , update the dataset in, , , represents the sampling points that have been evaluated historically (in the embodiment of the present application, the sampling points that have been evaluated historically are initial point 1, initial point 2, and initial point 3), is the target point of the current evaluation, , , Indicates the evaluation results corresponding to the sampling points that have been evaluated historically. The evaluation result of the target point of the current evaluation. Specifically, the specific data of the updated data set is: .

[0207] Retrain the hybrid kernel function proxy model using the updated data set to obtain an updated hybrid kernel function proxy model. The updated hybrid kernel function proxy model will make predictions based on more data points, thereby improving its prediction accuracy. The specific steps are as follows:

[0208] By calculating the distance between the target point and the historically evaluated sampling point, a new mixed covariance matrix element is obtained, and an updated mixed covariance matrix is ​​constructed based on the new mixed covariance matrix element. The updated mixed covariance matrix As shown below:

[0209] ;

[0210] Based on the new mixed covariance matrix The updated data set is used to recalculate the prediction mean and prediction variance of the model, and the prediction result of the target point is calculated using the prediction mean and prediction variance corresponding to the updated hybrid kernel function proxy model, which is used for the next step of acquisition function optimization and point selection. For example, for a new prediction point , the recalculated forecast mean and forecast variance are:

[0211] The predicted mean formula is as follows:

[0212] ;

[0213] The prediction variance formula is as follows:

[0214] ;

[0215] After updating the hybrid kernel function proxy model, you can go back to the step of optimizing the acquisition function and select the next target point through the updated model. This process is iterated continuously until certain termination conditions are met. The termination condition here can be that the number of iterations of the hybrid kernel function proxy model is greater than the iteration threshold or within a certain number of iterations of the hybrid kernel function proxy model, the predicted value of the predicted point is less than the preset value, and the hybrid kernel function proxy model training termination condition is met, and the hybrid kernel function proxy outputs the target parameter value combination in the iterative process. For example, when the number of iterations of the hybrid kernel function proxy model reaches 50 times, and the improvement of the evaluation result is lower than a preset value (the improvement of 5 consecutive iterations is less than 0.1%), the preset value and the number of iterations here can be set according to operating experience, and this application does not limit the specific values ​​of the preset value and the number of iterations.

[0216] In the iteration process of Bayesian optimization in the above embodiment, multiple key parameter combinations (initial point 1, initial point 2, initial point 3, target point) have been evaluated and corresponding evaluation results have been obtained. Assume that in the entire optimization process, the following key parameter combinations and their corresponding evaluation results are evaluated:

[0217] EVAL_BATCHSIZE=64,CG_UNROLL_FACTOR=3,DATA_CPU_THREADS=8,BATCH_SPLIT_FACTOR=4, the evaluation result is y_1=45;

[0218] EVAL_BATCHSIZE=128, CG_UNROLL_FACTOR=2, DATA_CPU_THREADS=4, BATCH_SPLIT_FACTOR=2, the evaluation result is y_2=35;

[0219] EVAL_BATCHSIZE=32, CG_UNROLL_FACTOR=4, DATA_CPU_THREADS=12, BATCH_SPLIT_FACTOR=1, the evaluation result is y_3=42;

[0220] EVAL_BATCHSIZE=100, CG_UNROLL_FACTOR=3, DATA_CPU_THREADS=10, BATCH_SPLIT_FACTOR=3, the evaluation result is y_4=30;

[0221] From these evaluated key parameter combinations, select the one with the best evaluation result (such as accuracy, performance score) as the target parameter value of the final key parameter combination. In this example, the highest evaluation result is y_4=30, and the corresponding parameter combination is: (EVAL_BATCHSIZE=100; CG_UNROLL_FACTOR=3; DATA_CPU_THREADS=10; BATCH_SPLIT_FACTOR=3;).

[0222] If the target parameter value of the key parameter combination with the best performance is determined, the target parameter value of this key parameter combination can be output from the model as the final optimization result and used in practical applications.

[0223] Step 106, training the speech processing model according to the loss of the speech sample and the target parameter value of the key parameter combination to obtain a target speech processing model, and using the target speech processing model to process the target speech.

[0224] In the present application, a target speech processing model is obtained by training the speech processing model using the loss of speech samples and the target parameter values ​​of the optimized key parameter combination. The target speech processing model is used to process the target speech, which may specifically include speech recognition, speech translation, and the like. This can improve the accuracy of speech processing and bring significant performance improvement and stronger adaptability to the speech processing model.

[0225] In order to ensure that the target parameter values ​​of the selected key parameter combination can indeed bring the best performance, additional verification can be performed on this parameter combination, such as retraining the model under multiple different random seeds to confirm the robustness and stability of the combination in practical applications. Through these specific steps, combined with the adaptive sampling strategy, Bayesian optimization can more efficiently find the best performance parameter combination and optimize the RNNT model training performance in the MLPerf benchmark.

[0226] In this application, Bayesian optimization is combined with an adaptive sampling strategy to efficiently optimize the training parameters of the RNNT model in the MLPerf evaluation, and intelligently adjust the parameters that have a decisive influence on the MLPerf test results, thereby significantly improving the training performance of the model. With Bayesian optimization as the core algorithm, the automatic tuning of key parameters is realized, and the target parameter combination is accurately predicted and selected, reducing the number of experiments and resource consumption. The adaptive sampling strategy further improves the optimization efficiency, and by dynamically adjusting the sampling point density, in-depth exploration is ensured in high-potential areas, which accelerates the convergence speed. Applying this method, intelligent and efficient performance optimization can be achieved in different test scenarios of MLPerf. This solution not only significantly improves the efficiency and accuracy of model training, but also greatly reduces resource consumption and provides an efficient parameter tuning process. In the end, it can bring significant performance improvements and stronger adaptability to the model in practical applications.

[0227] It should be understood that although Figure 1 The steps in the flowchart are shown in sequence as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. Moreover, Figure 1 At least part of the steps may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily executed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least part of the sub-steps or stages of other steps.

[0228] In one embodiment, Figure 2 As shown, a speech processing device is provided, comprising: a determination module 20, a construction module 21, a selection module 22, an output module 23 and a processing module 24, wherein:

[0229] A determination module 20 is used to process multiple speech samples in the speech training set to obtain a score for each speech sample, calculate the loss of the speech sample based on the score of each speech sample, determine a key parameter combination of the speech processing model, the key parameter combination includes multiple key parameters, and set a parameter search space for each key parameter in the key parameter combination;

[0230] A construction module 21 is used to select multiple initial points based on the parameter search space of each key parameter, evaluate the initial points, and construct a hybrid kernel function proxy model based on the initial point evaluation results and the speech processing model, wherein the kernel function of the hybrid kernel function proxy model is a hybrid kernel function constructed based on a radial basis function kernel and a Mattern kernel;

[0231] A selection module 22, used to determine an acquisition function based on a hybrid kernel function proxy model, and select target points based on the acquisition function;

[0232] An output module 23 is used to train the hybrid kernel function proxy model based on the parameter values ​​corresponding to the key parameter combination of the target point, and when the hybrid kernel function proxy model training termination condition is met, the hybrid kernel function proxy model outputs the target parameter value of the key parameter combination;

[0233] The processing module 24 trains the speech processing model according to the loss of the speech sample and the target parameter value of the key parameter combination to obtain the target speech processing model, and processes the target speech using the target speech processing model.

[0234] In one embodiment, the above device can implement another implementation of the speech processing method, and the specific steps are as follows:

[0235] The key parameter combinations that determine the speech processing model include:

[0236] Obtain a key parameter combination to be selected, the key parameter combination to be selected includes multiple key parameters to be selected, each key parameter to be selected is set to multiple target discrete value points, and multiple sampling combinations are generated based on the multiple target discrete value points;

[0237] Testing multiple sampling combinations to obtain performance indicators of the sampling combinations;

[0238] Analyze the performance indicators of the sampling combination and determine the sensitivity index of each key parameter to be selected;

[0239] Based on the sensitivity index of each key parameter to be selected, a key parameter combination of the speech processing model is determined from the key parameter set to be selected.

[0240] In one embodiment, the above device can implement another implementation of the speech processing method, and the specific steps are as follows:

[0241] Setting a parameter search space for each key parameter in the key parameter combination includes:

[0242] The Bayesian optimization method is used to search for the target discrete value points of each key parameter in the key parameter combination to find the parameter search space corresponding to each key parameter.

[0243] In one embodiment, the above device can implement another implementation of the speech processing method, and the specific steps are as follows:

[0244] The method further includes: constructing a hybrid kernel function, wherein constructing the hybrid kernel function includes:

[0245] Determine a first weight value corresponding to the radial basis function kernel and a second weight value corresponding to the Mattern kernel;

[0246] Constructing a hybrid kernel function formula based on the radial basis function kernel, the first weight value, the Mattern kernel and the second weight value;

[0247] The distance values ​​between the multiple initial points are calculated, and the distance values ​​between the multiple initial points are input into the mixed covariance matrix to obtain the mixed covariance matrix value.

[0248] In one embodiment, the above device can implement another implementation of the speech processing method, and the specific steps are as follows:

[0249] The mixed kernel function formula is as follows:

[0250] ;

[0251] ;

[0252] ;

[0253] in, represents the mixed kernel function, represents the radial basis function, represents the Mattern kernel function, Indicates the initial point With the initial point The Euclidean distance between represents the variance of the output, represents the length scale, is the first weight value, is the second weight value.

[0254] In one embodiment, the above device can implement another implementation of the speech processing method, and the specific steps are as follows:

[0255] Select prediction points based on adaptive sampling strategy;

[0256] The hybrid kernel function proxy model is constructed based on the initial point evaluation results and the speech processing model, including:

[0257] Calculate the distance between the prediction point and the initial point, and calculate the prediction result of the prediction point based on the distance between the prediction point and the initial point, the prediction mean formula, and the prediction variance formula;

[0258] The predicted mean formula is as follows:

[0259] ;

[0260] The prediction variance formula is as follows:

[0261] ;

[0262] in, Indicates the predicted point The predicted mean of Indicates the predicted point The prediction variance of represents the covariance vector between the predicted point and the initial point, represents the inverse of the covariance matrix of the key parameters, is the evaluation result vector of the initial point, Indicates the predicted point The autocovariance of Represents the covariance between the forecast point and the initial point minus the covariance adjustment term.

[0263] In one embodiment, the above device can implement another implementation of the speech processing method, and the specific steps are as follows:

[0264] Determining the acquisition function based on the hybrid kernel function proxy model includes: determining the acquisition function type, and determining the acquisition function according to the acquisition function type and the hybrid kernel function proxy model. The acquisition function is as follows:

[0265] ;

[0266] ;

[0267] in, Represents the predicted point and the expectation of improving the optimal point after sampling, Indicates the predicted point The predicted mean of Indicates the predicted point The predicted standard deviation of represents the known optimal evaluation result, represents the cumulative distribution function of the standard normal distribution, represents the probability density function of the standard normal distribution, represents the amount of improvement after normalization.

[0268] In one embodiment, the above device can implement another implementation of the speech processing method, and the specific steps are as follows:

[0269] The selection of target points based on the acquisition function includes:

[0270] The predicted value of the prediction point is calculated using the acquisition function. The predicted value is the expectation of the improvement of the optimal point after sampling the prediction point.

[0271] Determine whether the predicted value of the predicted point is greater than a preset threshold;

[0272] The prediction point whose predicted value is greater than the preset threshold is taken as the target point.

[0273] In one embodiment, the above device can implement another implementation of the speech processing method, and the specific steps are as follows:

[0274] The selection of prediction points based on the adaptive sampling strategy includes:

[0275] Divide the parameter search space of the key parameters to obtain multiple sub-parameter search spaces;

[0276] determining a level of the sub-parameter search space according to the number of prediction points determined as target points in the sub-parameter search space;

[0277] When the level in any sub-parameter search space is lower than a preset level, the prediction point sampling in the sub-parameter search space whose level is lower than the preset level is reduced.

[0278] In one embodiment, the above device can implement another implementation of the speech processing method, and the specific steps are as follows:

[0279] Training the hybrid kernel function proxy model based on the parameter values ​​corresponding to the key parameter combination of the target point includes:

[0280] The parameter values ​​corresponding to the key parameter combinations of the target points are input into the proxy model for training to obtain the target point evaluation results;

[0281] Based on the target point evaluation result, the data set is updated, and the hybrid kernel function proxy model is trained using the updated data set to obtain an updated hybrid kernel function proxy model;

[0282] Construct an updated mixed covariance matrix, and calculate the prediction mean and prediction variance corresponding to the updated mixed kernel function proxy model based on the updated mixed covariance matrix and the updated data set;

[0283] The prediction result of the new prediction point is calculated using the prediction mean and prediction variance corresponding to the updated hybrid kernel function proxy model.

[0284] In one embodiment, the above device can implement another implementation of the speech processing method, and the specific steps are as follows:

[0285] When the hybrid kernel function proxy model training termination conditions are met, the hybrid kernel function proxy model outputs the target parameter values ​​of the key parameter combinations including:

[0286] In response to the fact that the number of iterations of the hybrid kernel function proxy model is greater than the iteration threshold or that the predicted value of the prediction point is less than the preset value within a certain number of iterations of the hybrid kernel function proxy model, it is considered that the termination condition of the hybrid kernel function proxy model training is met, and the hybrid kernel function proxy model outputs the target parameter value of the key parameter combination in the iteration process.

[0287] In one embodiment, the above device can implement another implementation of the speech processing method, and the specific steps are as follows:

[0288] The method also includes: analyzing the performance indicators of the sampling combination, and determining the sensitivity index of the key parameters to be selected includes:

[0289] Calculate the first sensitivity index and the second sensitivity index of the key parameter to be selected based on the performance index of the sampling combination, the first sensitivity index calculation formula, and the second sensitivity index calculation formula;

[0290] The first sensitivity index calculation formula is as follows:

[0291] ;

[0292] The second sensitivity index calculation formula is as follows:

[0293] ;

[0294] in, Indicates that due to input parameters Output caused by changes variance, Representation output The total variance of represents the batch size, Indicates exclusion of input parameters Output caused by changes in other parameters other than variance, Indicates the performance impact.

[0295] For the specific definition of the speech processing device, please refer to the definition of the speech processing method above, which will not be repeated here. Each module in the above-mentioned speech processing device can be implemented in whole or in part by software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0296] In one embodiment, the present application also provides a computer program product, which includes a computer program stored on a non-transitory computer-readable storage medium, and the computer program includes program instructions. When the program instructions are executed by a computer, the computer can execute the speech processing methods provided by the above methods.

[0297] In one embodiment, a computer device is provided. The computer device may be a terminal, and its internal structure diagram may be as follows: Figure 3 As shown. The computer device includes a processor, a memory, a network interface, a display screen and an input device connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a voice processing method is implemented. The display screen of the computer device can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covering the display screen, or a key, trackball or touchpad set on the computer device housing, or an external keyboard, touchpad or mouse, etc.

[0298] Those skilled in the art will understand that Figure 3 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0299] In one embodiment, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the following steps when executing the computer program:

[0300] Step 101, processing multiple speech samples in a speech training set to obtain a score for each speech sample, and calculating the loss of the speech sample based on the score of each speech sample;

[0301] Step 102, determining a key parameter combination of a speech processing model, the key parameter combination including a plurality of key parameters, and setting a parameter search space for each key parameter in the key parameter combination;

[0302] Step 103, selecting multiple initial points based on the parameter search space of each key parameter, evaluating the initial points, and constructing a hybrid kernel function proxy model based on the initial point evaluation results and the speech processing model, wherein the kernel function of the hybrid kernel function proxy model is a hybrid kernel function constructed based on a radial basis function kernel and a Mattern kernel;

[0303] Step 104, determining an acquisition function based on the hybrid kernel function proxy model, and selecting target points based on the acquisition function;

[0304] Step 105, training the hybrid kernel function proxy model based on the parameter values ​​corresponding to the key parameter combination of the target point, and when the hybrid kernel function proxy model training termination condition is met, the hybrid kernel function proxy model outputs the target parameter value of the key parameter combination;

[0305] Step 106, training the speech processing model according to the loss of the speech sample and the target parameter value of the key parameter combination to obtain a target speech processing model, and using the target speech processing model to process the target speech.

[0306] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0307] The key parameter combinations that determine the speech processing model include:

[0308] Obtain a key parameter combination to be selected, the key parameter combination to be selected includes multiple key parameters to be selected, each key parameter to be selected is set to multiple target discrete value points, and multiple sampling combinations are generated based on the multiple target discrete value points;

[0309] Testing multiple sampling combinations to obtain performance indicators of the sampling combinations;

[0310] Analyze the performance indicators of the sampling combination and determine the sensitivity index of each key parameter to be selected;

[0311] Based on the sensitivity index of each key parameter to be selected, a key parameter combination of the speech processing model is determined from the key parameter set to be selected.

[0312] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0313] Setting a parameter search space for each key parameter in the key parameter combination includes:

[0314] The Bayesian optimization method is used to search for the target discrete value points of each key parameter in the key parameter combination to find the parameter search space corresponding to each key parameter.

[0315] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0316] The method further includes: constructing a hybrid kernel function, wherein constructing the hybrid kernel function includes:

[0317] Determine a first weight value corresponding to the radial basis function kernel and a second weight value corresponding to the Mattern kernel;

[0318] Constructing a hybrid kernel function formula based on the radial basis function kernel, the first weight value, the Mattern kernel and the second weight value;

[0319] The distance values ​​between the multiple initial points are calculated, and the distance values ​​between the multiple initial points are input into the mixed covariance matrix to obtain the mixed covariance matrix value.

[0320] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0321] The mixed kernel function formula is as follows:

[0322] ;

[0323] ;

[0324] ;

[0325] in, represents the mixed kernel function, represents the radial basis function, represents the Mattern kernel function, Indicates the initial point With the initial point The Euclidean distance between represents the variance of the output, represents the length scale, is the first weight value, is the second weight value.

[0326] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0327] The hybrid kernel function proxy model is constructed based on the initial point evaluation results and the speech processing model, including:

[0328] Select prediction points based on adaptive sampling strategy;

[0329] Calculate the distance between the prediction point and the initial point, and calculate the prediction result of the prediction point based on the distance between the prediction point and the initial point, the prediction mean formula, and the prediction variance formula;

[0330] The predicted mean formula is as follows:

[0331] ;

[0332] The prediction variance formula is as follows:

[0333] ;

[0334] in, Indicates the predicted point The predicted mean of Indicates the predicted point The prediction variance of represents the covariance vector between the predicted point and the initial point, represents the inverse of the covariance matrix of the key parameters, is the evaluation result vector of the initial point, Indicates the predicted point The autocovariance of Represents the covariance between the forecast point and the initial point minus the covariance adjustment term.

[0335] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0336] Determining the acquisition function based on the hybrid kernel function proxy model includes: determining the acquisition function type, and determining the acquisition function according to the acquisition function type and the hybrid kernel function proxy model. The acquisition function is as follows:

[0337] ;

[0338] ;

[0339] in, Represents the predicted point and the expectation of improving the optimal point after sampling, Indicates the predicted point The predicted mean of Indicates the predicted point The predicted standard deviation of represents the known optimal evaluation result, represents the cumulative distribution function of the standard normal distribution, represents the probability density function of the standard normal distribution, represents the amount of improvement after normalization.

[0340] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0341] The selection of target points based on the acquisition function includes:

[0342] The predicted value of the prediction point is calculated using the acquisition function. The predicted value is the expectation of the improvement of the optimal point after sampling the prediction point.

[0343] Determine whether the predicted value of the predicted point is greater than a preset threshold;

[0344] The prediction point whose predicted value is greater than the preset threshold is taken as the target point.

[0345] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0346] The selection of prediction points based on the adaptive sampling strategy includes:

[0347] Divide the parameter search space of the key parameters to obtain multiple sub-parameter search spaces;

[0348] determining a level of the sub-parameter search space according to the number of prediction points determined as target points in the sub-parameter search space;

[0349] When the level in any sub-parameter search space is lower than a preset level, the prediction point sampling in the sub-parameter search space whose level is lower than the preset level is reduced.

[0350] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0351] Training the hybrid kernel function proxy model based on the parameter values ​​corresponding to the key parameter combination of the target point includes:

[0352] The parameter values ​​corresponding to the key parameter combinations of the target points are input into the proxy model for training to obtain the target point evaluation results;

[0353] Based on the target point evaluation result, the data set is updated, and the hybrid kernel function proxy model is trained using the updated data set to obtain an updated hybrid kernel function proxy model;

[0354] Construct an updated mixed covariance matrix, and calculate the prediction mean and prediction variance corresponding to the updated mixed kernel function proxy model based on the updated mixed covariance matrix and the updated data set;

[0355] The prediction result of the new prediction point is calculated using the prediction mean and prediction variance corresponding to the updated hybrid kernel function proxy model.

[0356] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0357] When the hybrid kernel function proxy model training termination conditions are met, the hybrid kernel function proxy model outputs the target parameter values ​​of the key parameter combinations including:

[0358] In response to the fact that the number of iterations of the hybrid kernel function proxy model is greater than the iteration threshold or that the predicted value of the prediction point is less than the preset value within a certain number of iterations of the hybrid kernel function proxy model, it is considered that the termination condition of the hybrid kernel function proxy model training is met, and the hybrid kernel function proxy model outputs the target parameter value of the key parameter combination in the iteration process.

[0359] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0360] The performance indicators of the sampling combination are analyzed to determine the sensitivity index of the key parameters to be selected, including:

[0361] Calculate the first sensitivity index and the second sensitivity index of the key parameter to be selected based on the performance index of the sampling combination, the first sensitivity index calculation formula, and the second sensitivity index calculation formula;

[0362] The first sensitivity index calculation formula is as follows:

[0363] ;

[0364] The second sensitivity index calculation formula is as follows:

[0365] ;

[0366] in, Indicates that due to input parameters Output caused by changes variance, Representation output The total variance of represents the batch size, Indicates exclusion of input parameters Output caused by changes in other parameters other than variance, Indicates the performance impact.

[0367] In one embodiment, a computer readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the following steps are implemented:

[0368] Step 101, processing multiple speech samples in a speech training set to obtain a score for each speech sample, and calculating the loss of the speech sample based on the score of each speech sample;

[0369] Step 102, determining a key parameter combination of a speech processing model, the key parameter combination including a plurality of key parameters, and setting a parameter search space for each key parameter in the key parameter combination;

[0370] Step 103, selecting multiple initial points based on the parameter search space of each key parameter, evaluating the initial points, and constructing a hybrid kernel function proxy model based on the initial point evaluation results and the speech processing model, wherein the kernel function of the hybrid kernel function proxy model is a hybrid kernel function constructed based on a radial basis function kernel and a Mattern kernel;

[0371] Step 104, determining an acquisition function based on the hybrid kernel function proxy model, and selecting target points based on the acquisition function;

[0372] Step 105, training the hybrid kernel function proxy model based on the parameter values ​​corresponding to the key parameter combination of the target point, and when the hybrid kernel function proxy model training termination condition is met, the hybrid kernel function proxy model outputs the target parameter value of the key parameter combination;

[0373] Step 106, training the speech processing model according to the loss of the speech sample and the target parameter value of the key parameter combination to obtain a target speech processing model, and using the target speech processing model to process the target speech.

[0374] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0375] The key parameter combinations that determine the speech processing model include:

[0376] Obtain a key parameter combination to be selected, the key parameter combination to be selected includes multiple key parameters to be selected, each key parameter to be selected is set to multiple target discrete value points, and multiple sampling combinations are generated based on the multiple target discrete value points;

[0377] Testing multiple sampling combinations to obtain performance indicators of the sampling combinations;

[0378] Analyze the performance indicators of the sampling combination and determine the sensitivity index of each key parameter to be selected;

[0379] Based on the sensitivity index of each key parameter to be selected, a key parameter combination of the speech processing model is determined from the key parameter set to be selected.

[0380] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0381] Setting a parameter search space for each key parameter in the key parameter combination includes:

[0382] The Bayesian optimization method is used to search for the target discrete value points of each key parameter in the key parameter combination to find the parameter search space corresponding to each key parameter.

[0383] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0384] The method further includes: constructing a hybrid kernel function, wherein constructing the hybrid kernel function includes:

[0385] Determine a first weight value corresponding to the radial basis function kernel and a second weight value corresponding to the Mattern kernel;

[0386] Constructing a hybrid kernel function formula based on the radial basis function kernel, the first weight value, the Mattern kernel and the second weight value;

[0387] The distance values ​​between the multiple initial points are calculated, and the distance values ​​between the multiple initial points are input into the mixed covariance matrix to obtain the mixed covariance matrix value.

[0388] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0389] The mixed kernel function formula is as follows:

[0390] ;

[0391] ;

[0392] ;

[0393] in, represents the mixed kernel function, represents the radial basis function, represents the Mattern kernel function, Indicates the initial point With the initial point The Euclidean distance between represents the variance of the output, represents the length scale, is the first weight value, is the second weight value.

[0394] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0395] The hybrid kernel function proxy model is constructed based on the initial point evaluation results and the speech processing model, including:

[0396] Select prediction points based on adaptive sampling strategy;

[0397] Calculate the distance between the prediction point and the initial point, and calculate the prediction result of the prediction point based on the distance between the prediction point and the initial point, the prediction mean formula, and the prediction variance formula;

[0398] The predicted mean formula is as follows:

[0399] ;

[0400] The prediction variance formula is as follows:

[0401] ;

[0402] in, Indicates the predicted point The predicted mean of Indicates the predicted point The prediction variance of represents the covariance vector between the predicted point and the initial point, represents the inverse of the covariance matrix of the key parameters, is the evaluation result vector of the initial point, Indicates the predicted point The autocovariance of Represents the covariance between the forecast point and the initial point minus the covariance adjustment term.

[0403] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0404] Determining the acquisition function based on the hybrid kernel function proxy model includes: determining the acquisition function type, and determining the acquisition function according to the acquisition function type and the hybrid kernel function proxy model. The acquisition function is as follows:

[0405] ;

[0406] ;

[0407] in, Represents the predicted point and the expectation of improving the optimal point after sampling, Indicates the predicted point The predicted mean of Indicates the predicted point The predicted standard deviation of represents the known optimal evaluation result, represents the cumulative distribution function of the standard normal distribution, represents the probability density function of the standard normal distribution, represents the amount of improvement after normalization.

[0408] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0409] The selection of target points based on the acquisition function includes:

[0410] The predicted value of the prediction point is calculated using the acquisition function. The predicted value is the expectation of the improvement of the optimal point after sampling the prediction point.

[0411] Determine whether the predicted value of the predicted point is greater than a preset threshold;

[0412] The prediction point whose predicted value is greater than the preset threshold is taken as the target point.

[0413] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0414] The selection of prediction points based on the adaptive sampling strategy includes:

[0415] Divide the parameter search space of the key parameters to obtain multiple sub-parameter search spaces;

[0416] determining a level of the sub-parameter search space according to the number of prediction points determined as target points in the sub-parameter search space;

[0417] When the level in any sub-parameter search space is lower than a preset level, the prediction point sampling in the sub-parameter search space whose level is lower than the preset level is reduced.

[0418] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0419] Training the hybrid kernel function proxy model based on the parameter values ​​corresponding to the key parameter combination of the target point includes:

[0420] The parameter values ​​corresponding to the key parameter combinations of the target points are input into the proxy model for training to obtain the target point evaluation results;

[0421] Based on the target point evaluation result, the data set is updated, and the hybrid kernel function proxy model is trained using the updated data set to obtain an updated hybrid kernel function proxy model;

[0422] Construct an updated mixed covariance matrix, and calculate the prediction mean and prediction variance corresponding to the updated mixed kernel function proxy model based on the updated mixed covariance matrix and the updated data set;

[0423] The prediction result of the new prediction point is calculated using the prediction mean and prediction variance corresponding to the updated hybrid kernel function proxy model.

[0424] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0425] When the hybrid kernel function proxy model training termination conditions are met, the hybrid kernel function proxy model outputs the target parameter values ​​of the key parameter combinations including:

[0426] In response to the fact that the number of iterations of the hybrid kernel function proxy model is greater than the iteration threshold or that the predicted value of the prediction point is less than the preset value within a certain number of iterations of the hybrid kernel function proxy model, it is considered that the termination condition of the hybrid kernel function proxy model training is met, and the hybrid kernel function proxy model outputs the target parameter value of the key parameter combination in the iteration process.

[0427] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0428] The performance indicators of the sampling combination are analyzed to determine the sensitivity index of the key parameters to be selected, including:

[0429] Calculate the first sensitivity index and the second sensitivity index of the key parameter to be selected based on the performance index of the sampling combination, the first sensitivity index calculation formula, and the second sensitivity index calculation formula;

[0430] The first sensitivity index calculation formula is as follows:

[0431] ;

[0432] The second sensitivity index calculation formula is as follows:

[0433] ;

[0434] in, Indicates that due to input parameters Output caused by changes variance, Representation output The total variance of represents the batch size, Indicates exclusion of input parameters Output caused by changes in other parameters other than variance, Indicates the performance impact.

[0435] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be implemented by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0436] The technical features of the above embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0437] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for ordinary technicians in this field, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application.

Claims

1. A speech processing method, characterized in that: include: Processing multiple speech samples in the speech training set to obtain a score for each speech sample, and calculating the loss of the speech sample based on the score of each speech sample; Determine a key parameter combination of a speech processing model, the key parameter combination comprising a plurality of key parameters, and set a parameter search space for each key parameter in the key parameter combination; Selecting multiple initial points based on the parameter search space of each key parameter, evaluating the initial points, and constructing a hybrid kernel function proxy model based on the initial point evaluation results and the speech processing model, wherein the kernel function of the hybrid kernel function proxy model is a hybrid kernel function constructed based on a radial basis function kernel and a Mattern kernel; Determine an acquisition function according to the acquisition function type and the hybrid kernel function proxy model, and select target points based on the acquisition function; The hybrid kernel function proxy model is trained based on the parameter values ​​corresponding to the key parameter combination of the target point. When the hybrid kernel function proxy model training termination condition is met, the hybrid kernel function proxy model outputs the target parameter value of the key parameter combination. The speech processing model is trained according to the loss of the speech sample and the target parameter value of the key parameter combination to obtain a target speech processing model, and the target speech is processed using the target speech processing model.

2. The method according to claim 1, characterized in that The key parameter combination for determining the speech processing model includes: Acquire a key parameter combination to be selected, wherein the key parameter combination to be selected includes multiple key parameters to be selected, set each key parameter to be selected as multiple target discrete value points, and generate multiple sampling combinations based on the multiple target discrete value points; Testing multiple sampling combinations to obtain performance indicators of the sampling combinations; Analyze the performance indicators of the sampling combination to determine the sensitivity index of each key parameter to be selected; Based on the sensitivity index of each key parameter to be selected, a key parameter combination of the speech processing model is determined from the key parameter set to be selected.

3. The method according to claim 1, characterized in that The step of setting a parameter search space for each key parameter in the key parameter combination comprises: The Bayesian optimization method is used to search for the target discrete value points of each key parameter in the key parameter combination to find the parameter search space corresponding to each key parameter.

4. The method according to claim 1, characterized in that: The method further includes: constructing a hybrid kernel function, wherein constructing the hybrid kernel function includes: Determine a first weight value corresponding to the radial basis function kernel and a second weight value corresponding to the Mattern kernel; Constructing a hybrid kernel function formula based on the radial basis function kernel, the first weight value, the Mattern kernel, and the second weight value; The distance values ​​between the multiple initial points are calculated, and the distance values ​​between the multiple initial points are input into the mixed covariance matrix to obtain the mixed covariance matrix value.

5. The method according to claim 4, characterized in that The mixed kernel function formula is as follows: ； ； ； in, represents the mixed kernel function, represents the radial basis function, represents the Mattern kernel function, Indicates the initial point With the initial point The Euclidean distance between represents the variance of the output, l represents the length scale, is the first weight value, is the second weight value.

6. The method according to claim 1, characterized in that The construction of the hybrid kernel function proxy model based on the initial point evaluation result and the speech processing model includes: Select prediction points based on adaptive sampling strategy; Calculate the distance value between the predicted point and the initial point, and calculate the prediction result of the predicted point based on the distance value between the predicted point and the initial point, the prediction mean formula and the prediction variance formula; The predicted mean formula is as follows: ； The prediction variance formula is as follows: ； in, Indicates the predicted point The predicted mean of Indicates the predicted point The prediction variance of represents the covariance vector between the predicted point and the initial point, represents the inverse matrix of the covariance matrix of the key parameters, y is the evaluation result vector of the initial point, Indicates the predicted point The autocovariance of Represents the covariance between the forecast point and the initial point minus the covariance adjustment term.

7. The method according to claim 1, characterized in that Determining the acquisition function according to the acquisition function type and the hybrid kernel function proxy model includes: determining the acquisition function type, and determining the acquisition function according to the acquisition function type and the hybrid kernel function proxy model, wherein the acquisition function is as follows: ； ； in, Represents the predicted point and the expectation of improving the optimal point after sampling, Indicates the predicted point The predicted mean of Indicates the predicted point The predicted standard deviation of represents the known optimal evaluation result, represents the cumulative distribution function of the standard normal distribution, represents the probability density function of the standard normal distribution, and Z represents the improvement after standardization.

8. The method according to claim 1, characterized in that The target point selection based on the acquisition function includes: Calculate the predicted value of the predicted point by using the acquisition function, where the predicted value is the expectation of the improvement of the optimal point after sampling the predicted point; Determine whether the predicted value of the predicted point is greater than a preset threshold; The prediction point whose predicted value is greater than the preset threshold is taken as the target point.

9. The method according to claim 6, characterized in that The selecting of prediction points based on the adaptive sampling strategy includes: Divide the parameter search space of the key parameters to obtain multiple sub-parameter search spaces; determining a level of the sub-parameter search space according to the number of prediction points determined as target points in the sub-parameter search space; When the level in any sub-parameter search space is lower than a preset level, the prediction point sampling in the sub-parameter search space whose level is lower than the preset level is reduced.

10. The method according to claim 1, characterized in that The training of the hybrid kernel function proxy model based on the parameter values ​​corresponding to the key parameter combination of the target point includes: The parameter values ​​corresponding to the key parameter combinations of the target points are input into the proxy model for training to obtain the target point evaluation results; Based on the target point evaluation result, the data set is updated, and the hybrid kernel function proxy model is trained using the updated data set to obtain an updated hybrid kernel function proxy model; Construct an updated mixed covariance matrix, and calculate the prediction mean and prediction variance corresponding to the updated mixed kernel function proxy model based on the updated mixed covariance matrix and the updated data set; The prediction result of the new prediction point is calculated using the prediction mean and prediction variance corresponding to the updated hybrid kernel function proxy model.

11. The method according to claim 1, characterized in that: When the hybrid kernel function proxy model training termination condition is met, the hybrid kernel function proxy model outputs the target parameter value of the key parameter combination including: In response to the fact that the number of iterations of the hybrid kernel function proxy model is greater than the iteration threshold or that the predicted value of the prediction point is less than the preset value within a certain number of iterations of the hybrid kernel function proxy model, it is considered that the termination condition of the hybrid kernel function proxy model training is met, and the hybrid kernel function proxy model outputs the target parameter value of the key parameter combination in the iteration process.

12. The method according to claim 2, characterized in that: The performance indicators of the sampling combination are analyzed to determine the sensitivity index of the key parameters to be selected, including: Calculate the first sensitivity index and the second sensitivity index of the key parameter to be selected based on the performance index of the sampling combination, the first sensitivity index calculation formula, and the second sensitivity index calculation formula; The calculation formula of the first sensitivity index is as follows: ； The second sensitivity index calculation formula is as follows: ； in, represents the variance of output Y caused by the change of input parameter i, V(Y) represents the total variance of output Y, represents the batch size, represents the variance of output Y caused by changes in other parameters except input parameter i, Indicates the performance impact.

13. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 12 are implemented.

14. A computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 12 are implemented.

15. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 12 are implemented.

Images