A model training method and device, a chip, and a module equipment
By introducing a second speech quality assessment result and a weighted loss function into the training of the speech quality assessment model, the problem of subjective bias in the training of existing models is solved, and more accurate speech quality assessment is achieved.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- UNISOC CHONGQING TECH CO LTD
- Filing Date
- 2022-10-20
- Publication Date
- 2026-05-19
AI Technical Summary
Existing speech quality assessment models use average opinion scores as labels for training datasets, which leads to subjective bias during training and makes it impossible to accurately assess speech quality.
By introducing a second speech quality assessment result into the model training process, combining the first speech quality assessment result with the speech quality assessment result label, adjusting the model parameters, gradually weakening the influence of subjective factors, and using a weighted sum loss function to optimize the model training process.
The trained model can more objectively and accurately assess speech quality, reduce the influence of subjective factors, and improve assessment accuracy.
Smart Images
Figure CN115662472B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of computers, and more particularly to a model training method, apparatus, chip, and module device. Background Technology
[0002] With the development of wireless communication technology, real-time voice communication has become possible. However, during voice communication, voice quality is often affected by factors such as environmental noise, room reverberation, interference, and network packet loss, which can hinder voice communication. In order to monitor the voice quality of communication networks, online system providers need a tool that can evaluate voice quality in real time to detect communication quality issues.
[0003] Existing tools for assessing speech quality typically employ non-invasive speech quality assessment models that use mean opinion scores (MOSS) as labels for the training dataset. Since MOSS is the average of scores given by humans after assessing speech quality, the method of training the model based on this MOSS is subject to subjective bias. This results in speech quality assessment models that cannot accurately evaluate speech quality. Summary of the Invention
[0004] This application provides a model training method, apparatus, chip, and module device, enabling the trained model to more objectively and accurately evaluate speech quality.
[0005] Firstly, this application provides a model training method, which includes:
[0006] Based on the model parameters of the first evaluation model obtained in the i-th iteration training and the model parameters of the second evaluation model obtained in the (i-1)-th iteration training, the model parameters of the second evaluation model are adjusted in the i-th iteration training, where i is an integer greater than or equal to 1.
[0007] In the (i+1)th iteration of training, the sample audio data is input into the first evaluation model and the second evaluation model respectively, and the first speech quality evaluation result output by the first evaluation model and the second speech quality evaluation result output by the second evaluation model are obtained.
[0008] In the (i+1)th iteration of training, the model parameters of the first evaluation model are adjusted based on the first speech quality evaluation result, the second speech quality evaluation result, and the speech quality evaluation result label.
[0009] Typically, speech quality assessment models are trained based on speech quality assessment result labels. However, these labels contain subjective elements, which can cause the trained model to inaccurately assess speech quality. This application introduces a second speech quality assessment result to mitigate the subjective factors introduced by the speech quality assessment result labels, enabling the trained model to assess speech quality more objectively and accurately.
[0010] In one possible implementation, during the (i+1)th iteration of training, the model parameters of the first evaluation model are adjusted based on the first speech quality assessment result, the second speech quality assessment result, and the speech quality assessment result label. This includes: determining a first error value based on the speech quality assessment result label and the first speech quality assessment result during the (i+1)th iteration of training; determining a second error value based on the first speech quality assessment result and the second speech quality assessment result during the (i+1)th iteration of training; determining the total loss value by the weighted sum of the first error value and the second error value during the (i+1)th iteration of training; and adjusting the model parameters of the first evaluation model based on the total loss value during the (i+1)th iteration of training.
[0011] By using the labels of the speech quality assessment results and the output of the second speech quality assessment model, the first assessment model is trained together with the second speech quality assessment results, so that subjective evaluations do not have an excessive impact on the training of the first assessment model.
[0012] In one possible implementation, the weight of the first error value is negatively correlated with the number of training iterations, while the weight of the second error value is positively correlated with the number of training iterations.
[0013] By increasing the number of training iterations, the labels of speech quality assessment results are weakened, and subjective factors are gradually reduced, so that the trained model can more objectively and accurately assess speech quality.
[0014] In one possible implementation, the total loss value L satisfies the following formula: L=(1-a)L1+aL2, where L1 is the first error value, L2 is the second error value, a is the weight corresponding to the second error value L2, and (1-a) is the weight corresponding to the first error value L1.
[0015] In one possible implementation, the weight 'a' corresponding to the second error value L2 satisfies the following formula: a = exp(-5*(1-i / T)) 2 ), where T is a preset value and i is the number of iterations for training.
[0016] By increasing the number of training iterations, the labels of speech quality assessment results are weakened, and subjective factors are gradually reduced, so that the trained model can more objectively and accurately assess speech quality.
[0017] In one possible implementation, the model parameters of the second evaluation model obtained in the i-th iteration training are a weighted sum of the model parameters of the first evaluation model obtained in the i-th iteration training and the model parameters of the second evaluation model obtained in the (i-1)-th iteration training.
[0018] In one possible implementation, the model parameters of the second evaluation model obtained from the i-th iteration of training... Satisfy the following formula: in The model parameters of the second evaluation model obtained from the (i-1)th iteration of training are... Let b be the model parameters of the first evaluation model obtained from the i-th iteration of training. i represents the sliding weight coefficients corresponding to the i-th iteration of training.
[0019] In one possible implementation, the sliding weight coefficient b corresponds to the i-th iteration of training. i Satisfy the following formula: b i =min(i / T,0.99), where i is the number of training iterations and T is a preset value.
[0020] Initially, the model parameters of the second evaluation model are the same as those of the first evaluation model. In the early stages, the model parameters of the first evaluation model are mainly based on the labels of the speech quality evaluation results. As the number of iterations increases, the weight of the model parameters of the most recent first evaluation model is increased, thereby weakening the influence of the model parameters of the first evaluation model in the early stages, thus gradually weakening subjective factors.
[0021] Secondly, this application provides a model training apparatus, which includes a unit for performing the method described in the first aspect above.
[0022] Thirdly, this application provides a chip including a processor and a communication interface, wherein the processor is configured to cause the chip to perform the method described in the first aspect above.
[0023] Fourthly, this application provides a module device, which includes a communication module, a power module, a storage module, and a chip, wherein: the power module is used to provide electrical energy to the module device; the storage module is used to store data and instructions; the communication module is used to perform internal communication within the module device or to enable communication between the module device and external devices; and the chip is used to execute the method described in the first aspect above.
[0024] Fifthly, embodiments of this application disclose an electronic device including a memory and a processor. The memory is used to store a computer program, the computer program including program instructions, and the processor is configured to invoke the program instructions to execute the method described in the first aspect above.
[0025] In a sixth aspect, this application provides a computer-readable storage medium storing computer-readable instructions that, when executed on a communication device, cause the communication device to perform the method described in the first aspect.
[0026] In a seventh aspect, this application provides a computer program or computer program product, including code or instructions that, when run on a computer, cause the computer to perform the method described in the first aspect above. Attached Figure Description
[0027] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0028] Figure 1 This is a schematic flowchart of a model training method provided in an embodiment of this application;
[0029] Figure 2 This is a schematic diagram of an embodiment of a model training method provided in this application;
[0030] Figure 3 This is a schematic diagram of the structure of a model training device provided in an embodiment of this application;
[0031] Figure 4 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application;
[0032] Figure 5 This is a schematic diagram of the structure of a module device provided in an embodiment of this application. Detailed Implementation
[0033] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0034] The terminology used in the following embodiments of this application is for the purpose of describing particular embodiments only and is not intended to be limiting of this application. As used in the specification and appended claims of this application, the singular expressions “a,” “an,” “the,” “the,” “the,” and “this” are intended to include the plural expressions as well, unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used in this application refers to and includes any or all possible combinations of one or more of the listed items.
[0035] It should be noted that the terms "first," "second," "third," etc., used in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the term "comprising" and any variations thereof are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or server that includes a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to these processes, methods, products, or devices.
[0036] Please see Figure 1 , Figure 1 This is a flowchart of a model training method provided in an embodiment of this application, which includes steps 101-103. Figure 1 The execution subject of the method shown can be an electronic device, such as a desktop computer, laptop, tablet, wireless terminal in a smart grid, wireless terminal in transportation safety, wireless terminal in a smart city, wireless terminal in a smart home, wearable electronic device, etc. Alternatively, the execution subject can be a chip in an electronic device. Alternatively, the execution subject can be a server, which can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDN), and big data and artificial intelligence platforms. The embodiments in this application do not uniquely limit the execution subject of the method; the corresponding execution subject can also be any device, chip, or software module capable of implementing the method, or a combination thereof. For ease of subsequent description, Figure 1The method shown is implemented using an electronic device as an example. Wherein:
[0037] 101. The electronic device adjusts the model parameters of the second evaluation model in the i-th iteration training based on the model parameters of the first evaluation model obtained in the i-th iteration training and the model parameters of the second evaluation model obtained in the (i-1)-th iteration training, where i is an integer greater than or equal to 1.
[0038] In this model, the first and second evaluation models are Siamese network models, and initially, the model parameters in the first and second evaluation models are the same. The first evaluation model can be a Convolutional Neural Network (CNN), a Recurrent Neural Network (RNN), or other neural network models. Similarly, the second evaluation model can also be a Convolutional Neural Network (CNN), a Recurrent Neural Network (RNN), or other neural network models. Taking a CNN as an example, a CNN mainly consists of convolutional layers, pooling layers, and fully connected layers. The parameters in a CNN mainly include the kernel width, kernel height, and kernel weights. Therefore, if the first and second evaluation models are CNN neural network models, then the model parameters in both models can be the aforementioned kernel width, kernel height, and kernel weights. It should be noted that the first and second evaluation models can also be other network models; no restrictions are placed here.
[0039] For example, when i is 4, the model parameters of the second evaluation model in the fourth iteration training are obtained based on the model parameters of the first evaluation model obtained in the fourth iteration training and the model parameters of the second evaluation model obtained in the third iteration training.
[0040] Optionally, when i is 1, the model parameters of the second evaluation model in the first iteration training are obtained based on the model parameters of the first evaluation model obtained in the first iteration training and the model parameters of the second evaluation model obtained in the 0th iteration training. The second evaluation model trained in the 0th iteration is the initial second evaluation model, and the model parameters of the initial second evaluation model are the same as the model parameters of the initial first evaluation model.
[0041] In one possible embodiment, the model parameters of the second evaluation model obtained in the i-th iteration training are a weighted sum of the model parameters of the first evaluation model obtained in the i-th iteration training and the model parameters of the second evaluation model obtained in the (i-1)-th iteration training.
[0042] The weighted sum of the model parameters of the first evaluation model obtained in the i-th iteration and the model parameters of the second evaluation model obtained in the (i-1)-th iteration is the model parameters of the first evaluation model obtained in the i-th iteration multiplied by the corresponding weights, plus the model parameters of the second evaluation model obtained in the (i-1)-th iteration multiplied by the corresponding weights.
[0043] In one possible embodiment, the model parameters of the second evaluation model correspond one-to-one with the model parameters of the first evaluation model, and any model parameter of the second evaluation model is only related to the model parameters of its corresponding first evaluation model.
[0044] For example, the model parameters of the first evaluation model are A1, A2, and A3. The model parameters of the second evaluation model are B1, B2, and B3. Here, A1 of the first evaluation model corresponds to B1 of the second evaluation model, A2 of the first evaluation model corresponds to B2 of the second evaluation model, and A3 of the first evaluation model corresponds to B3 of the second evaluation model. The B1 of the second evaluation model obtained in the i-th iteration is the weighted sum of A1 of the first evaluation model obtained in the i-th iteration and B1 of the second evaluation model obtained in the (i-1)-th iteration. That is, the B1 of the second evaluation model obtained in the i-th iteration is only related to A1 and is independent of the other model parameters in the first evaluation model.
[0045] In one possible embodiment, the model parameters of the second evaluation model obtained from the i-th iteration of training... Satisfy the following formula: in The model parameters of the second evaluation model obtained from the (i-1)th iteration of training are... Let b be the model parameters of the first evaluation model obtained from the i-th iteration of training. i represents the sliding weight coefficients corresponding to the i-th iteration of training.
[0046] The weights of the model parameters of the second evaluation model obtained in the (i-1)th iteration training are inversely proportional to the weights of the model parameters of the first evaluation model obtained in the i-th iteration training. In other words, the greater the weight of the model parameters of the first evaluation model obtained in the i-th iteration training, the smaller the weight of the model parameters of the second evaluation model obtained in the (i-1)th iteration training.
[0047] In one possible embodiment, the sliding weight coefficient b corresponding to the i-th iteration training is... i Satisfy the following formula: b i =min(i / T,0.99), where i is the number of training iterations and T is a preset value.
[0048] Among them, b iThe maximum value is 0.99, and bi is directly proportional to the number of training iterations, that is, b i The value increases with the number of training iterations. For example, when T is 10, b5 is 5 / 10 = 0.5 in the 5th training iteration, and b6 is 6 / 10 = 0.6 in the 6th training iteration. It is clear that b6 in the 6th training iteration is greater than b5 in the 5th training iteration.
[0049] By increasing the weight of the model parameters of the first evaluation model obtained in the i-th iteration as the number of training iterations increases, the model parameters of the first evaluation model from more recent iterations have a greater impact on the model parameters of the second evaluation model obtained in the i-th iteration in later iterations, while the model parameters of the first evaluation model from more distant iterations have a smaller impact on the model parameters of the second evaluation model obtained in the i-th iteration. Since the early iterations are supervised primarily by the speech quality evaluation result labels, which introduces subjective factors, this method can gradually eliminate these subjective factors during iterative training.
[0050] In one possible embodiment, the model parameters of the second evaluation model obtained from the i-th iteration of training... Satisfy the following formula:
[0051]
[0052] in, This represents the model parameters of the first evaluation model obtained in the nth iteration of training, where n is a positive integer from 0 to i, and β... n This represents the weights corresponding to the model parameters of the first evaluation model obtained in the nth iteration of training. This formula represents the weighted sum and average of the model parameters of the first evaluation model obtained in the first i iterations of training. Where β... n The initial value of β is 0, and after each training iteration... n The coefficient is increased by a preset step size. For example, if the preset step size is 0.1, then β1 for the first training iteration is 0 + 0.1 = 0.1, β2 for the second training iteration is 0.1 + 0.1 = 0.2, and β3 for the third training iteration is 0.2 + 0.1 = 0.3.
[0053] For example, the model parameters of the first evaluation model obtained in the first iteration of training are 4, the model parameters of the first evaluation model obtained in the second iteration of training are 4.5, the model parameters of the first evaluation model obtained in the third iteration of training are 4.7, and the model parameters of the first evaluation model obtained in the fourth iteration of training are 5. β1 = 0.1, β2 = 0.2, β3 = 0.3, β4 = 0.4. The model parameters of the second evaluation model obtained in the fourth iteration of training are:
[0054]
[0055] As the number of training iterations increases, β n This increases the influence of the model parameters of the first evaluation model in the most recent training iteration on the model parameters of the second evaluation model obtained in the i-th training iteration. For example, the influence of the model parameters of the first evaluation model in the 5th training iteration on the model parameters of the second evaluation model obtained in the 5th training iteration is greater than the influence of the model parameters of the first evaluation model in the 4th training iteration on the model parameters of the second evaluation model obtained in the 5th training iteration. Consequently, the trained model can more objectively and accurately evaluate speech quality.
[0056] 102. In the (i+1)th iteration of training, the electronic device inputs the sample audio data into the first evaluation model and the second evaluation model respectively, and obtains the first speech quality evaluation result output by the first evaluation model and the second speech quality evaluation result output by the second evaluation model.
[0057] The outputs of both the first and second evaluation models are predicted values obtained from evaluating the sample audio data. The first and second speech quality evaluation results can be scores, which represent the speech quality of the sample audio data; a higher score indicates better speech quality. Alternatively, they can be grades, which also represent the speech quality of the sample audio data; a higher grade indicates better speech quality. Speech quality refers to aspects such as the clarity of the sample audio data.
[0058] 103. In the (i+1)th iteration of training, the electronic device adjusts the model parameters of the first evaluation model based on the first speech quality evaluation result, the second speech quality evaluation result, and the speech quality evaluation result label.
[0059] The voice quality assessment result label can be the Mean Opinion Score (MOS). This MOS is the average score obtained by multiple judges evaluating the sample audio data. The MOS typically ranges from 0 to 5, with 5 being the highest score and 0 being the lowest. For example, if three judges score the sample audio data as 3.5, 4, and 4.2, then the corresponding MOS for that sample audio data is (3.5 + 4 + 4.2) / 3 = 3.9.
[0060] In one possible embodiment, the electronic device adjusts the model parameters of the first evaluation model in the (i+1)th iteration of training based on the first speech quality evaluation result, the second speech quality evaluation result, and the speech quality evaluation result label. This includes: determining a first error value based on the speech quality evaluation result label and the first speech quality evaluation result in the (i+1)th iteration of training; determining a second error value based on the first speech quality evaluation result and the second speech quality evaluation result in the (i+1)th iteration of training; determining the total loss value by weighting the first error value and the second error value in the (i+1)th iteration of training; and adjusting the model parameters of the first evaluation model based on the total loss value in the (i+1)th iteration of training.
[0061] The first error value can be the root mean square error between the speech quality assessment result label and the first speech quality assessment result, or it can be the variance between the two results, or it can be any other type of difference; there are no restrictions here. Similarly, the second error value can be the root mean square error between the first and second speech quality assessment results, or it can be the variance between the two results, or it can be any other type of difference; there are no restrictions here. The weighted sum of the first and second error values can be the first error value multiplied by its corresponding weight plus the second error value multiplied by its corresponding weight.
[0062] For example, in the fourth iteration of training, the label of the speech quality assessment result corresponding to the sample audio data is 4.2. When the sample audio data is input into the first evaluation model, the first speech quality assessment result is 3.9. When the sample audio data is input into the second evaluation model, the second speech quality assessment result is 4.4. The first error value is 0.3, the second error value is 0.2, and the total loss value is the weighted sum between 0.3 and 0.2.
[0063] By using the labels of the speech quality assessment results and the output of the second speech quality assessment model, the first assessment model is trained together with the second speech quality assessment results, so that subjective evaluations do not have an excessive impact on the training of the first assessment model.
[0064] In one possible embodiment, the weight of the first error value is negatively correlated with the number of iterations, and the weight of the second error value is positively correlated with the number of iterations.
[0065] As the number of training iterations increases, the proportion of the first error value in the total loss gradually decreases, while the proportion of the second error value gradually increases. For example, in the third training iteration, the first error value accounts for 40% of the total loss, and the second error value accounts for 60%; in the fourth training iteration, the first error value accounts for 30% and the second error value accounts for 70%; and in the fifth training iteration, the first error value accounts for 20% and the second error value accounts for 80%.
[0066] By increasing the number of training iterations, the labels of speech quality assessment results are weakened, and subjective factors are gradually reduced, so that the trained model can more objectively and accurately assess speech quality.
[0067] In one possible embodiment, the total loss value L satisfies the following formula: L=(1-a)L1+aL2, where L1 is the first error value, L2 is the second error value, a is the weight corresponding to the second error value L2, and (1-a) is the weight corresponding to the first error value L1.
[0068] In one possible embodiment, the weight 'a' corresponding to the second error value L2 satisfies the following formula: a = exp(-5*(1-i / T)) 2 ), where T is a preset value and i is the number of iterations for training.
[0069] For example, if T is 10, then during the 5th training iteration, a = exp(-5*(1-5 / 10)). 2 =0.29.
[0070] Optionally, the weight 'a' corresponding to the second error value L2 is increased by a preset step size after each training iteration. For example, if the preset step size is 0.1, in the third training iteration, the weight 'a' is 0.6. After the third training iteration, the weight 'a' is increased by 0.1 for use in the fourth training iteration, so in the fourth training iteration, the weight 'a' is 0.7. It should be noted that the weight 'a' can also be increased by a preset step size before the start of each training iteration.
[0071] For example, in the third iteration of training, the preset step size is 0.1, the first error value is 0.3, the second error value is 0.2, and the weight 'a' corresponding to the second error value L2 is 0.6. Therefore, the weight corresponding to the first error value L1 is (1-0.6) = 0.4. The total loss value L = (1-0.6)^0.3 + 0.6 * 0.2 = 0.24. After the third iteration of training, the weight 'a' corresponding to the second error value L2 is 0.6 + 0.1.
[0072] By increasing the number of training iterations, the labels of speech quality assessment results are weakened, and subjective factors are gradually reduced, so that the trained model can more objectively and accurately assess speech quality.
[0073] In one possible implementation, before iterative training, various parameters need to be initialized, such as the weight 'a' and the sliding weight coefficient mentioned above.
[0074] The following is combined with Figure 2 For a further explanation of this iterative training process, please refer to [link / reference]. Figure 2 , Figure 2 For the i-th iteration of the training process, the electronic device first needs to determine the model parameters of the second evaluation model. These parameters are based on the model parameters of the first evaluation model, as described in step 101 above. Then, the electronic device inputs sample audio data into the first and second evaluation models to obtain the first and second speech quality evaluation results, as described in step 102 above. Based on the first and second speech quality evaluation results and the MOS label (speech quality evaluation result label), the electronic device determines the total loss value. Finally, it updates the model parameters of the first evaluation model based on the total loss value, as described in step 103 above.
[0075] Please see Figure 3 , Figure 3 This is a schematic diagram of a model training device provided in an embodiment of the present invention. The model training device can be an electronic device or a device with electronic device functions (such as a chip). Specifically, as shown... Figure 3 As shown, the model training device 300 may include:
[0076] The adjustment unit 301 is used to adjust the model parameters of the second evaluation model in the i-th iteration training based on the model parameters of the first evaluation model obtained in the i-th iteration training and the model parameters of the second evaluation model obtained in the (i-1)-th iteration training, where i is an integer greater than or equal to 1.
[0077] Processing unit 302 is used to input sample audio data into the first evaluation model and the second evaluation model respectively in the (i+1)th iteration training to obtain the first speech quality evaluation result output by the first evaluation model and the second speech quality evaluation result output by the second evaluation model.
[0078] The adjustment unit 301 is also used to adjust the model parameters of the first evaluation model based on the first speech quality evaluation result, the second speech quality evaluation result, and the speech quality evaluation result label in the (i+1)th iteration training.
[0079] In one possible implementation, the processing unit 302 is further configured to determine a first error value based on the speech quality evaluation result label and the first speech quality evaluation result in the (i+1)th iteration of training; the processing unit 302 is further configured to determine a second error value based on the first speech quality evaluation result and the second speech quality evaluation result in the (i+1)th iteration of training; the processing unit 302 is further configured to determine the total loss value by weighting the first error value and the second error value in the (i+1)th iteration of training; and the adjustment unit is further configured to adjust the model parameters of the first evaluation model based on the total loss value in the (i+1)th iteration of training.
[0080] In one possible implementation, the weight of the first error value is negatively correlated with the number of training iterations, while the weight of the second error value is positively correlated with the number of training iterations.
[0081] In one possible implementation, the total loss value L satisfies the following formula: L=(1-a)L1+aL2, where L1 is the first error value, L2 is the second error value, a is the weight corresponding to the second error value L2, and (1-a) is the weight corresponding to the first error value L1.
[0082] In one possible implementation, the model parameters of the second evaluation model obtained in the i-th iteration training are a weighted sum of the model parameters of the first evaluation model obtained in the i-th iteration training and the model parameters of the second evaluation model obtained in the (i-1)-th iteration training.
[0083] In one possible implementation, the model parameters of the second evaluation model obtained from the i-th iteration of training... Satisfy the following formula: in The model parameters of the second evaluation model obtained from the (i-1)th iteration of training are... Let b be the model parameters of the first evaluation model obtained from the i-th iteration of training. i represents the sliding weight coefficients corresponding to the i-th iteration of training.
[0084] In one possible implementation, the sliding weight coefficient b corresponds to the i-th iteration of training. i Satisfy the following formula: b i =min(i / T,0.99), where i is the number of training iterations and T is a preset value.
[0085] This application also provides a chip that can perform the relevant steps of the electronic device in the foregoing method embodiments. The chip includes a processor and a communication interface, and the processor is configured to cause the chip to perform the following operations:
[0086] Based on the model parameters of the first evaluation model obtained in the i-th iteration training and the model parameters of the second evaluation model obtained in the (i-1)-th iteration training, the model parameters of the second evaluation model are adjusted in the i-th iteration training, where i is an integer greater than or equal to 1; in the (i+1)-th iteration training, the sample audio data are input into the first evaluation model and the second evaluation model respectively to obtain the first speech quality evaluation result output by the first evaluation model and the second speech quality evaluation result output by the second evaluation model; in the (i+1)-th iteration training, the model parameters of the first evaluation model are adjusted based on the first speech quality evaluation result, the second speech quality evaluation result, and the speech quality evaluation result label.
[0087] In one possible implementation, during the (i+1)th iteration of training, the model parameters of the first evaluation model are adjusted based on the first speech quality assessment result, the second speech quality assessment result, and the speech quality assessment result label. This includes: determining a first error value based on the speech quality assessment result label and the first speech quality assessment result during the (i+1)th iteration of training; determining a second error value based on the first speech quality assessment result and the second speech quality assessment result during the (i+1)th iteration of training; determining the total loss value by the weighted sum of the first error value and the second error value during the (i+1)th iteration of training; and adjusting the model parameters of the first evaluation model based on the total loss value during the (i+1)th iteration of training.
[0088] In one possible implementation, the weight of the first error value is negatively correlated with the number of training iterations, while the weight of the second error value is positively correlated with the number of training iterations.
[0089] In one possible implementation, the total loss value L satisfies the following formula: L=(1-a)L1+aL2, where L1 is the first error value, L2 is the second error value, a is the weight corresponding to the second error value L2, and (1-a) is the weight corresponding to the first error value L1.
[0090] In one possible implementation, the model parameters of the second evaluation model obtained in the i-th iteration training are a weighted sum of the model parameters of the first evaluation model obtained in the i-th iteration training and the model parameters of the second evaluation model obtained in the (i-1)-th iteration training.
[0091] In one possible implementation, the model parameters of the second evaluation model obtained from the i-th iteration of training... Satisfy the following formula: in The model parameters of the second evaluation model obtained from the (i-1)th iteration of training are... Let b be the model parameters of the first evaluation model obtained from the i-th iteration of training. i represents the sliding weight coefficients corresponding to the i-th iteration of training.
[0092] In one possible implementation, the sliding weight coefficient b corresponds to the i-th iteration of training. i Satisfy the following formula: b i =min(i / T,0.99), where i is the number of training iterations and T is a preset value.
[0093] Please see Figure 4 , Figure 4 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. The electronic device 400 may include a memory 401 and a processor 402. Optionally, it may also include a communication interface 403. The memory 401, processor 402, and communication interface 403 are connected via one or more communication buses. The communication interface 403 is controlled by the processor 402 for sending and receiving information.
[0094] Memory 401 may include read-only memory and random access memory, and provides instructions and data to processor 402. A portion of memory 401 may also include non-volatile random access memory.
[0095] Communication interface 403 is used to receive or send data.
[0096] Processor 402 can be a Central Processing Unit (CPU), but it can also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor; optionally, processor 402 can also be any conventional processor. Wherein:
[0097] Memory 401 is used to store program instructions.
[0098] Processor 402 is used to call program instructions stored in memory 401.
[0099] The processor 402 calls the program instructions stored in the memory 401, causing the electronic device 400 to execute the method performed by the electronic device in the above method embodiment.
[0100] like Figure 5 As shown, Figure 5This is a schematic diagram of the structure of a module device provided in an embodiment of this application. The module device 500 can perform the relevant steps of the electronic device in the aforementioned method embodiment. The module device 500 includes: a communication module 501, a power module 502, a storage module 503, and a chip 504.
[0101] The power module 502 is used to provide power to the module device; the storage module 503 is used to store data and instructions; the communication module 501 is used for internal communication within the module device or for communication between the module device and external devices; and the chip 504 is used to execute the methods performed by the electronic device in the above method embodiments.
[0102] It should be noted that for any aspects of the devices, chips, electronic devices, and module devices in this application that are not mentioned in the previous embodiments, as well as the specific implementation methods of each step, please refer to [link to relevant documentation]. Figures 1-2 The embodiments shown and the foregoing content will not be repeated here.
[0103] This application also provides a computer-readable storage medium storing instructions that, when executed on a processor, enable the implementation of the method flow described in the above method embodiments.
[0104] This application also provides a computer program product, which, when run on a processor, enables the implementation of the method flow described in the above method embodiments.
[0105] This application also provides a chip that can execute the relevant steps of the xx device in the foregoing method embodiments. The chip includes a processor and a communication interface, and the processor is configured to cause the chip to perform the following operations:
[0106] Regarding the modules / units included in the various devices and products described in the above embodiments, they can be software modules / units, hardware modules / units, or a combination of both. For example, for various devices and products applied to or integrated into a chip, all of their modules / units can be implemented using hardware methods such as circuits, or at least some modules / units can be implemented using software programs that run on the chip's integrated processor, while the remaining (if any) modules / units can be implemented using hardware methods such as circuits. For various devices and products applied to or integrated into a chip module, all of their modules / units can be implemented using hardware methods such as circuits. Different modules / units can be located in the same part (e.g., chip, circuit module, etc.) or different components of the chip module, or at least some modules / units... It can be implemented using software programs that run on the processor integrated within the chip module. The remaining (if any) modules / units can be implemented using hardware methods such as circuits. For various devices and products applied to or integrated into the terminal, the modules / units they contain can all be implemented using hardware methods such as circuits. Different modules / units can be located in the same component (e.g., chip, circuit module, etc.) or different components within the terminal. Alternatively, at least some modules / units can be implemented using software programs that run on the processor integrated within the terminal, while the remaining (if any) modules / units can be implemented using hardware methods such as circuits.
[0107] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, as some operations can be performed in other orders or simultaneously according to this application. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions and modules involved are not necessarily essential to this application.
[0108] The descriptions of the various embodiments provided in this application can be referenced mutually. Each embodiment has its own emphasis, and parts not described in detail in a certain embodiment can be referred to the relevant descriptions of other embodiments. For the sake of convenience and brevity, for example, the functions and operations of the various devices and equipment provided in the embodiments of this application can be referred to the relevant descriptions of the method embodiments of this application. The method embodiments and the device embodiments can also be referenced, combined or cited from each other.
[0109] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application.
Claims
1. A model training method, characterized in that, The method includes: Based on the model parameters of the first evaluation model obtained in the i-th iteration training and the model parameters of the second evaluation model obtained in the (i-1)-th iteration training, the model parameters of the second evaluation model are adjusted in the i-th iteration training, where i is an integer greater than or equal to 1; the first evaluation model and the second evaluation model are Siamese network models; initially, the model parameters in the first evaluation model are the same as the model parameters in the second evaluation model; the model parameters of the second evaluation model correspond one-to-one with the model parameters of the first evaluation model, and any model parameter of the second evaluation model is related to the model parameter of its corresponding first evaluation model; In the (i+1)th iteration of training, the sample audio data is input into the first evaluation model and the second evaluation model respectively, and the first speech quality evaluation result output by the first evaluation model and the second speech quality evaluation result output by the second evaluation model are obtained. In the (i+1)th iteration of training, a first error value is determined based on the speech quality assessment result label and the first speech quality assessment result; In the (i+1)th iteration of training, a second error value is determined based on the first speech quality assessment result and the second speech quality assessment result; In the (i+1)th iteration of training, the weighted sum of the first error value and the second error value is determined as the total loss value; In the (i+1)th iteration of training, the model parameters of the first evaluation model are adjusted based on the total loss value; the weight of the first error value is negatively correlated with the number of iterations, and the weight of the second error value is positively correlated with the number of iterations.
2. The method according to claim 1, characterized in that, The total loss value L satisfies the following formula: L=(1-a)L1+aL2, where L1 is the first error value, L2 is the second error value, a is the weight corresponding to the second error value L2, and (1-a) is the weight corresponding to the first error value L1.
3. The method according to claim 2, characterized in that, The weight 'a' corresponding to the second error value L2 satisfies the following formula: a Where T is a preset value and i is the number of iterations for training.
4. The method according to any one of claims 1-3, characterized in that, The model parameters of the second evaluation model obtained in the i-th iteration training are the weighted sum of the model parameters of the first evaluation model obtained in the i-th iteration training and the model parameters of the second evaluation model obtained in the (i-1)-th iteration training.
5. The method according to claim 4, characterized in that, The model parameters of the second evaluation model obtained from the i-th iteration of training Satisfy the following formula: , wherein The model parameters of the second evaluation model obtained from the (i-1)th iteration training are... The model parameters of the first evaluation model obtained from the i-th iteration of training are: represents the sliding weight coefficients corresponding to the i-th iteration of training.
6. The method according to claim 5, characterized in that, The sliding weight coefficients corresponding to the i-th iteration training Satisfy the following formula: Where i is the number of iterations for training, and T is a preset value.
7. A model training device, characterized in that, The device includes: An adjustment unit is used to adjust the model parameters of the second evaluation model in the i-th iteration of training, based on the model parameters of the first evaluation model obtained in the i-th iteration and the model parameters of the second evaluation model obtained in the (i-1)-th iteration, where i is an integer greater than or equal to 1; the first evaluation model and the second evaluation model are Siamese network models; initially, the model parameters in the first evaluation model are the same as the model parameters in the second evaluation model; the model parameters of the second evaluation model correspond one-to-one with the model parameters of the first evaluation model, and any model parameter of the second evaluation model is related to the model parameter of its corresponding first evaluation model; The processing unit is used to input sample audio data into the first evaluation model and the second evaluation model respectively in the (i+1)th iteration training to obtain the first speech quality evaluation result output by the first evaluation model and the second speech quality evaluation result output by the second evaluation model. The adjustment unit is further configured to: determine a first error value based on the speech quality assessment result label and the first speech quality assessment result in the (i+1)th iteration training; determine a second error value based on the first speech quality assessment result and the second speech quality assessment result in the (i+1)th iteration training; determine the total loss value by weighting the first error value and the second error value in the (i+1)th iteration training; and adjust the model parameters of the first evaluation model based on the total loss value in the (i+1)th iteration training; wherein the weight of the first error value is negatively correlated with the number of iteration training times, and the weight of the second error value is positively correlated with the number of iteration training times.
8. A chip, characterized in that, The chip includes a processor and a communication interface, the processor being configured to cause the chip to perform the method as described in any one of claims 1 to 6.
9. A module device, characterized in that, The module device includes a communication module, a power module, a storage module, and a chip, wherein: The power module is used to provide electrical energy to the module device; The storage module is used to store data and instructions; The communication module is used for internal communication within the module device, or for communication between the module device and external devices; The chip is used to perform the method as described in any one of claims 1 to 6.
10. An electronic device, characterized in that, The electronic device includes a memory and a processor. The memory is used to store a computer program, the computer program including program instructions, and the processor is configured to invoke the program instructions to cause the electronic device to perform the method as described in any one of claims 1 to 6.
11. A computer-readable storage medium, characterized in that, The computer storage medium stores computer-readable instructions, which, when executed on the communication device, cause the communication device to perform the method of any one of claims 1 to 6.