Speech recognition model training method and device and speech recognition method and device

Abstract

The embodiment of the invention provides a voice recognition model training method and device and a voice recognition method and device. The voice recognition model training method comprises the steps that preference audio data and non-preference audio data associated with an initial voice recognition model are acquired; selecting a to-be-fine-tuned network layer in the initial speech recognition model, configuring a low-rank adapter for a weight matrix of the to-be-fine-tuned network layer, and obtaining an intermediate speech recognition model according to a configuration result; predicting the preference audio data and the non-preference audio data by using the intermediate voice recognition model to perform voice recognition, and calculating a target loss value according to a voice recognition result; and updating parameters associated with the low-rank adapter in the to-be-fine-tuned network layer based on the target loss value until a target speech recognition model meeting a training stop condition is obtained.

Description

Technical Field

[0001] The embodiments of this specification relate to the field of speech recognition technology, and in particular to a speech recognition model training method and device, and a speech recognition method and device. Background Art

[0002] With the development of computer and Internet technology, automatic speech recognition technology has been applied in more and more scenarios. Automatic speech recognition (ASR) is the core of correctly understanding others and having natural conversations. The speech recognition technology in the prior art is limited by the quality of the speech itself. Background noise and accents of various dialects make it difficult to understand the speaker's words. To address this problem, the prior art mainly realizes speech recognition through a universal large language model, that is, through training on a large-scale public data set, it has relatively rigorous logical reasoning ability and language expression ability. Although the large language model performs well in logical reasoning and language generation, it lacks professional knowledge in vertical fields. In order to make the large language model truly applicable to vertical fields, it is usually necessary to perform field fine-tuning. However, the existing fine-tuning method not only consumes a lot of computing resources, but also takes up more model training time; therefore, an effective solution is urgently needed to solve the above problems. Summary of the invention

[0003] In view of this, an embodiment of this specification provides a speech recognition model training method. One or more embodiments of this specification also relate to a speech recognition model training device, a speech recognition method, a speech recognition device, a computing device, a computer-readable storage medium and a computer program product to solve the technical defects existing in the prior art.

[0004] According to a first aspect of an embodiment of this specification, a speech recognition model training method is provided, comprising: Obtaining preferred audio data and non-preferred audio data associated with an initial speech recognition model; Selecting a network layer to be fine-tuned in the initial speech recognition model, configuring a low-rank adapter for a weight matrix of the network layer to be fine-tuned, and obtaining an intermediate speech recognition model according to the configuration result; Using the intermediate speech recognition model to predict the preferred audio data and the non-preferred audio data for speech recognition, and calculating a target loss value according to the speech recognition result; The parameters of the low-rank adapter in the network layer to be fine-tuned are updated based on the target loss value until a target speech recognition model that meets the training stop condition is obtained.

[0005] Optionally, the determination of the non-preferred audio data includes: Acquire sample voice data, and determine sample text data corresponding to the sample voice data; Inputting the sample speech data into the initial speech recognition model for speech recognition to obtain predicted text data; Candidate audio data is constructed based on the predicted text data, and non-preferred audio data is generated according to the candidate audio data and the sample text data.

[0006] Optionally, the using the intermediate speech recognition model to predict the preferred audio data and the non-preferred audio data for speech recognition, and calculating a target loss value according to the speech recognition result, includes: Using the intermediate speech recognition model, speech recognition is performed on the preferred audio data and the non-preferred audio data respectively to obtain preferred text data corresponding to the preferred audio data and non-preferred text data corresponding to the non-preferred audio data; A target loss value is calculated based on the preference audio data and the preference text data and the non-preference audio data and the non-preference text data.

[0007] Optionally, the calculating the target loss value based on the preferred audio data and the preferred text data and the non-preferred audio data and the non-preferred text data includes: Calculating a negative log-likelihood loss value based on the preferred audio data and the preferred text data and the non-preferred audio data and the non-preferred text data; Calculating a relative proportional loss value according to the preferred audio data and the preferred text data and the non-preferred audio data and the non-preferred text data; A weighted calculation is performed on the negative log-likelihood loss value and the relative proportional loss value to obtain a target loss value.

[0008] Optionally, the calculation formula of the target loss value includes: ; in, represents the target loss value, represents the negative log-likelihood loss value, Represents the relative proportional loss value, and λ represents the weight.

[0009] Optionally, updating the parameters of the low-rank adapter in the network layer to be fine-tuned based on the target loss value until a target speech recognition model that satisfies a training stop condition is obtained includes: Determining parameters associated with the low-rank adapter in the network layer to be fine-tuned; The parameters of the associated low-rank adapter are updated through a back-propagation strategy and a gradient descent strategy until a target speech recognition model that meets the training stop condition is obtained.

[0010] Optionally, after the step of updating the parameters associated with the low-rank adapter in the network layer to be fine-tuned based on the target loss value is executed, the method further includes: Check whether the updated intermediate speech recognition model meets the training stop condition; If not, use the updated intermediate speech recognition model as the intermediate speech recognition model, obtain supplementary preference audio data and supplementary non-preference audio data, use the supplementary preference audio data as the preference audio data, and the supplementary non-preference audio data as the non-preference audio data, and perform the steps of using the intermediate speech recognition model to predict the preference audio data and the non-preference audio data for speech recognition, and calculate the target loss value according to the speech recognition result; If so, the updated intermediate speech recognition model is used as the target speech recognition model.

[0011] According to a second aspect of the embodiments of this specification, there is provided a speech recognition method, comprising: Collecting user audio data of users; Inputting the user audio data into a target speech recognition model for processing to obtain user text data, wherein the target speech recognition model is obtained by the speech recognition model training method; Reply audio data is determined based on the user text data, and the reply audio data is played to the user.

[0012] According to a third aspect of the embodiments of this specification, a speech recognition model training device is provided, comprising: An acquisition module is configured to acquire the preferred audio data and the non-preferred audio data associated with the initial speech recognition model; A selection module is configured to select a network layer to be fine-tuned in the initial speech recognition model, configure a low-rank adapter for a weight matrix of the network layer to be fine-tuned, and obtain an intermediate speech recognition model according to the configuration result; a recognition module configured to use the intermediate speech recognition model to predict the preferred audio data and the non-preferred audio data for speech recognition, and calculate a target loss value according to the speech recognition result; The training module is configured to update the parameters of the low-rank adapter in the network layer to be fine-tuned based on the target loss value until a target speech recognition model that meets the training stop condition is obtained.

[0013] According to a fourth aspect of the embodiments of this specification, there is provided a speech recognition device, including: A collection module, configured to collect user audio data of a user; A processing module, configured to input the user audio data into a target speech recognition model for processing to obtain user text data, wherein the target speech recognition model is obtained by the speech recognition model training method; The playing module is configured to determine the reply audio data according to the user text data, and play the reply audio data to the user.

[0014] According to a fifth aspect of an embodiment of this specification, a computing device is provided, including: Memory and processor; The memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions. When the computer-executable instructions are executed by the processor, the steps of the above-mentioned speech recognition model training method or speech recognition method are implemented.

[0015] According to the sixth aspect of the embodiments of this specification, a computer-readable storage medium is provided, which stores computer-executable instructions, which, when executed by a processor, implement the steps of the above-mentioned speech recognition model training method or speech recognition method.

[0016] According to the seventh aspect of the embodiments of this specification, a computer program product is provided, including a computer program or instructions, which, when executed by a processor, implement the steps of the above-mentioned speech recognition model training method or speech recognition method.

[0017] The speech recognition model training method provided in this embodiment can save more computing resources and improve the model training efficiency in the fine-tuning stage of the speech recognition model. After obtaining the preferred audio data and non-preferred audio data associated with the initial speech recognition model, the network layer to be fine-tuned can be selected in the initial speech recognition model, and a low-rank adapter can be configured for the weight matrix of the network layer to be fine-tuned, and an intermediate speech recognition model can be obtained according to the configuration result; the low-rank decomposition of the weight matrix of the model is realized through the low-rank adapter, so as to realize the approximate representation of the parameters that need to be updated in the fine-tuning process of the model through the low-rank matrix, thereby reducing the number of parameters in the fine-tuning stage. On this basis, the intermediate speech recognition model can be used to predict the preferred audio data and non-preferred audio data for speech recognition, and the target loss value can be calculated according to the speech recognition result; based on this, the parameters of the associated low-rank adapter in the fine-tuning network layer can be updated based on the target loss value until the target speech recognition model that meets the training stop condition is obtained. In the fine-tuning stage of the speech recognition model, it is not necessary to calculate the implicit reward in conjunction with the reference model, which effectively simplifies the training process and saves more computing resources. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 is a flow chart of a speech recognition model training method provided by an embodiment of this specification; Figure 2 is a flow chart of a speech recognition method provided by an embodiment of this specification; Figure 3 It is a structural schematic diagram of a speech recognition model training device provided by an embodiment of this specification; Figure 4 is a structural diagram of a speech recognition device provided by an embodiment of this specification; Figure 5 It is a structural block diagram of a computing device provided by an embodiment of this specification. DETAILED DESCRIPTION

[0019] Many specific details are described in the following description to facilitate a full understanding of this specification. However, this specification can be implemented in many other ways than those described herein, and those skilled in the art can make similar generalizations without violating the connotation of this specification, so this specification is not limited to the specific implementation disclosed below.

[0020] The terms used in one or more embodiments of this specification are only for the purpose of describing specific embodiments, and are not intended to limit one or more embodiments of this specification. The singular forms of "a", "said" and "the" used in one or more embodiments of this specification and the appended claims are also intended to include plural forms, unless the context clearly indicates other meanings. It should also be understood that the term "and / or" used in one or more embodiments of this specification refers to and includes any or all possible combinations of one or more associated listed items.

[0021] It should be understood that although the terms first, second, etc. may be used to describe various information in one or more embodiments of this specification, this information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of one or more embodiments of this specification, the first may also be referred to as the second, and similarly, the second may also be referred to as the first. Depending on the context, the word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining".

[0022] In addition, it should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in one or more embodiments of this specification are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0023] First, the terms involved in one or more embodiments of this specification are explained.

[0024] LoRA (Low-Rank Adaptation) is a fine-tuning technique for large pre-trained models (such as large language models LLMs), which adds a small number of trainable parameters to the model while keeping the original model parameters unchanged. The core idea is to adjust the behavior of the pre-trained model by introducing a small number of trainable parameters without retraining the entire model, thereby significantly reducing the computing resources and time required for training.

[0025] ORPO (Odds Ratio Preference Optimization) is a new large language model (LLM) alignment method. By directly optimizing the output preference of the model instead of relying on the traditional reference model, the training process is simplified and the adaptability and robustness of the model are improved. In the ORPO method, OR stands for odds ratio, and OR loss is part of the ORPO training loss function. Rejected answers are weakly penalized, while selected answers are strongly rewarded. A hyperparameter lambda is introduced to weight the OR loss, so that the model can learn human preferences.

[0026] Large Language Model: A deep learning-based neural network model that can process and generate human language and perform various natural language processing (NLP) tasks such as text generation and machine translation by training massive text data.

[0027] Negative Log-Likelihood Loss, negative log-likelihood loss (NLL), is a loss function commonly used in machine learning, especially classification problems. It is used to measure the difference between the probability distribution predicted by the model and the true label.

[0028] SFT stands for Supervised Fine-Tuning. It is a technique that takes a pre-trained network model and retrains it on a small amount of supervised data for a specific task. It is an important training strategy in the field of deep learning and natural language processing. It uses labeled data to further train the pre-trained large language model, aiming to optimize the performance of the model on a specific task.

[0029] In this specification, a speech recognition model training method is provided. One or more embodiments of this specification also involve a speech recognition model training device, a speech recognition method, a speech recognition device, a computing device, a computer-readable storage medium, and a computer program product, which are described in detail one by one in the following embodiments.

[0030] See also Figure 1 , Figure 1 A flowchart of a speech recognition model training method provided according to an embodiment of the present specification is shown, which specifically includes the following steps.

[0031] Step S102, obtaining the preferred audio data and non-preferred audio data associated with the initial speech recognition model.

[0032] The speech recognition model training method provided in this embodiment can be applied to speech recognition models deployed in any field, such as voice assistants in telephone communication scenarios, voice assistants in smart home appliance control scenarios, voice assistants in shopping mall navigation scenarios, voice assistants in hospital guidance scenarios, etc. The trained model is deployed in the corresponding scenario, and can be recognized according to the user's speech content, and then feedback the audio of the user's speech content to the user, thereby helping the user understand the business, control home appliances, navigate, guide, etc.

[0033] This embodiment describes the speech recognition model training method by taking the speech recognition model to be trained in the speech recognition model training method as a model required for a telephone communication scenario as an example. The descriptions of the speech recognition model training methods in other scenarios can refer to the same or corresponding descriptions in this embodiment, and this embodiment will not go into details here.

[0034] Specifically, the initial speech recognition model refers to a speech recognition model that has general speech recognition capabilities in the current field but needs to be fine-tuned so that it can be applied to the current field after fine-tuning and produce speech recognition results that are more in line with the field. For example, CAD in the engineering field means computer-aided design, while in the medical field, it means coronary artery disease; after fine-tuning the speech recognition model in the medical scenario, if the audio of the user's speech contains CAD characters, the model can recognize it as coronary artery disease instead of computer-aided design, making it more in line with the current deployment field.

[0035] Accordingly, the preferred audio data specifically refers to the sample pairs constructed by the audio samples and labels that conform to the user's expression logic, where the labels refer to the text corresponding to the audio. Correspondingly, the non-preferred audio data specifically refers to the sample pairs constructed by the audio samples and labels that do not conform to the user's expression logic. By combining the preferred audio data and the non-preferred audio data to fine-tune the initial speech recognition model, the model can be closer to the real results and away from the false results, thereby improving the prediction accuracy of the model.

[0036] Based on this, in order to save more computing resources and improve the model training efficiency in the fine-tuning stage of the speech recognition model, after obtaining the preferred audio data and non-preferred audio data associated with the initial speech recognition model, the network layer to be fine-tuned can be selected in the initial speech recognition model, and a low-rank adapter can be configured for the weight matrix of the network layer to be fine-tuned, and the intermediate speech recognition model can be obtained according to the configuration result; the low-rank decomposition of the weight matrix of the model can be achieved through the low-rank adapter, so as to achieve the low-rank matrix to approximate the parameters that need to be updated in the fine-tuning process of the model, thereby reducing the number of parameters in the fine-tuning stage. On this basis, the intermediate speech recognition model can be used to predict the preferred audio data and non-preferred audio data for speech recognition, and the target loss value can be calculated according to the speech recognition results; based on this, the parameters of the associated low-rank adapter in the fine-tuning network layer can be updated based on the target loss value until the target speech recognition model that meets the training stop condition is obtained. In the fine-tuning stage of the speech recognition model, there is no need to calculate the implicit reward in conjunction with the reference model, which effectively simplifies the training process and saves more computing resources.

[0037] In one or more optional implementation manners of this embodiment, the determination of the non-preferred audio data includes: Acquire sample speech data and determine sample text data corresponding to the sample speech data; input the sample speech data into the initial speech recognition model for speech recognition to obtain predicted text data; construct candidate audio data based on the predicted text data, and generate non-preferred audio data based on the candidate audio data and the sample text data.

[0038] Specifically, the sample voice data refers to any audio data that needs to be processed, and correspondingly, the sample text data refers to the text description corresponding to the sample audio data. Correspondingly, the predicted text data refers to the predicted text description output after the sample audio data is input into the initial speech recognition model. At this time, the predicted text data obtained does not match the scenario where the model needs to be deployed. Therefore, the non-preferred audio data can be constructed by combining the sample text data and the candidate audio data, which is convenient for the subsequent fine-tuning of the speech recognition model in combination with the preferred audio data.

[0039] Based on this, in order to obtain sufficient non-preferred audio data for subsequent model training, sample voice data can be obtained, and sample text data corresponding to the sample voice data can be determined; at this time, the sample voice data can be input into the initial speech recognition model for speech recognition, and the predicted text data can be obtained using the model output before fine-tuning; thereafter, candidate audio data can be constructed based on the predicted text data, and non-preferred audio data can be generated based on the candidate audio data and the sample text data for use in subsequent model training.

[0040] In actual applications, before training the model, the obtained preferred audio data can be manually labeled to complete the construction of Yangben pairs. For example, if the audio content is welcome, the corresponding labeling result is "Welcome"; and the non-preferred audio data can be output by the speech recognition model before fine-tuning. For example, for the same audio content, the model outputs "Welcome hmm hmm". After that, the data can be provided to the model in pairs for training. At the same time, the model can generate new prediction data during the iteration process. The model can be adjusted according to the dynamic changes in the field, self-correct and gradually optimize the strategy through interaction with the environment, so as to match the scenarios that need to be deployed for downstream business use.

[0041] In summary, by using the speech recognition model before fine-tuning to construct non-preferred audio data, it can ensure that the data is richer and meets the needs of subsequent model fine-tuning, thereby improving the prediction accuracy of the trained model.

[0042] Step S104, selecting a network layer to be fine-tuned in the initial speech recognition model, and configuring a low-rank adapter for the weight matrix of the network layer to be fine-tuned, and obtaining an intermediate speech recognition model according to the configuration result.

[0043] Specifically, after obtaining the preferred audio data and non-preferred audio data of the training model, in order to reduce the amount of fine-tuning parameters in the model fine-tuning stage and improve the model training efficiency, the amount of fine-tuning parameters can be reduced by introducing a low-rank adapter. In other words, the network layer to be fine-tuned can be selected in the initial speech recognition model first, and a low-rank adapter can be configured for the weight matrix of the network layer to be fine-tuned, so as to obtain an intermediate speech recognition model according to the configuration result. Subsequently, by fine-tuning the intermediate speech recognition model, the model parameters updated in the fine-tuning stage can be reduced due to the introduction of the low-rank adapter, thereby achieving the purpose of saving computing resources.

[0044] The network layer to be fine-tuned specifically refers to the network layer in the initial speech recognition model that needs to update parameters, and the self-attention layer and feedforward network layer of the initial speech model can be selected. Correspondingly, the low-rank adapter specifically refers to the low-rank matrix configured for the weight matrix of the network layer to be fine-tuned. Correspondingly, the intermediate speech recognition model specifically refers to the speech recognition model obtained after the low-rank adapter is introduced.

[0045] In practical applications, LoRA technology can be used to fine-tune the model. LoRA is an efficient fine-tuning method. It will select the most critical layer for performance for fine-tuning, avoid fine-tuning every layer, and reduce computational overhead. Therefore, in the speech recognition model training scenario, the self-attention layer and feedforward network layer in the speech recognition model can be selected for subsequent fine-tuning.

[0046] In specific implementation, a low-rank adapter can be introduced for the weight matrix W of the network layer to be fine-tuned, and added to the original weight matrix, so that the fine-tuning of the model only needs to learn the low-rank matrix without changing the original large weight matrix. It can be understood that this process is to perform low-rank decomposition on the weight matrix of the network layer to be fine-tuned of the speech recognition model, and then introduce a low-rank matrix to approximate the parameter part that needs to be updated in the fine-tuning process of the model, rather than directly selecting a part of the original parameters for replacement or modification.

[0047] Specifically, the low-rank adapter usually involves two low-rank matrices A and B. The product of these two matrices (i.e., A⋅B) is used to approximate the incremental update part (i.e., ΔW) of the original weight matrix W. During the training process, only the two low-rank matrices A and B are optimized, while the original model weight W remains unchanged. After the training is completed, the increment ΔW (i.e., A⋅B) of the low-rank matrix is ​​superimposed on the original weight matrix to obtain the new model weight, and the adaptation of the speech recognition model in the corresponding scenario is completed. This process can be expressed by the following formula (1): (1) Where W represents the weight matrix, represents the low-rank update term, A and B are the learned low-rank matrices, △W is the incremental update part of the weight matrix, Represents the weight matrix after parameter update.

[0048] Step S106: Use the intermediate speech recognition model to predict the preferred audio data and the non-preferred audio data for speech recognition, and calculate a target loss value based on the speech recognition result.

[0049] Specifically, after the low-rank adapter is introduced for the speech recognition model, the fine-tuning training of the intermediate speech recognition model can be started. At this time, the intermediate speech recognition model can be used to predict the preferred audio data and non-preferred audio data for speech recognition, and then the target loss value is calculated according to the speech recognition results, so that the model can be adjusted according to the target loss value. The target loss value specifically refers to the loss value calculated after the model prediction combined with the input, output and label of the model.

[0050] In one or more optional implementations of this embodiment, the using the intermediate speech recognition model to predict the preferred audio data and the non-preferred audio data for speech recognition, and calculating the target loss value according to the speech recognition result, includes: The preferred audio data and the non-preferred audio data are respectively subjected to speech recognition using the intermediate speech recognition model to obtain preferred text data corresponding to the preferred audio data and non-preferred text data corresponding to the non-preferred audio data; a target loss value is calculated based on the preferred audio data and the preferred text data as well as the non-preferred audio data and the non-preferred text data.

[0051] Specifically, the preferred text data specifically refers to the text description obtained after the preferred audio data is voice recognized by using the intermediate voice recognition model. The non-preferred text data specifically refers to the text description obtained after the non-preferred audio data is voice recognized by using the intermediate voice recognition model.

[0052] Based on this, when calculating the target loss value, the intermediate speech recognition model can be used to perform speech recognition on the preferred audio data and non-preferred audio data respectively, and then the preferred text data corresponding to the preferred audio data and the non-preferred text data corresponding to the non-preferred audio data can be obtained according to the recognition results; at this time, the target loss value can be calculated based on the preferred audio data and the preferred text data as well as the non-preferred audio data and the non-preferred text data for subsequent model fine-tuning.

[0053] In summary, by using the model to process non-preferred audio data and preferred audio data separately, and calculating the target loss value based on this, the model can be made more in line with domain prediction requirements after fine-tuning, thereby improving business service effects.

[0054] In one or more optional implementations of this embodiment, the calculating the target loss value based on the preferred audio data and the preferred text data and the non-preferred audio data and the non-preferred text data includes: A negative log-likelihood loss value is calculated based on the preference audio data and the preference text data as well as the non-preference audio data and the non-preference text data; a relative proportional loss value is calculated based on the preference audio data and the preference text data as well as the non-preference audio data and the non-preference text data; and a weighted calculation is performed on the negative log-likelihood loss value and the relative proportional loss value to obtain a target loss value.

[0055] Based on this, when calculating the target loss value according to the prediction results of the model, the negative log-likelihood loss value can be first calculated based on the preferred audio data and preferred text data as well as the non-preferred audio data and non-preferred text data; at the same time, the relative proportional loss value can also be calculated based on the preferred audio data and preferred text data as well as the non-preferred audio data and non-preferred text data; on this basis, by performing weighted calculations on the negative log-likelihood loss value and the relative proportional loss value, the loss value calculation accuracy can be achieved, and the target loss value can be obtained for use in model fine-tuning.

[0056] In practical applications, the optimization strategy of ORPO (Odds Ratio Preference Optimization Algorithm) can be used for model training. As a method for LLM alignment, ORPO can combine instruction fine-tuning and preference alignment into a single process, thereby simplifying the training process. Unlike DPO (Direct Preference Optimization), ORPO does not require a reference model to calculate implicit rewards, which effectively saves memory and computing resources.

[0057] ORPO optimizes the model's preferences by combining the traditional negative log-likelihood loss (NLL) with an odds ratio-based loss. Selected and rejected responses may have a lot in common, such as the same domain, the same format, etc., so the probability of generating task-relevant but incorrect answers increases. DPO can reduce the probability of rejected responses while increasing the probability of selected responses. For STF, it is trained on prompts paired with the selected answers. The dataset used for SFT can be the same as that used for preference optimization, but without the "rejected" answers. It can therefore be determined to fine-tune a base LLM so that it learns to punish and prefer certain answers while learning how to answer instructions.

[0058] In specific implementation, ORPO modifies the training loss by adding negative log-likelihood loss and OR loss (OR stands for odds ratio). The calculation formula of Odds Ratio is as follows (2): (2) Among them, x represents input, y represents output, θ represents model parameters, and odds represents the occurrence / odds ratio, the ratio of event occurrence to non-occurrence.

[0059] The calculation formula for OR loss is as follows (3): (3) Among them, OR means odds ratio, The output representing victory, Output indicating failure.

[0060] In one or more optional implementations of this embodiment, the target loss value is calculated by the following formula (4): (4) in, represents the target loss value, represents the negative log-likelihood loss value, Represents the relative proportional loss value, and λ represents the weight.

[0061] It can be understood that OR loss weakly penalizes rejected answers and strongly rewards selected answers. A hyperparameter lambda is included here for weighted OR loss. Through the loss of ORPO, the model not only learns the content during SFT, but also learns the preferences of users corresponding to the scene. In other words, ORPO is more computationally efficient.

[0062] In summary, by combining the negative log-likelihood loss value and the relative proportional loss value to calculate the target loss value, the subsequent model fine-tuning can make the model have stronger predictive ability, which is convenient for downstream business use.

[0063] Step S108, updating the parameters of the low-rank adapter in the network layer to be fine-tuned based on the target loss value until a target speech recognition model that meets the training stop condition is obtained.

[0064] Specifically, after the target loss value is calculated based on the prediction results of the model, the parameters of the associated low-rank adapter in the fine-tuning network layer can be updated based on the target loss value, so that the model can be fine-tuned by updating only some parameters, and iterating in this way until the target speech recognition model that meets the training stop condition is obtained. Among them, the training stop condition specifically refers to the condition for stopping the training of the speech recognition model, including but not limited to the loss value comparison condition, the number of iterations condition or the verification set verification condition.

[0065] In one or more optional implementations of this embodiment, updating the parameters of the low-rank adapter associated with the network layer to be fine-tuned based on the target loss value until a target speech recognition model that meets the training stop condition is obtained includes: Determine the parameters associated with the low-rank adapter in the network layer to be fine-tuned; update the parameters associated with the low-rank adapter through a back-propagation strategy and a gradient descent strategy until a target speech recognition model that meets the training stop condition is obtained.

[0066] Based on this, when updating parameters, since a low-rank adapter is introduced, the parameters of the associated low-rank adapter can be determined in the network layer to be fine-tuned first; thereafter, the backpropagation strategy and gradient descent strategy can be used to update the parameters of the associated low-rank adapter until the target speech recognition model that meets the training stop conditions is obtained.

[0067] In practical applications, by training the low-rank adapter while keeping the weights of the original model unchanged, the amount of parameter updates can be effectively reduced. In LoRA, the parameters of the low-rank adapter are trainable, while the weights of the original large model remain fixed. In other words, the number of parameters of the large model will not increase too much, and only a small number of parameters need to be trained, which greatly saves computing resources and storage requirements. During the training process, the low-rank adapter adapts to new tasks and data through optimization. The training only involves the parameters of the low-rank matrix, not the parameters of the entire model, so the cost of fine-tuning is greatly reduced.

[0068] Therefore, the parameters of the low-rank adapter are updated through back-propagation and gradient descent, and standard gradient updates are performed to optimize only the parameters of the low-rank adapter. Standard optimization methods (such as Adam, SGD, etc.) are used to train the adapter to ensure that it can adjust the appropriate parameters according to the task objectives. During training, since only the parameters of the low-rank adapter are updated, most of the parameters of the large model remain fixed, so the computational and storage overhead is relatively small.

[0069] In summary, by adopting the back-propagation strategy and gradient descent strategy to update the parameters of the low-rank adapter, the model can reduce the amount of parameter updates in the fine-tuning stage without reducing the prediction accuracy of the model, thereby saving more computing resources.

[0070] In one or more optional implementations of this embodiment, after the step of updating the parameters associated with the low-rank adapter in the network layer to be fine-tuned based on the target loss value is executed, the step further includes: Check whether the updated intermediate speech recognition model meets the training stop condition; if not, use the updated intermediate speech recognition model as the intermediate speech recognition model, and obtain supplementary preference audio data and supplementary non-preferential audio data, use the supplementary preference audio data as the preference audio data, and the supplementary non-preferential audio data as the non-preferential audio data, and execute the steps of using the intermediate speech recognition model to predict the preference audio data and the non-preferential audio data for speech recognition, and calculate the target loss value according to the speech recognition result; if yes, use the updated intermediate speech recognition model as the target speech recognition model.

[0071] Based on this, when conducting model training, in order to train a speech recognition model that meets the expectations, it is possible to detect whether the updated intermediate speech recognition model meets the training stop conditions; if not, it means that the model still needs to continue training, so the updated intermediate speech recognition model can be used as the intermediate speech recognition model, and supplementary preference audio data and supplementary non-preferential audio data can be obtained, and the supplementary preference audio data can be used as preference audio data, and the supplementary non-preferential audio data can be used as non-preferential audio data, and the steps of using the intermediate speech recognition model to predict the preference audio data and the non-preferential audio data for speech recognition, and calculating the target loss value based on the speech recognition results can be performed; if so, the updated intermediate speech recognition model can be used as the target speech recognition model for subsequent business use.

[0072] In actual applications, after obtaining the target speech recognition model through the above training process, the target speech recognition model can be deployed to the corresponding scenario, such as a telephone communication scenario. For the speech content of the user's telephone communication, the speech recognition model can be used to determine its corresponding text description, and then select the audio that can answer the user's question for reply, thereby effectively improving the user's telephone communication experience. The application of the speech recognition model in the telephone communication scenario can be business consulting and other applications, so that the user can talk to the voice assistant to meet the user's consultation needs.

[0073] The speech recognition model training method provided in this embodiment can save more computing resources and improve the model training efficiency in the fine-tuning stage of the speech recognition model. After obtaining the preferred audio data and non-preferred audio data associated with the initial speech recognition model, the network layer to be fine-tuned can be selected in the initial speech recognition model, and a low-rank adapter can be configured for the weight matrix of the network layer to be fine-tuned, and an intermediate speech recognition model can be obtained according to the configuration result; the low-rank decomposition of the weight matrix of the model is realized through the low-rank adapter, so as to realize the approximate representation of the parameters that need to be updated in the fine-tuning process of the model through the low-rank matrix, thereby reducing the number of parameters in the fine-tuning stage. On this basis, the intermediate speech recognition model can be used to predict the preferred audio data and non-preferred audio data for speech recognition, and the target loss value can be calculated according to the speech recognition result; based on this, the parameters of the associated low-rank adapter in the fine-tuning network layer can be updated based on the target loss value until the target speech recognition model that meets the training stop condition is obtained. In the fine-tuning stage of the speech recognition model, it is not necessary to calculate the implicit reward in conjunction with the reference model, which effectively simplifies the training process and saves more computing resources.

[0074] See also Figure 2 , Figure 2 A flow chart of a speech recognition method provided according to an embodiment of the present specification is shown, which specifically includes the following steps.

[0075] Step S202: collecting user audio data of the user.

[0076] Step S204: input the user audio data into a target speech recognition model for processing to obtain user text data, wherein the target speech recognition model is obtained by the speech recognition model training method.

[0077] Step S206, determining reply audio data according to the user text data, and playing the reply audio data to the user.

[0078] For the contents not described in detail in the speech recognition method provided in this embodiment, please refer to the same or corresponding description in the above embodiments, and this embodiment will not be described in detail here.

[0079] Specifically, the user audio data specifically refers to the audio clip of the user's speech, and correspondingly, the user text data specifically refers to the text description obtained after converting the user audio data. Correspondingly, the reply audio data specifically refers to the audio clip of the reply to the user's question or consultation content.

[0080] Based on this, after the target speech recognition model is deployed to the corresponding scenario, after the user audio data is collected from the user, the user audio data can be input into the target speech recognition model for processing, and then the user text data can be output through the processing results of the model; on this basis, the reply audio data can be determined according to the user text data, and the reply audio data can be played to the user, thereby completing the automatic conversation with the user.

[0081] In actual applications, the reply audio data can be determined by matching in a pre-established voice library, or generated by a large language model. During specific implementation, it can be set according to actual needs, and this embodiment does not make any limitations here.

[0082] For example, the hospital registration machine can support user voice registration by deploying a speech recognition model, making it more convenient for users to register. In this process, when the user says "I want to go to the orthopedics department" in front of the registration machine, the audio clip of the user's speech can be collected and then input into the speech recognition model to obtain the text description "I want to go to the orthopedics department" corresponding to the audio clip. After that, the voice content corresponding to "I want to go to the orthopedics department" can be found in the preset voice library, such as "There are N experts in the orthopedics clinic, namely...", and the corresponding audio is played to the user. Through continuous dialogue and interaction with the user, the user can complete the hospital registration and treatment without any operation.

[0083] Corresponding to the above method embodiment, this specification also provides a speech recognition model training device embodiment, Figure 3 FIG. 1 is a schematic diagram showing the structure of a speech recognition model training device provided by an embodiment of the present specification. Figure 3 As shown, the device comprises: An acquisition module 302 is configured to acquire preferred audio data and non-preferred audio data associated with an initial speech recognition model; A selection module 304 is configured to select a network layer to be fine-tuned in the initial speech recognition model, configure a low-rank adapter for the weight matrix of the network layer to be fine-tuned, and obtain an intermediate speech recognition model according to the configuration result; A recognition module 306 is configured to use the intermediate speech recognition model to predict the preferred audio data and the non-preferred audio data for speech recognition, and calculate a target loss value according to the speech recognition result; The training module 308 is configured to update the parameters of the low-rank adapter associated with the network layer to be fine-tuned based on the target loss value until a target speech recognition model that meets the training stop condition is obtained.

[0084] In an optional embodiment, the determination of the non-preferred audio data includes: Acquire sample speech data and determine sample text data corresponding to the sample speech data; input the sample speech data into the initial speech recognition model for speech recognition to obtain predicted text data; construct candidate audio data based on the predicted text data, and generate non-preferred audio data based on the candidate audio data and the sample text data.

[0085] In an optional embodiment, the using the intermediate speech recognition model to predict the preferred audio data and the non-preferred audio data for speech recognition, and calculating the target loss value according to the speech recognition result, includes: The preferred audio data and the non-preferred audio data are respectively subjected to speech recognition using the intermediate speech recognition model to obtain preferred text data corresponding to the preferred audio data and non-preferred text data corresponding to the non-preferred audio data; a target loss value is calculated based on the preferred audio data and the preferred text data as well as the non-preferred audio data and the non-preferred text data.

[0086] In an optional embodiment, the calculating the target loss value based on the preferred audio data and the preferred text data and the non-preferred audio data and the non-preferred text data includes: A negative log-likelihood loss value is calculated based on the preference audio data and the preference text data as well as the non-preference audio data and the non-preference text data; a relative proportional loss value is calculated based on the preference audio data and the preference text data as well as the non-preference audio data and the non-preference text data; and a weighted calculation is performed on the negative log-likelihood loss value and the relative proportional loss value to obtain a target loss value.

[0087] In an optional embodiment, the calculation formula of the target loss value includes: ; in, represents the target loss value, represents the negative log-likelihood loss value, Represents the relative proportional loss value, and λ represents the weight.

[0088] In an optional embodiment, the updating of the parameters of the low-rank adapter in the network layer to be fine-tuned based on the target loss value until a target speech recognition model that satisfies the training stop condition is obtained includes: Determine the parameters associated with the low-rank adapter in the network layer to be fine-tuned; update the parameters associated with the low-rank adapter through a back-propagation strategy and a gradient descent strategy until a target speech recognition model that meets the training stop condition is obtained.

[0089] In an optional embodiment, after the step of updating the parameters associated with the low-rank adapter in the network layer to be fine-tuned based on the target loss value is executed, the method further includes: Check whether the updated intermediate speech recognition model meets the training stop condition; if not, use the updated intermediate speech recognition model as the intermediate speech recognition model, and obtain supplementary preference audio data and supplementary non-preferential audio data, use the supplementary preference audio data as the preference audio data, and the supplementary non-preferential audio data as the non-preferential audio data, and execute the steps of using the intermediate speech recognition model to predict the preference audio data and the non-preferential audio data for speech recognition, and calculate the target loss value according to the speech recognition result; if yes, use the updated intermediate speech recognition model as the target speech recognition model.

[0090] The speech recognition model training device provided in this embodiment can save more computing resources and improve the model training efficiency in the fine-tuning stage of the speech recognition model. After obtaining the preferred audio data and non-preferred audio data associated with the initial speech recognition model, the network layer to be fine-tuned can be selected in the initial speech recognition model, and a low-rank adapter can be configured for the weight matrix of the network layer to be fine-tuned, and an intermediate speech recognition model can be obtained according to the configuration result; the low-rank decomposition of the weight matrix of the model is realized through the low-rank adapter, so as to realize the approximate representation of the parameters that need to be updated in the fine-tuning process of the model through the low-rank matrix, thereby reducing the number of parameters in the fine-tuning stage. On this basis, the intermediate speech recognition model can be used to predict the preferred audio data and non-preferred audio data for speech recognition, and the target loss value can be calculated according to the speech recognition result; on this basis, the parameters of the associated low-rank adapter in the fine-tuning network layer can be updated based on the target loss value until the target speech recognition model that meets the training stop condition is obtained. In the fine-tuning stage of the speech recognition model, it is not necessary to calculate the implicit reward in conjunction with the reference model, which effectively simplifies the training process and saves more computing resources.

[0091] The above is a schematic scheme of a speech recognition model training device of this embodiment. It should be noted that the technical scheme of the speech recognition model training device and the technical scheme of the speech recognition model training method mentioned above belong to the same concept, and the details not described in detail in the technical scheme of the speech recognition model training device can be found in the description of the technical scheme of the speech recognition model training method mentioned above.

[0092] Corresponding to the above method embodiment, this specification also provides a speech recognition device embodiment, Figure 4 FIG. 1 shows a schematic diagram of the structure of a speech recognition device provided by an embodiment of the present specification. Figure 4 As shown, the device comprises: A collection module 402 is configured to collect user audio data of a user; The processing module 404 is configured to input the user audio data into a target speech recognition model for processing to obtain user text data, wherein the target speech recognition model is obtained by the speech recognition model training method; The playing module 406 is configured to determine the reply audio data according to the user text data, and play the reply audio data to the user.

[0093] The above is a schematic scheme of a speech recognition device of this embodiment. It should be noted that the technical scheme of the speech recognition device and the technical scheme of the above speech recognition method belong to the same concept, and the details not described in detail in the technical scheme of the speech recognition device can be referred to the description of the technical scheme of the above speech recognition method.

[0094] Figure 5 The structure block diagram of a computing device 500 provided according to an embodiment of the present specification is shown. The components of the computing device 500 include but are not limited to a memory 510 and a processor 520. The processor 520 is connected to the memory 510 via a bus 530, and the database 550 is used to store data.

[0095] The computing device 500 also includes an access device 540 that enables the computing device 500 to communicate via one or more networks 560. Examples of these networks include a public switched telephone network (PSTN), a local area network (LAN), a wide area network (WAN), a personal area network (PAN), or a combination of communication networks such as the Internet. The access device 540 may include one or more of any type of network interface (e.g., a network interface card (NIC)) that is wired or wireless, such as an IEEE 802.11 wireless local area network (WLAN) wireless interface, a world-wide interoperability for microwave access (Wi-MAX) interface, an Ethernet interface, a universal serial bus (USB) interface, a cellular network interface, a Bluetooth interface, and a near field communication (NFC).

[0096] In one embodiment of the present specification, the above components of the computing device 500 and Figure 5 Other components not shown in the figure may also be connected to each other, for example, via a bus. It should be understood that Figure 5 The computing device structure block diagram shown is only for the purpose of illustration, and is not intended to limit the scope of this specification. Those skilled in the art can add or replace other components as needed.

[0097] The computing device 500 may be any type of stationary or mobile computing device, including a mobile computer or mobile computing device (e.g., a tablet computer, a personal digital assistant, a laptop computer, a notebook computer, a netbook, etc.), a mobile phone (e.g., a smart phone), a wearable computing device (e.g., a smart watch, smart glasses, etc.), or other types of mobile devices, or a stationary computing device such as a desktop computer or a personal computer (PC). The computing device 500 may also be a mobile or stationary server.

[0098] Among them, the processor 520 is used to execute the following computer-executable instructions, which, when executed by the processor, implement the steps of the above-mentioned speech recognition model training method or speech recognition method.

[0099] The above is a schematic scheme of a computing device of this embodiment. It should be noted that the technical scheme of the computing device and the technical scheme of the above-mentioned speech recognition model training method or speech recognition method belong to the same concept, and the details not described in detail in the technical scheme of the computing device can be referred to the description of the technical scheme of the above-mentioned speech recognition model training method or speech recognition method.

[0100] An embodiment of the present specification also provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, can implement the steps of the above-mentioned speech recognition model training method or speech recognition method.

[0101] The above is a schematic scheme of a computer-readable storage medium of this embodiment. It should be noted that the technical scheme of the storage medium and the technical scheme of the above-mentioned speech recognition model training method or speech recognition method belong to the same concept, and the details not described in detail in the technical scheme of the storage medium can be referred to the description of the technical scheme of the above-mentioned speech recognition model training method or speech recognition method.

[0102] An embodiment of the present specification also provides a computer program product, including a computer program or instructions, which, when executed by a processor, implements the steps of the above-mentioned speech recognition model training method or speech recognition method.

[0103] The above is a schematic scheme of a computer program product of this embodiment. It should be noted that the technical scheme of the computer program product and the technical scheme of the above-mentioned speech recognition model training method or speech recognition method belong to the same concept, and the details not described in detail in the technical scheme of the computer program product can be referred to the description of the technical scheme of the above-mentioned speech recognition model training method or speech recognition method.

[0104] The above is a description of a specific embodiment of the specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0105] The computer instructions include computer program codes, which may be in source code form, object code form, executable files or some intermediate forms, etc. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal and software distribution medium, etc. It should be noted that the content contained in the computer-readable medium may be appropriately increased or decreased according to the requirements of patent practice. For example, in some regions, according to patent practice, computer-readable media do not include electric carrier signals and telecommunication signals.

[0106] It should be noted that, for the above-mentioned method embodiments, for the sake of simplicity of description, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the embodiments of this specification are not limited by the order of the actions described, because according to the embodiments of this specification, some steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the embodiments of this specification.

[0107] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0108] The preferred embodiments of this specification disclosed above are only used to help explain this specification. The optional embodiments do not describe all the details in detail, nor do they limit the invention to the specific implementation methods described. Obviously, many modifications and changes can be made according to the content of the embodiments of this specification. This specification selects and specifically describes these embodiments in order to better explain the principles and practical applications of the embodiments of this specification, so that technicians in the relevant technical field can understand and use this specification well.

Claims

1. A speech recognition model training method, characterized in that: include: Obtaining preferred audio data and non-preferred audio data associated with an initial speech recognition model; Selecting a network layer to be fine-tuned in the initial speech recognition model, configuring a low-rank adapter for a weight matrix of the network layer to be fine-tuned, and obtaining an intermediate speech recognition model according to the configuration result; Using the intermediate speech recognition model to predict the preferred audio data and the non-preferred audio data for speech recognition, and calculating a target loss value according to the speech recognition result; The parameters of the low-rank adapter in the network layer to be fine-tuned are updated based on the target loss value until a target speech recognition model that meets the training stop condition is obtained.

2. The speech recognition model training method according to claim 1, characterized in that: The determination of the non-preferred audio data includes: Acquire sample voice data, and determine sample text data corresponding to the sample voice data; Inputting the sample speech data into the initial speech recognition model for speech recognition to obtain predicted text data; Candidate audio data is constructed based on the predicted text data, and non-preferred audio data is generated according to the candidate audio data and the sample text data.

3. The speech recognition model training method according to claim 1, characterized in that: The using the intermediate speech recognition model to predict the preferred audio data and the non-preferred audio data for speech recognition, and calculating the target loss value according to the speech recognition result, includes: Using the intermediate speech recognition model, speech recognition is performed on the preferred audio data and the non-preferred audio data respectively to obtain preferred text data corresponding to the preferred audio data and non-preferred text data corresponding to the non-preferred audio data; A target loss value is calculated based on the preference audio data and the preference text data and the non-preference audio data and the non-preference text data.

4. The speech recognition model training method according to claim 3, characterized in that: The calculating the target loss value based on the preferred audio data and the preferred text data and the non-preferred audio data and the non-preferred text data comprises: Calculating a negative log-likelihood loss value based on the preferred audio data and the preferred text data and the non-preferred audio data and the non-preferred text data; Calculating a relative proportional loss value according to the preferred audio data and the preferred text data and the non-preferred audio data and the non-preferred text data; A weighted calculation is performed on the negative log-likelihood loss value and the relative proportional loss value to obtain a target loss value.

5. The speech recognition model training method according to claim 4, characterized in that: The calculation formula of the target loss value includes: ； in, represents the target loss value, represents the negative log-likelihood loss value, Represents the relative proportional loss value, and λ represents the weight.

6. The speech recognition model training method according to claim 1, characterized in that: The updating of the parameters associated with the low-rank adapter in the network layer to be fine-tuned based on the target loss value until a target speech recognition model that satisfies the training stop condition is obtained includes: Determining parameters associated with the low-rank adapter in the network layer to be fine-tuned; The parameters of the associated low-rank adapter are updated through a back-propagation strategy and a gradient descent strategy until a target speech recognition model that meets the training stop condition is obtained.

7. The speech recognition model training method according to any one of claims 1 to 6, characterized in that: After the step of updating the parameters associated with the low-rank adapter in the network layer to be fine-tuned based on the target loss value is executed, the method further includes: Check whether the updated intermediate speech recognition model meets the training stop condition; If not, use the updated intermediate speech recognition model as the intermediate speech recognition model, obtain supplementary preference audio data and supplementary non-preference audio data, use the supplementary preference audio data as the preference audio data, and the supplementary non-preference audio data as the non-preference audio data, and perform the steps of using the intermediate speech recognition model to predict the preference audio data and the non-preference audio data for speech recognition, and calculate the target loss value according to the speech recognition result; If so, the updated intermediate speech recognition model is used as the target speech recognition model.

8. A speech recognition method, characterized in that: include: Collecting user audio data of users; Inputting the user audio data into a target speech recognition model for processing to obtain user text data, wherein the target speech recognition model is obtained by the speech recognition model training method according to any one of claims 1 to 7; Reply audio data is determined based on the user text data, and the reply audio data is played to the user.

9. A speech recognition model training device, characterized in that: include: An acquisition module is configured to acquire the preferred audio data and the non-preferred audio data associated with the initial speech recognition model; A selection module is configured to select a network layer to be fine-tuned in the initial speech recognition model, configure a low-rank adapter for a weight matrix of the network layer to be fine-tuned, and obtain an intermediate speech recognition model according to the configuration result; a recognition module configured to use the intermediate speech recognition model to predict the preferred audio data and the non-preferred audio data for speech recognition, and calculate a target loss value according to the speech recognition result; The training module is configured to update the parameters of the low-rank adapter in the network layer to be fine-tuned based on the target loss value until a target speech recognition model that meets the training stop condition is obtained.

10. A speech recognition device, characterized in that: include: A collection module, configured to collect user audio data of a user; A processing module, configured to input the user audio data into a target speech recognition model for processing to obtain user text data, wherein the target speech recognition model is obtained by the speech recognition model training method according to any one of claims 1 to 7; The playing module is configured to determine the reply audio data according to the user text data, and play the reply audio data to the user.

11. A computing device, characterized in that: include: Memory and processor; The memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions. When the computer-executable instructions are executed by the processor, the steps of the method described in any one of claims 1 to 8 are implemented.

12. A computer-readable storage medium, characterized in that: It stores computer executable instructions, which, when executed by a processor, implement the steps of the method described in any one of claims 1 to 8.

13. A computer program product, characterized in that The method comprises a computer program or an instruction, which, when executed by a processor, implements the steps of the method according to any one of claims 1 to 8.

Images