Training methods and devices for speech recognition models; speech recognition methods and devices

Abstract

This disclosure provides a training method and apparatus for a speech recognition model, a speech recognition method and apparatus, a computer-readable storage medium, and an electronic device. The method includes: extracting initial speech feature data from sample speech signals; determining sample speech feature data based on the length of the initial speech feature data; recognizing the sample speech feature data using a speech recognition model to be trained to obtain recognized text; determining a loss value representing the error between the recognized text and the labeled text based on a preset loss function; adjusting the parameters of the speech recognition model based on the loss value; and determining the adjusted speech recognition model as the target speech recognition model after training, in response to the initial speech recognition model meeting a preset training termination condition. This disclosure improves the recognition speed of the speech recognition model, eliminates the need to modify the loss function, and enhances the universality of the model training method.

Description

Technical Field

[0001] This disclosure relates to the field of computer technology, and in particular to a method and apparatus for training a speech recognition model, a speech recognition method and apparatus, a computer-readable storage medium, and an electronic device. Background Technology

[0002] Automatic speech recognition (ASR) is the study of speech, using speech signal processing and pattern recognition to enable machines to automatically recognize human speech. Currently, commonly used speech recognition models can convert a user's speech signal into corresponding text.

[0003] In some situations, the recognition speed of speech recognition models is crucial. For example, some applications need to convert users' speech into text in real time and display it on the screen as subtitles. In such cases, the speed at which the speech recognition model outputs text directly affects the usability of the application. Therefore, improving the speech recognition speed of speech recognition models is a problem that needs to be solved. Summary of the Invention

[0004] To address the aforementioned technical problems, this disclosure is proposed. Embodiments of this disclosure provide a method and apparatus for training a speech recognition model, a speech recognition method and apparatus, a computer-readable storage medium, and an electronic device.

[0005] Embodiments of this disclosure provide a method for training a speech recognition model. The method includes: extracting initial speech feature data from sample speech signals, wherein the sample speech signals correspond to preset labeled text; determining sample speech feature data for training the speech recognition model based on the length of the initial speech feature data; recognizing the sample speech feature data using the speech recognition model to be trained to obtain recognized text; determining a loss value representing the error between the recognized text and the labeled text based on a preset loss function; adjusting the parameters of the speech recognition model based on the loss value; and determining the speech recognition model with adjusted parameters as the target speech recognition model obtained after training, in response to the initial speech recognition model after parameter adjustment meeting a preset training termination condition.

[0006] According to another aspect of the present disclosure, a training apparatus for a speech recognition model is provided. The apparatus includes: a first extraction module for extracting initial speech feature data from sample speech signals, wherein the sample speech signals correspond to preset labeled text; a first determination module for determining sample speech feature data for training the speech recognition model based on the length of the initial speech feature data; a first recognition module for recognizing the sample speech feature data using the speech recognition model to be trained to obtain recognized text; a second determination module for determining a loss value representing the error between the recognized text and the labeled text based on a preset loss function; an adjustment module for adjusting the parameters of the speech recognition model based on the loss value; and a third determination module for determining the speech recognition model after parameter adjustment as the target speech recognition model obtained after training, in response to the initial speech recognition model after parameter adjustment meeting a preset training termination condition.

[0007] According to another aspect of the present disclosure, a speech recognition method is provided, the method comprising: acquiring a target speech signal; extracting speech feature data from the target speech signal; recognizing the speech feature data using a pre-trained target speech recognition model to obtain recognized text; and outputting the recognized text in response to determining the semantic integrity of the recognized text.

[0008] According to another aspect of the present disclosure, a speech recognition device is provided, the device comprising: an acquisition module for acquiring a target speech signal; a second extraction module for extracting speech feature data from the target speech signal; a second recognition module for recognizing the speech feature data using a pre-trained target speech recognition model to obtain recognized text; and an output module for outputting the recognized text in response to determining that the recognized text is semantically complete.

[0009] According to another aspect of the present disclosure, a computer-readable storage medium is provided, which stores a computer program for execution by a processor to implement the training method or speech recognition method of the above-described speech recognition model.

[0010] According to another aspect of the present disclosure, an electronic device is provided, comprising: a processor; a memory for storing processor-executable instructions; and a processor for reading executable instructions from the memory and executing the instructions to implement the training method or speech recognition method of the above-described speech recognition model.

[0011] Based on the speech recognition model training method and apparatus, speech recognition method and apparatus, computer-readable storage medium and electronic device provided in the above embodiments of this disclosure, when training the speech recognition model, initial speech feature data is first extracted from sample speech signals. Then, based on the length of the initial speech feature data, sample speech feature data is determined. The speech recognition model is then trained using the sample speech feature data and the corresponding labeled text. When the speech recognition model meets the training termination condition, the trained target speech recognition model is obtained. Since the length of the sample speech feature data used during training is adjusted, the trained target speech recognition model can accurately recognize shorter speech signals. When using the speech recognition model for speech recognition, only a shorter speech signal (i.e., a semantically incomplete speech signal) needs to be input into the target speech recognition model to enable the model to accurately output the corresponding text, thereby improving the recognition speed of the speech recognition model. No modification to the loss function is required, thus improving the universality of the model training method.

[0012] The technical solutions of this disclosure will be further described in detail below with reference to the accompanying drawings and embodiments. Attached Figure Description

[0013] The above and other objects, features, and advantages of this disclosure will become more apparent from the more detailed description of the embodiments thereof in conjunction with the accompanying drawings. The drawings are provided to further illustrate the embodiments of this disclosure and form part of the specification. They are used together with the embodiments of this disclosure to explain the disclosure and do not constitute a limitation thereof. In the drawings, the same reference numerals generally represent the same components or steps;

[0014] Figure 1 This is the system diagram to which this disclosure applies;

[0015] Figure 2 This is a flowchart illustrating a training method for a speech recognition model provided in an exemplary embodiment of this disclosure.

[0016] Figure 3 This is a flowchart illustrating a training method for a speech recognition model provided in another exemplary embodiment of this disclosure;

[0017] Figure 4 This is a flowchart illustrating a training method for a speech recognition model provided in another exemplary embodiment of this disclosure;

[0018] Figure 5 This is a schematic flowchart of a speech recognition method provided in an exemplary embodiment of this disclosure;

[0019] Figure 6 This is a schematic diagram of the structure of a training apparatus for a speech recognition model provided in an exemplary embodiment of this disclosure;

[0020] Figure 7 This is a schematic diagram of the structure of a training apparatus for a speech recognition model provided in another exemplary embodiment of this disclosure;

[0021] Figure 8 This is a schematic diagram of the structure of a speech recognition device provided in an exemplary embodiment of the present disclosure;

[0022] Figure 9 This is a structural diagram of an electronic device provided in an exemplary embodiment of this disclosure. Detailed Implementation

[0023] Hereinafter, exemplary embodiments according to the present disclosure will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are merely some embodiments of the present disclosure, and not all embodiments of the present disclosure, and it should be understood that the present disclosure is not limited to the exemplary embodiments described herein.

[0024] It should be noted that, unless otherwise specifically stated, the relative arrangement, numerical expressions, and values ​​of the components and steps set forth in these embodiments do not limit the scope of this disclosure.

[0025] Those skilled in the art will understand that the terms "first," "second," etc., in the embodiments of this disclosure are only used to distinguish different steps, devices, or modules, and do not represent any specific technical meaning, nor do they indicate a necessary logical order between them.

[0026] It should also be understood that in the embodiments disclosed herein, "a plurality of" may refer to two or more, and "at least one" may refer to one, two or more.

[0027] It should also be understood that any component, data or structure mentioned in the embodiments of this disclosure can generally be understood as one or more unless expressly defined or given to the contrary in the context.

[0028] Furthermore, the term "and / or" in this disclosure is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. Additionally, the character " / " in this disclosure generally indicates that the preceding and following related objects have an "or" relationship.

[0029] It should also be understood that the description of the various embodiments in this disclosure emphasizes the differences between the various embodiments, and the similarities or similarities can be referred to each other. For the sake of brevity, they will not be described in detail.

[0030] At the same time, it should be understood that, for ease of description, the dimensions of the various parts shown in the accompanying drawings are not drawn according to actual scale.

[0031] The following description of at least one exemplary embodiment is merely illustrative and is in no way intended to limit this disclosure or its application or use.

[0032] Techniques, methods, and equipment known to those skilled in the art may not be discussed in detail, but where appropriate, such techniques, methods, and equipment should be considered part of the specification.

[0033] It should be noted that similar labels and letters in the following figures indicate similar items; therefore, once an item is defined in one figure, it does not need to be discussed further in subsequent figures.

[0034] The embodiments disclosed herein can be applied to electronic devices such as terminal devices, computer systems, and servers, and can operate together with a wide range of other general-purpose or special-purpose computing system environments or configurations. Examples of well-known terminal devices, computing systems, environments, and / or configurations suitable for use with electronic devices such as terminal devices, computer systems, and servers include, but are not limited to: personal computer systems, server computer systems, thin clients, thick clients, handheld or laptop devices, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputer systems, mainframe computer systems, and distributed cloud computing environments including any of the above systems, etc.

[0035] Electronic devices such as terminal devices, computer systems, and servers can be described in the general context of computer system executable instructions (such as program modules) executed by a computer system. Typically, program modules can include routines, programs, object programs, components, logic, data structures, etc., which perform specific tasks or implement specific abstract data types. Computer systems / servers can be implemented in distributed cloud computing environments, where tasks are executed by remote processing devices linked through communication networks. In distributed cloud computing environments, program modules can reside on local or remote computing system storage media, including storage devices.

[0036] Application Overview

[0037] Existing methods for training neural network-based speech recognition models typically involve modifying the training loss. A penalty term added during the loss function calculation can influence the model's parameter updates through backpropagation, thereby incentivizing the model to recognize results faster. However, this approach is strongly correlated with the training loss and only applies to specific loss functions (such as the Transducer loss function). It is not applicable to other loss functions (such as the CTC (Connectionist Temporal Classification) loss function). Therefore, the method of adding a penalty term during the loss function calculation to improve the recognition speed of speech recognition models lacks universality for training with different loss functions.

[0038] To improve the recognition speed of different types of speech recognition models and enhance the universality of model training methods, this embodiment adjusts the length of speech feature data during speech recognition model training and then uses the adjusted speech feature data for model training. This allows the model to accurately output the corresponding text even with a short input speech signal, thereby improving the recognition speed of the speech recognition model without modifying the loss function and enhancing the universality of the model training method.

[0039] Exemplary System

[0040] Figure 1 An exemplary system architecture 100 is shown for a training method or apparatus for a speech recognition model to which embodiments of the present disclosure may be applied.

[0041] like Figure 1 As shown, system architecture 100 may include terminal device 101, network 102, and server 103. Network 102 is used as a medium to provide a communication link between terminal device 101 and server 103. Network 102 may include various connection types, such as wired or wireless communication links or fiber optic cables, etc.

[0042] Users can use terminal device 101 to interact with server 103 via network 102 to receive or send messages, etc. Various client applications can be installed on terminal device 101, such as human-computer interaction applications, search applications, web browser applications, instant messaging tools, etc.

[0043] Terminal device 101 can be various electronic devices, including but not limited to mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), vehicle terminals, etc., as well as fixed terminals such as digital TVs, desktop computers, etc.

[0044] Server 103 can be a server that provides various services, such as a background model training server that trains a speech recognition model using sample speech signals uploaded by terminal device 101. The background model training server can train a target speech recognition model. Server 103 can further send the trained target speech recognition model to terminal device 101, or use the target speech recognition model to recognize the speech signals received from terminal device 101 and feed back the recognized text to terminal device 101.

[0045] It should be noted that the speech recognition model training method or speech recognition method provided in the embodiments of this disclosure can be executed by the server 103 or by the terminal device 101. Accordingly, the speech recognition model training device or speech recognition device can be set in the server 103 or in the terminal device 101.

[0046] It should be understood that Figure 1 The number of terminal devices, networks, and servers shown is merely illustrative. Depending on implementation needs, any number of terminal devices, networks, and servers can be included. If the sample speech signals used to train the model do not need to be obtained remotely, the above system architecture may exclude the network and only include servers or terminal devices.

[0047] Exemplary methods

[0048] Figure 2 This is a schematic flowchart illustrating a training method for a speech recognition model provided in an exemplary embodiment of this disclosure. This embodiment can be applied to electronic devices (such as...). Figure 1 On the terminal device 101 or server 103 shown, such as Figure 2 As shown, the method includes the following steps:

[0049] Step 201: Extract initial speech feature data from the sample speech signal.

[0050] In this embodiment, the electronic device can extract initial speech feature data from sample speech signals, wherein the sample speech signals correspond to preset labeled text.

[0051] The initial speech feature data may include, but is not limited to, at least one of the following types of feature data: FBANK (Mel-scale Filter Bank) features, MFCC (Mel-scale Frequency Cepstral Coefficients) features, etc. Electronic devices can extract the initial speech feature data by using methods that extract the corresponding type of feature data.

[0052] The above-mentioned annotation text is a pre-set text that represents the correct semantics of the sample speech signal.

[0053] Step 202: Based on the length of the initial speech feature data, determine the sample speech feature data to be used to train the speech recognition model.

[0054] In this embodiment, the electronic device can determine the sample speech feature data used to train the speech recognition model based on the length of the initial speech feature data. Here, the length refers to the length of the time dimension, which can represent the duration of the sample speech signal; that is, the length of the sample speech signal is the same as the length of the sample speech feature data. For example, when the initial speech feature data is an FBANK feature, the initial speech feature data represents the correspondence between time and frequency, that is, the distribution of signal strength at different frequencies corresponding to each time point. The time range representing the data distribution of signal strength at different frequencies is the length of the initial speech feature data.

[0055] As an example, the length of the initial speech feature data can be randomly compressed, or the length of the initial speech feature data can be compressed according to a set length, and the compressed initial speech feature data can be determined as the sample speech feature data.

[0056] Step 203: Using the speech recognition model to be trained, the sample speech feature data is recognized to obtain the recognized text.

[0057] In this embodiment, the electronic device can use the speech recognition model to be trained to recognize the sample speech feature data and obtain the recognized text.

[0058] Speech recognition models can be implemented using various structural types. Typically, a speech recognition model includes an encoding network and a decoding network. The encoding network encodes the input sample speech feature data to obtain encoded data, and the decoding network decodes the encoded data to obtain speech prediction data. The speech prediction data represents the probability that the recognized text output by the speech recognition model matches the labeled text. As an example, the encoding network can be implemented using neural networks such as Conformer and Transformer, and the decoding network can be implemented using networks such as Transformer networks or CTC (Connectionist Temporal Classification) decoding networks.

[0059] Step 204: Based on a preset loss function, determine the loss value that represents the error between the recognized text and the labeled text.

[0060] In this embodiment, the electronic device can determine a loss value representing the error between the recognized text and the labeled text based on a preset loss function.

[0061] The loss function described above corresponds to the structural type of the speech recognition model, and the specific type of loss function is not limited in this embodiment. For example, the loss function may include, but is not limited to, at least one of the following: CTC loss function, Transducer loss function, cross-entropy loss function, etc.

[0062] As an example, when the decoding network in the speech recognition model is constructed by a CTC decoder, the loss function can be expressed as follows (1):

[0063] L = -logp ctc (y|X) (1)

[0064] Where X represents the encoded data, y represents the output predicted speech label sequence (i.e., the recognized text), and p ctc (y|X) represents the probability that y matches the labeled speech tag sequence (i.e., labeled text) when the encoded data input to the CTC decoder is X. Taking the negative logarithm of this probability value and minimizing L during training is equivalent to maximizing this probability value.

[0065] Step 205: Adjust the parameters of the speech recognition model based on the loss value.

[0066] In this embodiment, the electronic device can adjust the parameters of the speech recognition model based on the loss value.

[0067] Specifically, gradient descent and backpropagation methods can be used to iteratively update the parameters of the speech recognition model, gradually reducing the aforementioned loss value. Typically, steps 201-205 above need to be executed once per iteration, i.e., the model is iteratively trained using multiple sets of training samples (including sample speech signals and corresponding labeled text). The adjusted speech recognition model obtained after each iteration serves as the initial speech recognition model for the next iteration, and this process is repeated multiple times until the adjusted speech recognition model meets the preset training termination conditions.

[0068] Step 206: In response to the initial speech recognition model after parameter adjustment meeting the preset training termination condition, the speech recognition model after parameter adjustment is determined as the target speech recognition model obtained after training.

[0069] In this embodiment, the electronic device can determine the speech recognition model with adjusted parameters as the target speech recognition model obtained after training if the initial speech recognition model after parameter adjustment meets the preset training termination condition.

[0070] Specifically, steps 201-205 are iteratively executed using multiple sets of training samples. After each training iteration, it is determined whether the current model meets the training termination condition. When the training termination condition is met, the model with adjusted parameters is the target speech recognition model obtained after training. The training termination condition may include, but is not limited to, at least one of the following: the loss value of the above loss function converges, the training time exceeds a preset duration, and the number of training iterations exceeds a preset number.

[0071] The speech recognition model training method provided in the above embodiments of this disclosure first extracts initial speech feature data from sample speech signals during training. Then, based on the length of the initial speech feature data, it determines the sample speech feature data. The speech recognition model is then trained using the sample speech feature data and the corresponding labeled text. When the speech recognition model meets the training termination condition, the trained target speech recognition model is obtained. Since the length of the sample speech feature data used during training is adjusted, the trained target speech recognition model can accurately recognize shorter speech signals. When using the speech recognition model for speech recognition, only a shorter speech signal (i.e., a semantically incomplete speech signal) needs to be input into the target speech recognition model to enable the model to accurately output the corresponding text, thereby improving the recognition speed of the speech recognition model. This eliminates the need to modify the loss function and improves the universality of the model training method.

[0072] In some alternative implementations, such as Figure 3 As shown, step 202 includes:

[0073] Step 2021: Randomly determine the target length of the speech feature data to be deleted from the initial speech feature data within a preset length range.

[0074] As an example, the preset length range is represented as [0, T], and the target length is represented as t, where T is any preset value. Typically, the distribution of random samples from [0, T] can follow a continuous uniform distribution, as shown in equation (2) below:

[0075]

[0076] Where [a, b] is the range of sampled values, and in this embodiment, the range of sampled values ​​is [0, T], x is the sampled value, and f(x) is the probability when the sampled value is x.

[0077] Step 2022: Determine the sample speech feature data based on the target length of the speech feature data to be deleted and the length of the initial speech feature data.

[0078] As an example, let the length of the initial speech feature data be W and the target length be t. We can randomly determine the starting position for deleting the speech feature data to be deleted from the initial speech feature data (or determine the starting or ending position of the initial speech feature data as the starting position for deletion). Based on the starting position for deletion and the target length t, we can determine the speech feature data to be deleted from the initial speech feature data and delete the speech feature data to be deleted from the initial speech feature data, thereby obtaining sample speech feature data of length Wt.

[0079] This embodiment randomly determines the target length of the speech feature data to be deleted, and then deletes the speech feature data to be deleted from the initial speech feature data. This achieves random compression of the length of the initial speech feature data, improves the diversity of the sample speech feature data after length compression, and thus helps to improve the speed and accuracy of real-time speech recognition of the trained target speech recognition model.

[0080] In some alternative implementations, such as Figure 4 As shown, step 2022 includes:

[0081] Step 20221: Compare the target length of the speech feature data to be deleted with the length of the initial speech feature data, and determine whether the initial speech feature data meets the compression conditions based on the comparison result.

[0082] Specifically, to ensure the integrity of the compressed sample speech feature data is not significantly compromised, it is necessary to compare the target length of the speech feature data to be deleted with the length of the initial speech feature data. For example, if the comparison result indicates that the difference between the length of the initial speech feature data and the target length is less than a preset threshold, or that the ratio of the target length to the initial speech feature data is greater than a preset ratio, it means the length of the retained speech feature data after deletion is too short, and the initial speech feature data is determined not to meet the compression conditions. Conversely, if the comparison result indicates that the difference between the length of the initial speech feature data and the target length is greater than or equal to a preset threshold, or that the ratio of the target length to the initial speech feature data is less than or equal to a preset ratio, it means the length of the retained speech feature data after deletion is sufficiently long, and the initial speech feature data is determined to meet the compression conditions.

[0083] Step 20222: In response to determining that the initial speech feature data meets the compression conditions, determine the deletion start position from the initial speech feature data.

[0084] Specifically, the starting position for deletion can be determined in various ways, such as randomly determining the starting position for deletion from the initial speech feature data.

[0085] Optionally, the electronic device can determine the deletion start position from at least one target position of the initial speech feature data. The at least one target position includes at least one of the following: the start position of the initial speech feature data, and the end position of the initial speech feature data.

[0086] Typically, speech recognition models recognize user-generated speech signals in real time. The input signal begins when the user speaks, and after a period of time, the model can perform speech recognition based on the acquired speech signal. Therefore, a segment of speech feature data of the target length before the end of the initial speech feature data can be deleted. This allows the trained speech recognition model to correctly and quickly recognize shorter speech signals. Furthermore, deleting this segment allows the target speech recognition model to accurately and quickly recognize speech signals excluding the initial segment even if speech signal acquisition is interrupted during the recognition process. Determining the deletion start position in this way further increases the diversity of sample speech feature data, contributing to improved recognition accuracy of the trained target speech recognition model.

[0087] Step 20223: Based on the deletion start position and target length, determine the speech feature data to be deleted from the initial speech feature data, and delete the speech feature data to be deleted from the initial speech feature data to obtain sample speech feature data.

[0088] As an example, if the deletion start position is the end position of the initial speech feature data, then the speech feature data of the target length before the end position can be determined as the speech feature data to be deleted; if the deletion start position is the start position of the initial speech feature data, then the speech feature data of the target length after the start position can be determined as the speech feature data to be deleted.

[0089] This embodiment compares the target length and the length of the initial speech feature data. Based on the comparison result, when the initial speech feature data meets the compression conditions, the speech feature data to be deleted is deleted from the initial speech feature data. This achieves effective control over the integrity of the sample speech feature data obtained after deletion, avoiding the deletion of a large amount of effective information from the initial speech feature data, thereby helping to further improve the recognition accuracy of the trained target speech recognition model.

[0090] In some alternative implementations, the method also includes:

[0091] In response to the determination that the initial speech feature data does not meet the compression conditions, the initial speech feature data is determined as sample speech feature data.

[0092] Specifically, if the initial speech feature data does not meet the compression conditions, it means that the length of the speech feature data to be deleted is too long, i.e., the length of the retained speech feature data is too short. This results in excessive damage to the integrity of the sample speech feature data used for training, affecting the recognition accuracy of the trained target speech recognition model. In this case, not compressing the sample speech feature data can ensure the integrity of the information contained in the sample speech feature data, while maintaining the length diversity of the large number of sample speech feature data required for model training, thereby helping to improve the recognition accuracy of the trained target speech recognition model.

[0093] In some alternative implementations, step 20221 can be performed as follows:

[0094] First, the upper limit of the deletion length is determined based on the preset ratio and the length of the initial speech feature data.

[0095] As an example, if the preset ratio is 1 / 2 and the length of the initial speech feature data is W, then the maximum length to be deleted is W / 2.

[0096] Determine if the target length exceeds the maximum deletion length limit.

[0097] If the target length does not exceed the upper limit of the deletion length, the initial speech feature data is determined to meet the compression conditions.

[0098] If the target length exceeds the upper limit of the deletion length, it is determined that the initial speech feature data does not meet the compression conditions.

[0099] Specifically, as an example, let the target length be t. If t < W / 2, it is determined that the target length does not exceed the upper limit of the deletion length; if t ≥ W / 2, it is determined that the target length exceeds the upper limit of the deletion length. If the above deletion start position is the end position of the initial speech feature data, if t < W / 2, then the initial speech feature data in the time range of [0, W] is pruned at the end, and the time range of the obtained sample feature data is [0, Wt].

[0100] It should be noted that the criterion for determining whether the target length exceeds the upper limit of the deletion length can be set arbitrarily. For example, unlike the example above, if t≤W / 2, the target length is determined not to exceed the upper limit of the deletion length; if t is greater than W / 2, the target length is determined to exceed the upper limit of the deletion length.

[0101] This embodiment determines the upper limit of the deletion length and judges whether the compression conditions are met based on the upper limit of the deletion length, thereby further limiting the deletion of too much effective information from the initial speech feature data. The target speech recognition model obtained by training with the sample speech feature data obtained in this embodiment has improved its recognition speed and recognition accuracy.

[0102] Figure 5 This is a schematic flowchart of a speech recognition method provided in an exemplary embodiment of this disclosure. This embodiment can be applied to electronic devices (such as...). Figure 1 On the terminal device 101 or server 103 shown, such as Figure 5 As shown, the method includes the following steps:

[0103] Step 501: Acquire the target speech signal.

[0104] In this embodiment, the electronic device can acquire the target voice signal locally or remotely. The target voice signal can be a signal acquired in real time by the voice acquisition device.

[0105] Step 502: Extract speech feature data from the target speech signal.

[0106] In this embodiment, the electronic device can extract speech feature data from the target speech signal. The method for extracting speech feature data from the target speech signal can be referred to the above. Figure 2 The content described in step 201 of the corresponding embodiment will not be repeated here.

[0107] Step 503: Using a pre-trained target speech recognition model, the speech feature data is recognized to obtain the recognized text.

[0108] In this embodiment, the electronic device can use a pre-trained target speech recognition model to recognize speech feature data and obtain recognized text.

[0109] Among them, the target speech recognition model can be pre-based on the above. Figures 2-4 The speech recognition model is trained using the training method described in any of the corresponding embodiments. For a detailed description of the target speech recognition model, please refer to the above embodiments; it will not be repeated here.

[0110] Step 504: In response to determining the semantic integrity of the identified text, output the identified text.

[0111] In this embodiment, the electronic device can output the identified text in response to determining the semantic integrity of the identified text.

[0112] Specifically, since the target speech recognition model performs speech recognition in real time, the target speech signal may be a complete speech signal or an incomplete speech signal. Because the target speech recognition model is pre-trained using speech feature data that has undergone length adjustment, it can recognize incomplete speech signals. Therefore, when the model performs speech recognition in real time, even if the acquired target speech signal is incomplete, it can still obtain a recognized text with complete semantics and output the recognized text, thus achieving rapid output of recognized text with complete semantics. If the semantics of the recognized text obtained based on the current target speech signal are incomplete, new speech signals can be received and combined with the aforementioned target speech signal to continue speech recognition until a recognized text with complete semantics is obtained.

[0113] The semantically complete recognized text can be a complete sentence, a complete word, or a complete phrase. For example, if the target speech signal acquired in real time corresponds to the user's voice "kicking the ball," and the target speech recognition model recognizes the signal and obtains the recognized text "kicking football," then the semantic completeness of the recognized text is determined, and the recognized text can be output.

[0114] Optionally, the method of outputting the recognized text may include, but is not limited to, at least one of the following: displaying the recognized text on the screen, sending the recognized text to other devices connected to the aforementioned electronic device, etc.

[0115] The speech recognition method provided in the above embodiments of this disclosure performs real-time speech recognition by using a pre-trained target speech recognition model. Since the target speech recognition model is pre-trained using sample speech feature data with adjusted length, it can recognize incomplete speech signals. Therefore, when recognizing speech signals, it can quickly output recognized text, which reduces the delay time of real-time output of recognized text.

[0116] Exemplary device

[0117] Figure 6 This is a schematic diagram of the structure of a training apparatus for a speech recognition model provided in an exemplary embodiment of this disclosure. This embodiment can be applied to electronic devices, such as... Figure 6As shown, the training device for the speech recognition model includes: a first extraction module 601, used to extract initial speech feature data from sample speech signals, wherein the sample speech signals correspond to preset labeled text; a first determination module 602, used to determine sample speech feature data for training the speech recognition model based on the length of the initial speech feature data; a first recognition module 603, used to recognize the sample speech feature data using the speech recognition model to be trained, to obtain recognized text; a second determination module 604, used to determine a loss value representing the error between the recognized text and the labeled text based on a preset loss function; an adjustment module 605, used to adjust the parameters of the speech recognition model based on the loss value; and a third determination module 606, used to determine the speech recognition model after parameter adjustment as the target speech recognition model obtained after training, in response to the initial speech recognition model after parameter adjustment meeting a preset training termination condition.

[0118] In this embodiment, the first extraction module 601 can extract initial speech feature data from the sample speech signal, wherein the sample speech signal corresponds to a preset labeled text.

[0119] The initial speech feature data may include, but is not limited to, at least one of the following types of feature data: FBANK (Mel-scale Filter Bank) features, MFCC (Mel-scale Frequency Cepstral Coefficients) features, etc. Electronic devices can extract the initial speech feature data by using methods that extract the corresponding type of feature data.

[0120] The above-mentioned annotation text is a pre-set text that represents the correct semantics of the sample speech signal.

[0121] In this embodiment, the first determining module 602 can determine the sample speech feature data used to train the speech recognition model based on the length of the initial speech feature data. Here, the length refers to the length of the time dimension, which can represent the duration of the sample speech signal; that is, the length of the sample speech signal is the same as the length of the sample speech feature data. For example, when the initial speech feature data is an FBANK feature, the initial speech feature data represents the correspondence between time and frequency, that is, the distribution of signal intensity at different frequencies corresponding to each time point. The time range representing the data distribution of signal intensity at different frequencies is the length of the initial speech feature data.

[0122] As an example, the length of the initial speech feature data can be randomly compressed, or the length of the initial speech feature data can be compressed according to a set length, and the compressed initial speech feature data can be determined as the sample speech feature data.

[0123] In this embodiment, the first recognition module 603 can use the speech recognition model to be trained to recognize the sample speech feature data and obtain the recognized text.

[0124] Speech recognition models can be implemented using various structural types. Typically, a speech recognition model includes an encoding network and a decoding network. The encoding network encodes the input sample speech feature data to obtain encoded data, and the decoding network decodes the encoded data to obtain speech prediction data. The speech prediction data represents the probability that the recognized text output by the speech recognition model matches the labeled text. As an example, the encoding network can be implemented using neural networks such as Conformer and Transformer, and the decoding network can be implemented using networks such as Transformer networks or CTC (Connectionist Temporal Classification) decoding networks.

[0125] In this embodiment, the second determining module 604 can determine a loss value representing the error between the recognized text and the labeled text based on a preset loss function.

[0126] The loss function described above corresponds to the structural type of the speech recognition model, and the specific type of loss function is not limited in this embodiment. For example, the loss function may include, but is not limited to, at least one of the following: CTC loss function, Transducer loss function, cross-entropy loss function, etc.

[0127] In this embodiment, the adjustment module 605 can adjust the parameters of the speech recognition model based on the loss value.

[0128] Specifically, gradient descent and backpropagation methods can be used to iteratively update the parameters of the speech recognition model, gradually reducing the aforementioned loss value. Typically, steps 201-205 above need to be executed once per iteration, i.e., the model is iteratively trained using multiple sets of training samples (including sample speech signals and corresponding labeled text). The adjusted speech recognition model obtained after each iteration serves as the initial speech recognition model for the next iteration, and this process is repeated multiple times until the adjusted speech recognition model meets the preset training termination conditions.

[0129] In this embodiment, the third determining module 606 can determine the speech recognition model after adjusting the parameters as the target speech recognition model obtained after training in response to the initial speech recognition model after adjusting the parameters meeting the preset training termination condition.

[0130] Specifically, steps 601-605 above are repeatedly executed using multiple sets of training samples to iteratively train the model. After each training iteration, it is determined whether the current model meets the training termination condition. When the training termination condition is met, the model with adjusted parameters is the target speech recognition model obtained after training. The training termination condition may include, but is not limited to, at least one of the following: the loss value of the above loss function converges, the training time exceeds a preset duration, and the number of training iterations exceeds a preset number.

[0131] Reference Figure 7 , Figure 7 This is a schematic diagram of the structure of a training apparatus for a speech recognition model provided in another exemplary embodiment of this disclosure.

[0132] In some optional implementations, the first determining module 602 includes: a first determining unit 6021, used to randomly determine the target length of the speech feature data to be deleted in the initial speech feature data from a preset length range; and a second determining unit 6022, used to determine the sample speech feature data based on the target length of the speech feature data to be deleted and the length of the initial speech feature data.

[0133] In some optional implementations, the second determining unit 6022 includes: a comparison subunit 60221, used to compare the target length of the speech feature data to be deleted with the length of the initial speech feature data, and determine whether the initial speech feature data meets the compression conditions based on the comparison result; a first determining subunit 60222, used to determine the deletion start position from the initial speech feature data in response to determining that the initial speech feature data meets the compression conditions; and a second determining subunit 60223, used to determine the speech feature data to be deleted from the initial speech feature data based on the deletion start position and the target length, and delete the speech feature data to be deleted from the initial speech feature data to obtain sample speech feature data.

[0134] In some optional implementations, the second determining unit 6022 further includes a third determining subunit 60224, which is used to determine the initial speech feature data as sample speech feature data in response to determining that the initial speech feature data does not meet the compression conditions.

[0135] In some optional implementations, the comparison subunit 60221 is further used to: determine the upper limit of the deletion length based on a preset ratio and the length of the initial speech feature data; determine whether the target length exceeds the upper limit of the deletion length; if the target length does not exceed the upper limit of the deletion length, determine that the initial speech feature data meets the compression conditions; if the target length exceeds the upper limit of the deletion length, determine that the initial speech feature data does not meet the compression conditions.

[0136] In some alternative implementations, the first determining subunit 60222 is further configured to: determine a deletion start position from at least one target position of the initial speech feature data, wherein the at least one target position includes at least one of the following: the start position of the initial speech feature data, and the end position of the initial speech feature data.

[0137] The speech recognition model training apparatus provided in the above embodiments of this disclosure first extracts initial speech feature data from sample speech signals during speech recognition model training. Then, based on the length of the initial speech feature data, it determines the sample speech feature data. The speech recognition model is then trained using the sample speech feature data and the corresponding labeled text. When the speech recognition model meets the training termination condition, the trained target speech recognition model is obtained. Since the length of the sample speech feature data used during training is adjusted, the trained target speech recognition model can accurately recognize shorter speech signals. When using the speech recognition model for speech recognition, only a shorter speech signal (i.e., a semantically incomplete speech signal) needs to be input into the target speech recognition model to enable the model to accurately output the corresponding text, thereby improving the recognition speed of the speech recognition model. This eliminates the need to modify the loss function and improves the universality of the model training method.

[0138] Figure 8 This is a schematic diagram of the structure of a voice recognition device provided in an exemplary embodiment of this disclosure. This embodiment can be applied to electronic devices, such as... Figure 8 As shown, the speech recognition device includes: an acquisition module 801 for acquiring a target speech signal; a second extraction module 802 for extracting speech feature data from the target speech signal; a second recognition module 803 for recognizing the speech feature data using a pre-trained target speech recognition model to obtain recognized text; and an output module 804 for outputting the recognized text in response to determining the semantic integrity of the recognized text.

[0139] In this embodiment, the acquisition module 801 can acquire the target voice signal from a local source or from a remote source. The target voice signal can be a signal acquired in real time by the voice acquisition device.

[0140] In this embodiment, the second extraction module 802 can extract speech feature data from the target speech signal. The method for extracting speech feature data from the target speech signal can be referred to the above. Figure 2 The content described in step 201 of the corresponding embodiment will not be repeated here.

[0141] In this embodiment, the second recognition module 803 can use a pre-trained target speech recognition model to recognize speech feature data and obtain recognized text.

[0142] Among them, the target speech recognition model can be pre-based on the above. Figures 2-4 The speech recognition model is trained using the training method described in any of the corresponding embodiments. For a detailed description of the target speech recognition model, please refer to the above embodiments; it will not be repeated here.

[0143] In this embodiment, the output module 804 can output the recognized text in response to determining the semantic integrity of the recognized text.

[0144] Specifically, since the target speech recognition model performs speech recognition in real time, the target speech signal may be a complete speech signal or an incomplete speech signal. Because the target speech recognition model is pre-trained using speech feature data that has undergone length adjustment, it can recognize incomplete speech signals. Therefore, when the model performs speech recognition in real time, even if the acquired target speech signal is incomplete, it can still obtain a recognized text with complete semantics and output the recognized text, thus achieving rapid output of recognized text with complete semantics. If the semantics of the recognized text obtained based on the current target speech signal are incomplete, new speech signals can be received and combined with the aforementioned target speech signal to continue speech recognition until a recognized text with complete semantics is obtained.

[0145] The semantically complete recognized text can be a complete sentence, a complete word, or a complete phrase. For example, if the target speech signal acquired in real time corresponds to the user's voice "kicking the ball," and the target speech recognition model recognizes the signal and obtains the recognized text "kicking football," then the semantic completeness of the recognized text is determined, and the recognized text can be output.

[0146] The speech recognition device provided in the above embodiments of this disclosure performs real-time speech recognition by using a pre-trained target speech recognition model. Since the target speech recognition model is pre-trained using sample speech feature data with adjusted length, it can recognize incomplete speech signals. Therefore, when recognizing speech signals, it can quickly output recognized text, which reduces the delay time of real-time output of recognized text.

[0147] Exemplary electronic devices

[0148] Below, for reference Figure 9 To describe an electronic device according to embodiments of the present disclosure. The electronic device may be as follows: Figure 1 The terminal device 101 and server 103 shown, or either one or both, or a standalone device independent of them, can communicate with the terminal device 101 and server 103 to receive the collected input signals from them.

[0149] Figure 9A block diagram of an electronic device according to an embodiment of the present disclosure is shown.

[0150] like Figure 9 As shown, the electronic device 900 includes one or more processors 901 and memory 902.

[0151] The processor 901 may be a central processing unit (CPU) or other form of processing unit with data processing capabilities and / or instruction execution capabilities, and may control other components in the electronic device 900 to perform desired functions.

[0152] The memory 902 may include one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. Volatile memory may include, for example, random access memory (RAM) and / or cache memory. Non-volatile memory may include, for example, read-only memory (ROM), hard disk, flash memory, etc. One or more computer program instructions may be stored on the computer-readable storage medium, and the processor 901 may execute the program instructions to implement the training method or speech recognition method of the speech recognition model of the various embodiments of this disclosure described above, and / or other desired functions. Various contents, such as sample speech signals, may also be stored in the computer-readable storage medium.

[0153] In one example, the electronic device 900 may also include an input device 903 and an output device 904, which are interconnected via a bus system and / or other forms of connection mechanism (not shown).

[0154] For example, when the electronic device is a terminal device 101 or a server 103, the input device 903 can be a microphone, mouse, keyboard, or other devices used to input sample voice signals, various commands, etc. When the electronic device is a standalone device, the input device 903 can be a communication network connector used to receive input sample voice signals, various commands, etc. from the terminal device 101 and the server 103.

[0155] The output device 904 can output various information to the outside, including recognized text. The output device 904 may include, for example, a display, a speaker, a printer, and a communication network and its connected remote output devices, etc.

[0156] Of course, for the sake of simplicity, Figure 9 Only some of the components of the electronic device 900 relevant to this disclosure are shown, omitting components such as buses, input / output interfaces, etc. In addition, the electronic device 900 may include any other suitable components depending on the specific application.

[0157] Exemplary computer program products and computer-readable storage media

[0158] In addition to the methods and apparatus described above, embodiments of this disclosure may also be computer program products comprising computer program instructions that, when executed by a processor, cause the processor to perform the steps in the training method or speech recognition method of the speech recognition model according to various embodiments of this disclosure as described in the "Exemplary Methods" section of this specification.

[0159] The computer program product can be written in any combination of one or more programming languages ​​to perform the operations of the embodiments of this disclosure. The programming languages ​​include object-oriented programming languages ​​such as Java and C++, as well as conventional procedural programming languages ​​such as C or similar languages. The program code can be executed entirely on a user's computing device, partially on a user's computing device, as a standalone software package, partially on a user's computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0160] Furthermore, embodiments of this disclosure may also be computer-readable storage media storing computer program instructions thereon, which, when executed by a processor, cause the processor to perform the steps in the training method or speech recognition method of the speech recognition model according to various embodiments of this disclosure as described in the "Exemplary Methods" section above.

[0161] The computer-readable storage medium may be any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may, for example, include, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatuses, or devices, or any combination thereof. More specific examples of readable storage media (a non-exhaustive list) include: electrical connections having one or more wires, portable disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0162] The basic principles of this disclosure have been described above with reference to specific embodiments. However, it should be noted that the advantages, benefits, and effects mentioned in this disclosure are merely examples and not limitations, and should not be considered as essential features of each embodiment of this disclosure. Furthermore, the specific details disclosed above are for illustrative and facilitative purposes only, and are not limitations. These details do not limit the scope of this disclosure to the necessity of employing the aforementioned specific details for implementation.

[0163] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For system embodiments, since they largely correspond to method embodiments, the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments.

[0164] The block diagrams of devices, apparatuses, devices, and systems disclosed herein are merely illustrative examples and are not intended to require or imply that they must be connected, arranged, or configured in the manner shown in the block diagrams. As those skilled in the art will recognize, these devices, apparatuses, devices, and systems can be connected, arranged, and configured in any manner. Words such as “comprising,” “including,” “having,” etc., are open-ended terms meaning “including but not limited to,” and are used interchangeably with them. The terms “or” and “and” as used herein refer to the terms “and / or,” and are used interchangeably with them unless the context clearly indicates otherwise. The term “such as” as used herein refers to the phrase “such as but not limited to,” and is used interchangeably with it.

[0165] The methods and apparatus of this disclosure may be implemented in many ways. For example, they may be implemented by software, hardware, firmware, or any combination of software, hardware, and firmware. The above-described order of steps for the methods is for illustrative purposes only, and the steps of the methods of this disclosure are not limited to the order specifically described above unless otherwise specifically stated. Furthermore, in some embodiments, this disclosure may also be implemented as a program recorded on a recording medium, the program including machine-readable instructions for implementing the methods according to this disclosure. Thus, this disclosure also covers recording media storing programs for performing the methods according to this disclosure.

[0166] It should also be noted that in the apparatus, devices, and methods of this disclosure, the components or steps can be disassembled and / or recombined. These disassemblies and / or recombinations should be considered as equivalent solutions to this disclosure.

[0167] The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use this disclosure. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein may be applied to other aspects without departing from the scope of this disclosure. Therefore, this disclosure is not intended to be limited to the aspects shown herein, but rather to be carried out within the widest scope consistent with the principles and novel features disclosed herein.

[0168] The above description has been given for purposes of illustration and description. Furthermore, this description is not intended to limit the embodiments of this disclosure to the forms disclosed herein. Although numerous exemplary aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, alterations, additions, and sub-combinations thereof.

Claims

1. A method for training a speech recognition model, comprising: extracting initial speech feature data from a sample speech signal, wherein the sample speech signal corresponds to a preset labeled text; determining sample speech feature data for training a speech recognition model based on a length of the initial speech feature data; recognizing the sample speech feature data by using the speech recognition model to be trained to obtain recognized text; determining a loss value representing an error between the recognized text and the labeled text based on a preset loss function; adjusting parameters of the speech recognition model based on the loss value; in response to the initial speech recognition model after the parameters are adjusted meeting a preset training end condition, determining the speech recognition model after the parameters are adjusted as a target speech recognition model obtained after training; the determining of the sample speech feature data based on the length of the initial speech feature data and the target length of the speech feature data to be deleted comprises: randomly determining a target length of speech feature data to be deleted in the initial speech feature data from a preset length range; determining the sample speech feature data based on the target length of the speech feature data to be deleted and the length of the initial speech feature data.

2. The method of claim 1, wherein, the determining of the sample speech feature data based on the target length of the speech feature data to be deleted and the length of the initial speech feature data comprises: comparing the target length of the speech feature data to be deleted and the length of the initial speech feature data, and determining whether the initial speech feature data meets a compression condition based on a comparison result; in response to determining that the initial speech feature data meets the compression condition, determining a deletion start position from the initial speech feature data; based on the deletion start position and the target length, determining speech feature data to be deleted from the initial speech feature data, and deleting the speech feature data to be deleted from the initial speech feature data to obtain the sample speech feature data.

3. The method of claim 2, wherein, The method further comprises: in response to determining that the initial speech feature data does not meet the compression condition, determining the initial speech feature data as the sample speech feature data.

4. The method of claim 2, wherein, The comparing of the target length of the speech feature data to be deleted and the length of the initial speech feature data, and the determining of whether the initial speech feature data meets a compression condition based on a comparison result comprises: determining an upper limit of deletion length based on a preset ratio and the length of the initial speech feature data; determining whether the target length exceeds the upper limit of deletion length; if the target length does not exceed the upper limit of deletion length, determining that the initial speech feature data meets the compression condition; if the target length exceeds the upper limit of deletion length, determining that the initial speech feature data does not meet the compression condition.

5. The method of claim 2, wherein, The determining of the deletion start position from the initial speech feature data comprises: determine a deletion start position from at least one target position of the initial speech feature data, wherein the at least one target position comprises at least one of a start position of the initial speech feature data and an end position of the initial speech feature data. 6.A speech recognition method, comprising: obtaining a target speech signal; extracting speech feature data from the target speech signal; recognizing the speech feature data by using a pre-trained target speech recognition model to obtain recognized text, wherein the target speech recognition model is pre-trained based on the method of any one of claims 1-5; in response to determining that the recognized text is semantically complete, outputting the recognized text. 7.A training apparatus of a speech recognition model, comprising: a first extraction module configured to extract initial speech feature data from a sample speech signal, wherein the sample speech signal corresponds to a preset labeled text; a first determination module configured to determine sample speech feature data for training a speech recognition model based on a length of the initial speech feature data; a first recognition module configured to recognize the sample speech feature data by using a speech recognition model to be trained to obtain recognized text; a second determination module configured to determine a loss value representing an error between the recognized text and the labeled text based on a preset loss function; an adjustment module configured to adjust parameters of the speech recognition model based on the loss value; a third determination module configured to determine, in response to the initial speech recognition model after the parameters are adjusted meeting a preset training end condition, the speech recognition model after the parameters are adjusted as a target speech recognition model obtained after training. The first determination module comprises: a first determination unit configured to randomly determine a target length of speech feature data to be deleted in the initial speech feature data from a preset length range; a second determination unit configured to determine the sample speech feature data based on the target length of the speech feature data to be deleted and the length of the initial speech feature data. 8.A speech recognition apparatus, comprising: an acquisition module configured to acquire a target speech signal; a second extraction module configured to extract speech feature data from the target speech signal; a second recognition module configured to recognize the speech feature data by using a pre-trained target speech recognition model to obtain recognized text, wherein the target speech recognition model is pre-trained based on the method of any one of claims 1-5; an output module configured to output the recognized text in response to determining that the recognized text is semantically complete. 9.A computer-readable storage medium, the storage medium storing a computer program, the computer program being configured to be executed by a processor to implement the method of any one of claims 1-6. 10.An electronic device, comprising: a processor; a memory configured to store executable instructions of the processor; the processor configured to read the executable instructions from the memory and execute the instructions to implement the method of any one of claims 1-6.

Images