Speech recognition model training method, speech recognition method and device

Abstract

The application provides a speech recognition model training method, a speech recognition method and device, and relates to the technical field of speech processing. The method comprises the following steps: in each iteration process, a training sample set is obtained, the training sample set comprises multiple training samples, each training sample comprises a sample speech signal, a text corresponding to the sample speech signal and a label of the text, the label is used for indicating whether the text is a complete sentence, for each training sample in the training sample set, taking the acoustic feature of the sample speech signal in the training sample as the input of a speech recognition model, outputting the speech recognition text and the predicted label of the sample speech signal, adjusting the parameters of the speech recognition model according to the speech recognition text and the predicted label of the sample speech signal obtained in each iteration process, and the text corresponding to the sample speech signal and the label of the text, until a stop training condition is met, and a trained speech recognition model is obtained. The accuracy of continuous speech recognition can be improved.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to a speech recognition model training method, a speech recognition method, and an apparatus. Background Technology

[0002] Automatic speech recognition (ASR) is the process of converting audio captured by a microphone into text. End-to-end speech recognition is a current research hotspot in ASR tasks. Continuous speech recognition, in particular, refers to the continuous input of speech into a speech recognition system for processing and the output of recognized text.

[0003] In related technologies, when performing continuous speech recognition, continuous speech is input into the speech recognition system. The speech recognition system identifies the positions in the speech that need to be broken into sentences based on the volume and pause time of the speech. For example, it automatically breaks into sentences when it detects a long silence.

[0004] However, each user has a different speaking speed and speaking habits, which may result in longer pauses in a sentence. If the sentence breaks are identified based on volume and pause duration, it is easy to make misjudgments, which may split a sentence into two sentences, resulting in semantic incoherence and low speech recognition accuracy. Summary of the Invention

[0005] This application provides a speech recognition model training method, a speech recognition method, and an apparatus, which can determine the accurate punctuation position based on the semantics of the recognized text, avoid misjudgment that causes semantic incoherence, and improve the accuracy of continuous speech recognition.

[0006] Firstly, this application provides a method for training a speech recognition model, including:

[0007] During any iteration, a training sample set is obtained, which includes multiple training samples. Each training sample includes a sample speech signal, the text corresponding to the sample speech signal, and a label for the text. The label is used to indicate whether the text is a complete sentence.

[0008] For each training sample in the training sample set, the acoustic features of the sample speech signal in the training sample are used as the input of the speech recognition model, and the speech recognition text and predicted label of the sample speech signal are output.

[0009] Based on the speech recognition text and predicted label of the sample speech signal obtained in each iteration, as well as the text corresponding to the sample speech signal and the label of the text, the parameters of the speech recognition model are adjusted until the training stop condition is met.

[0010] The speech recognition model determined by the iterative process that satisfies the aforementioned stopping training condition is defined as the trained speech recognition model.

[0011] Secondly, this application provides a speech recognition method, including:

[0012] Acquire the continuous speech signal to be recognized;

[0013] The first speech segment of the continuous speech signal to be recognized is input into the speech recognition model to obtain the first recognized text of the first speech segment and the label of the first recognized text. The label is used to indicate whether the recognized text is a complete sentence. The speech recognition model is trained according to the method described in the first aspect.

[0014] If it is determined that the label of the first recognized text indicates that the first recognized text is not a complete sentence, then the target speech segment after concatenating the second speech segment with the first speech segment is re-inputted into the speech recognition model to obtain the target recognized text of the target speech segment and the label of the target recognized text. The second speech segment is the next speech segment of the first speech segment.

[0015] If it is determined that the label of the target recognition text indicates that the target recognition text is a complete sentence, the target recognition text is output.

[0016] Thirdly, this application provides a speech recognition model training device, comprising:

[0017] The acquisition module is used to acquire a training sample set during any iteration. The training sample set includes multiple training samples. Each training sample includes a sample speech signal, the text corresponding to the sample speech signal, and a label for the text. The label is used to indicate whether the text is a complete sentence.

[0018] The model training module is used to: for each training sample in the training sample set, take the acoustic features of the sample speech signal in the training sample as the input of the speech recognition model, and output the speech recognition text and predicted label of the sample speech signal;

[0019] Based on the speech recognition text and predicted label of the sample speech signal obtained in each iteration, as well as the text corresponding to the sample speech signal and the label of the text, the parameters of the speech recognition model are adjusted until the training stop condition is met.

[0020] The speech recognition model determined by the iterative process that satisfies the aforementioned stopping training condition is defined as the trained speech recognition model.

[0021] Fourthly, this application provides a voice recognition device, comprising:

[0022] The acquisition module is used to acquire the continuous speech signal to be recognized;

[0023] The processing module is used to input the first speech segment of the continuous speech signal to be recognized into the speech recognition model to obtain the first recognized text of the first speech segment and the label of the first recognized text. The label is used to indicate whether the recognized text is a complete sentence. The speech recognition model is trained according to the method described in the first aspect.

[0024] The processing module is further configured to: if it is determined that the label of the first recognized text indicates that the first recognized text is not a complete sentence, then re-input the target speech segment after concatenating the second speech segment with the first speech segment into the speech recognition model to obtain the target recognized text of the target speech segment and the label of the target recognized text, wherein the second speech segment is the next speech segment of the first speech segment;

[0025] If it is determined that the label of the target recognition text indicates that the target recognition text is a complete sentence, the target recognition text is output.

[0026] Fifthly, this application provides a computer device, including: a processor and a memory, the memory for storing a computer program, and the processor for calling and running the computer program stored in the memory to perform the method of the first aspect or the second aspect.

[0027] Sixthly, this application provides a computer-readable storage medium for storing a computer program that causes a computer to perform the method of the first aspect or the second aspect.

[0028] In a seventh aspect, this application provides a computer program product, including a computer program, characterized in that, when the computer program is executed by a processor, it implements the steps of the method described in the first or second aspect.

[0029] In summary, the speech recognition model training method provided in this application trains the speech recognition model based on a training sample set. Each training sample includes a sample speech signal, the corresponding text, and a ground truth label for that text. The label indicates whether the text corresponding to the sample speech signal is a complete sentence. During training, the acoustic features of the sample speech signals in the training samples are used as input to the speech recognition model, which outputs the speech recognition text and its predicted label. Then, based on the speech recognition text and predicted label, as well as the corresponding text and its ground truth label, the parameters of the speech recognition model are adjusted until the training stop condition is met, resulting in a trained speech recognition model. Therefore, the trained speech recognition model can learn whether the text corresponding to a speech signal is a complete sentence. During continuous speech recognition, it can determine the accurate punctuation position based on the semantics of the recognized text and automatically punctuate, avoiding misjudgments that cause semantic incoherence and improving the accuracy of continuous speech recognition. Attached Figure Description

[0030] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0031] Figure 1 This is a schematic diagram illustrating an application scenario of a speech recognition method provided in an embodiment of this application;

[0032] Figure 2 A flowchart illustrating a speech recognition model training method provided in this application embodiment;

[0033] Figure 3 A schematic diagram illustrating the processing of acoustic features of a sample speech signal by an encoder-decoder, as provided in an embodiment of this application.

[0034] Figure 4 A schematic diagram of a Transformer-based speech recognition model structure provided for an embodiment of this application;

[0035] Figure 5 A flowchart illustrating a speech recognition method provided in an embodiment of this application;

[0036] Figure 6 This is a schematic diagram illustrating the process of a speech recognition method provided in an embodiment of this application;

[0037] Figure 7 A schematic diagram of the structure of a speech recognition model training device provided in an embodiment of this application;

[0038] Figure 8 This is a schematic diagram of the structure of a speech recognition device provided in an embodiment of this application;

[0039] Figure 9 This is a schematic block diagram of the computer device 300 provided in the embodiments of this application. Detailed Implementation

[0040] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0041] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or server that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or devices.

[0042] 1. Artificial Intelligence (AI) is the theory, methods, technology, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to achieve optimal results. In other words, AI is a comprehensive technology within computer science that attempts to understand the essence of intelligence and produce a new kind of intelligent machine that can react in a way similar to human intelligence. AI studies the design principles and implementation methods of various intelligent machines, enabling them to have perception, reasoning, and decision-making capabilities. AI technology is a comprehensive discipline involving a wide range of fields, encompassing both hardware and software technologies. Fundamental AI technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing technology, operating / interactive systems, and mechatronics. AI software technologies mainly include computer vision, speech processing, natural language processing, and machine learning / deep learning.

[0043] 2. Key technologies in speech technology include Automatic Speech Recognition (ASR), Text-to-Speech (TTS), and Voiceprint Recognition. Enabling computers to see, hear, speak, and feel is the future direction of human-computer interaction.

[0044] 3. Machine Learning (ML): This is a multidisciplinary field involving probability theory, statistics, approximation theory, convex analysis, and algorithm complexity theory. It specifically studies how computers can simulate or implement human learning behavior to acquire new knowledge or skills and reorganize existing knowledge structures to continuously improve their performance. Machine learning is the core of artificial intelligence and the fundamental way to endow computers with intelligence; its applications span all areas of artificial intelligence. Machine learning and deep learning typically include techniques such as artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and instructional learning.

[0045] 4. Deep Learning (DL): A branch of machine learning, it's an algorithm that attempts to perform high-level abstraction of data using multiple processing layers containing complex structures or multiple nonlinear transformations. Deep learning learns the inherent patterns and hierarchical representations of training sample data. The information gained during this learning process greatly aids in interpreting data such as text, images, and sound. The ultimate goal of deep learning is to enable machines to possess analytical and learning capabilities like humans, capable of recognizing data such as text, images, and sound. Deep learning is a complex machine learning algorithm, and its performance in speech and image recognition far surpasses previous related technologies.

[0046] 5. Neural Network (NN): A deep learning model in the fields of machine learning and cognitive science that mimics the structure and function of biological neural networks.

[0047] 6. Continuous speech recognition: This refers to the continuous input of speech into a speech recognition system for recognition, and the output of the recognized text.

[0048] The technical solutions provided in this application mainly relate to technologies such as speech processing, machine learning, and deep learning in artificial intelligence, specifically speech recognition technology. These can be illustrated through the following embodiments.

[0049] In continuous speech recognition scenarios, the technology that identifies sentence breaks based on volume and pause time is prone to misjudgment, easily splitting a sentence into two sentences, resulting in semantic incoherence and low speech recognition accuracy.

[0050] To address this technical problem, this application trains a speech recognition model based on a training sample set. Each training sample includes a sample speech signal, the corresponding text, and a ground truth label for that text. The label indicates whether the text corresponding to the sample speech signal is a complete sentence. During training, the acoustic features of the sample speech signals in the training samples are used as input to the speech recognition model. The output is the speech-recognized text of the sample speech signal and its predicted label. Then, based on the speech-recognized text and predicted label of the sample speech signal, as well as the corresponding text and its ground truth label, the parameters of the speech recognition model are adjusted until the training stops, resulting in a trained speech recognition model. Thus, the trained speech recognition model can learn whether the text corresponding to a speech signal is a complete sentence. During continuous speech recognition, it can determine the accurate punctuation points based on the semantics of the recognized text and automatically punctuate, avoiding misjudgments that cause semantic incoherence and improving the accuracy of continuous speech recognition.

[0051] It should be understood that the technical solution of this application can be applied to the following scenarios, but is not limited to:

[0052] For example, Figure 1 This is a schematic diagram illustrating an application scenario of a speech recognition method provided in an embodiment of this application, such as... Figure 1 As shown, this application scenario involves a terminal device 110 and a server 120, and the terminal device 110 can communicate with the server 120.

[0053] In some possible ways, Figure 1 The application scenarios shown can also include: base stations, core network side equipment, etc., in addition, Figure 1 An exemplary terminal device and a server are shown, but in practice, other numbers of terminal devices and servers may be included, and this application does not limit this.

[0054] In some possible ways, Figure 1 The server 120 in this context can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms. This application does not impose any restrictions on this.

[0055] Optionally, in this embodiment, the terminal device 110 can be a device with rich human-computer interaction methods, internet access capabilities, typically running various operating systems, and possessing strong processing capabilities. The terminal device 110 can be a smartphone, tablet computer, smart speaker, or other similar device, but is not limited to these.

[0056] In some possible implementations, such as Figure 1 The terminal device 110 shown can install and run an application with voice recognition capabilities. This application could be a voice assistant application or an application with remote conferencing capabilities. The application can also have functions such as data recording, audio / video playback, and data querying. When this application runs on the terminal device 110, it can interact with the server 120.

[0057] In some implementations, server 120 is used to provide background services for applications with speech recognition capabilities. Optionally, server 120 undertakes the main speech recognition processing work, and terminal device 110 undertakes the secondary speech recognition processing work; or, terminal device 110 undertakes the main speech recognition processing work, and server 120 undertakes the secondary speech recognition processing work; or, server 120 or terminal device 110 may each undertake the speech recognition processing work.

[0058] The speech recognition method provided in this application can be applied to products such as in-vehicle terminals, speech recognition products, voiceprint recognition products, intelligent voice assistants, and smart speakers. It can be applied to the front end of the above products or implemented through the interaction between terminal devices and servers.

[0059] For example, in an application with online conferencing capabilities, if real-time continuous speech recognition is required for the entire conference recording, the speech recognition method provided in this application embodiment can be used to perform continuous speech recognition. The accurate punctuation position can be determined based on the semantics of the recognized text, avoiding misjudgment that causes semantic incoherence and improving the accuracy of continuous speech recognition.

[0060] The technical solution of this application will be described in detail below:

[0061] Figure 2 A flowchart of a speech recognition model training method provided in this application embodiment is shown below. Figure 2 As shown, the method includes the following steps:

[0062] S101. In any iteration, obtain a training sample set. The training sample set includes multiple training samples. Each training sample includes a sample speech signal, the text corresponding to the sample speech signal, and a label for the text. The label is used to indicate whether the text is a complete sentence.

[0063] Specifically, speech recognition models typically require multiple training iterations, with one training sample set used for each iteration. The training sample set includes multiple training samples, each containing a sample speech signal, the corresponding text, and a label for that text. The label indicates whether the text is a complete sentence. For example, if the text in one training sample is "Important content of this meeting," the semantics of the text are incomplete, and the label would be "Incomplete sentence," or "Sentence label." Conversely, if the text in another training sample is "The important content of this meeting includes the following points," the semantics of the text are complete, and the label would be "Complete sentence," or "Sentence end label."

[0064] In one embodiment, obtaining a training sample set may involve obtaining a preset number of training samples from a target training sample set to form the training sample set.

[0065] As an feasible approach, the target training sample set can be obtained in the following way:

[0066] S1. For each speech data in the original speech dataset, obtain a sub-training sample set of the speech data. Each training sample in the sub-training sample set is determined according to the segmentation of the speech data.

[0067] Optionally, in one feasible approach, a subset of training samples of the speech data can be obtained, specifically:

[0068] S11. Obtain the duration range of the speech corresponding to each character in the text of the speech data.

[0069] Specifically, a Gaussian Mixture Model-Hidden Markov Model (GMM-HMMASR) system based on frame-level alignment can be pre-trained. Based on the pre-trained GMM-HMM ASR system, the speech data is forcibly aligned. That is, the speech data is input into the GMM-HMM ASR system, and the output is the text of the speech data and the duration range of each character in the text. For example, if the speech data consists of 15 frames, then 15 characters will be output. Here, the text of the speech data refers to the text corresponding to the speech data.

[0070] S12. Based on the number of text characters m in the voice data, perform text extraction on the voice data to obtain m first texts, and add a label to each of the m first texts.

[0071] Specifically, for example, the text of voice data is a complete sentence. Assuming that the text has a total of m characters, the range of the intercepted characters is [1, n], that is, from the 1st character to the nth character, where 1 ≤ n ≤ m. One piece of text generates m new first texts. For example, the text "The sky is very blue" can be expanded into three first texts: "The sky", "The sky is very", and "The sky is very blue". Add labels to each of the m first texts, specifically according to whether the text is a complete sentence. For example, after adding labels to the above three first texts: "The sky", "The sky is very", and "The sky is very blue", they are respectively: "The sky <mos>"The sky is very..." <mos>"The sky is very blue" <eos>,Label <mos>Indicates that the text is within a sentence, that is, it indicates that "The sky is very" is not a complete sentence. Tag <eos>It indicates that the text is the end of a sentence, that is, it indicates that "the sky is very blue" is a complete sentence.

[0072] S13. For each of the m first texts, the speech data is segmented according to the duration range of the speech corresponding to each character in the first text and the speech data to obtain the sample speech signal corresponding to the first text.

[0073] Specifically, taking "sky is very blue" as an example, step S12 yields three first texts. For each first text, the speech data is segmented based on the duration range of the speech corresponding to each character in the first text and the speech data, resulting in a sample speech signal corresponding to the first text. For example, if the first text is "sky is very blue", the speech data "sky is very blue" is segmented based on the duration range of the speech corresponding to the two characters "sky is very blue", resulting in a sample speech signal corresponding to the first text "sky is very blue".

[0074] S14. Based on m first texts, the label of each first text, and the sample speech signal corresponding to each first text, obtain a sub-training sample set of speech data, wherein the first text, the label of the first text, and the sample speech signal corresponding to the first text constitute a training sample.

[0075] Specifically, taking the above example of "sky is very blue", step S12 yields three first texts and a label for each first text, and step S13 yields the sample speech signals corresponding to the three first texts. Each first text, the label of the first text, and the sample speech signal corresponding to the first text constitute a training sample, that is, the sub-training sample set of the speech data "sky is very blue" includes three training samples.

[0076] S2. The set of sub-training sample sets of each speech data in the original speech dataset is combined to form the target training sample set.

[0077] Specifically, for example, the original speech dataset includes 100 speech data points, and the set of sub-training sample sets of the 100 speech data points constitutes the target training sample set.

[0078] S102. For each training sample in the training sample set, take the acoustic features of the sample speech signal in the training sample as the input of the speech recognition model, and output the speech recognition text and predicted label of the sample speech signal.

[0079] Specifically, in one implementable embodiment, the speech recognition model may include an encoder and a decoder. Optionally, in one embodiment, the encoder and decoder may be a Transformer-based encoder-decoder model; in another embodiment, the encoder may use a Conformer structure and the decoder may be a Transformer structure; in yet another embodiment, both the encoder and decoder may be general neural network structures, which may specifically include recurrent neural network structures. This embodiment does not impose any limitations on these aspects.

[0080] Optionally, when the speech recognition model includes an encoder and a decoder, in step S102, the acoustic features of the sample speech signals in the training samples are used as the input to the speech recognition model, and the output is the speech recognition text and predicted label of the sample speech signals. Specifically, it can be:

[0081] S1021. Using the acoustic features of the sample speech signals in the training samples as the input of the encoder, the output is the hidden layer representation of the sample speech signals.

[0082] Specifically, in the embodiments of this application, the acoustic features of the sample speech signal can be the spectral features, Fbank features (Filter Banks), Mel-Frequency Cepstral Coefficients (MFCC) features, etc., and the acoustic features are all in the form of vector groups.

[0083] In this method, the acoustic features of the sample speech signals in the training samples are used as the input of the encoder. The encoder outputs the hidden layer representation of the sample speech signals. The hidden layer representation of the sample speech signals is also in the form of a vector group, and the dimension of the hidden layer representation of the sample speech signals is the same as the dimension of the acoustic features of the sample speech signals.

[0084] S1022. Using the hidden layer representation of the sample speech signal as the input of the decoder, the output is the speech recognition text and predicted label of the sample speech signal.

[0085] Optionally, in one implementable manner, S1022 may specifically be:

[0086] S10221. Using the hidden layer representation and sentence beginning identifier of the sample speech signal as input to the decoder, output the first recognized character of the sample speech signal.

[0087] S10222: Using the hidden layer representation of the sample speech signal and the first recognized character as the input of the decoder, output the second recognized character of the sample speech signal.

[0088] S10223, using the hidden layer representation of the sample speech signal and the second recognized character as input to the decoder, until the output termination marker, the speech recognition text of the sample speech signal, and the predicted label.

[0089] Figure 3 This is a schematic diagram illustrating the encoder-decoder processing of acoustic features of a sample speech signal, as provided in an embodiment of this application. Figure 3 As shown, the acoustic features of the sample speech signal are used as the input of the encoder, and the hidden layer representation of the sample speech signal is output. Then, the hidden layer representation and sentence beginning identifier of the sample speech signal are used as the input of the decoder, and the first recognized character of the sample speech signal is output. The hidden layer representation and the first recognized character of the sample speech signal are used as the input of the decoder, and the second recognized character of the sample speech signal is output. The hidden layer representation and the second recognized character of the sample speech signal are used as the input of the decoder, until the termination identifier, the speech recognition text of the sample speech signal and the prediction label are output.

[0090] S103. Based on the speech recognition text and predicted labels of the sample speech signals obtained in each iteration, as well as the text and labels corresponding to the sample speech signals, adjust the parameters of the speech recognition model until the training stop condition is met.

[0091] Specifically, the training stop condition can be a preset condition, such as achieving a certain accuracy rate in speech recognition, for example, a speech recognition accuracy rate greater than or equal to a preset threshold. Other conditions can also be used for training stop, and this embodiment does not limit the specific conditions.

[0092] Optionally, in S103, the parameters of the speech recognition model are adjusted based on the speech recognition text and predicted labels of the sample speech signals obtained in each iteration, as well as the text corresponding to the sample speech signals and the labels of the text. Specifically, this may include:

[0093] S1031. Construct a loss function based on the speech recognition text and predicted labels of the sample speech signals, as well as the text corresponding to the sample speech signals and the labels of the text.

[0094] Specifically, the loss function in this embodiment can be the cross-entropy loss function or other loss functions, and this embodiment does not limit it.

[0095] S1032. Adjust the parameters of the speech recognition model using backpropagation based on the loss function.

[0096] S104. The speech recognition model determined by the iterative process that satisfies the stopping training condition is determined as the trained speech recognition model.

[0097] The speech recognition model training method provided in this embodiment trains the speech recognition model based on a training sample set. Each training sample includes a sample speech signal, the corresponding text, and a ground truth label for that text. The label indicates whether the text corresponding to the sample speech signal is a complete sentence. During training, the acoustic features of the sample speech signals in the training samples are used as input to the speech recognition model, which outputs the speech recognition text and its predicted label. Then, based on the speech recognition text and predicted label, as well as the corresponding text and its ground truth label, the parameters of the speech recognition model are adjusted until the training stops, resulting in a trained speech recognition model. Thus, the trained speech recognition model can learn whether the text corresponding to a speech signal is a complete sentence. During continuous speech recognition, it can determine the accurate punctuation positions based on the semantics of the recognized text and automatically punctuate, avoiding misjudgments that cause semantic incoherence and improving the accuracy of continuous speech recognition.

[0098] The following section uses a specific encoder-decoder model structure to explain in detail the training method for speech recognition models.

[0099] For example, Figure 4 This application provides a schematic diagram of a Transformer-based speech recognition model structure. It should be noted that... Figure 4 This is merely an example; the speech recognition model structure involved in the embodiments of this application includes, but is not limited to, those described above. Figure 4 As shown.

[0100] like Figure 4 As shown, the speech recognition model includes an encoding component and a decoding component.

[0101] The encoding component includes at least one encoder. Each encoder includes a linear layer, two normalization layers (laver Norm), a multi-head attention layer, and feedforward neural network layers. Optionally, it may also include two convolutional layers plus activation functions, and optionally, additional neural network modules and two convolutional layers plus activation functions. For example... Figure 4 The sublayer shown, consisting of a normalization layer, a multi-head attention layer, another normalization layer, and a feedforward neural network layer, can be N. e 1, N e It is a positive integer. The number of additional neural network modules can also be M.

[0102] The decoding component includes at least one decoder. Each decoder includes a normalization layer, a multi-head attention layer, a normalization layer, a multi-head attention layer, a normalization layer, a feedforward neural network layer, a normalization layer, a normalization layer, a linear layer, and a loss function. For example, Figure 4 The sublayer shown, consisting of a normalization layer, a multi-head attention layer, another normalization layer, another multi-head attention layer, a normalization layer, a feedforward neural network layer, and another normalization layer, can be N. d 1, N d It is a positive integer.

[0103] Combination Figure 4 The encoder takes the acoustic features of the sample speech signal as input and outputs the hidden layer representation of the sample speech signal. The acoustic features of the sample speech signal can be spectral features, F-bank features (Filter Banks), Mel-Frequency Cepstral Coefficients (MFCC) features, etc., and are all in the form of vector groups. For example, the acoustic features of the sample speech signal are X... n*m Where n represents the dimension of the acoustic features, which is fixed for different inputs, and m represents the number of frames, which varies for different speech lengths. From the encoder input to the encoder output, the size of the encoder input, intermediate calculation results, and encoder output all remain constant at n*m. The encoder processing procedure is as follows:

[0104] S11, Acoustic characteristics of sample speech signals X n*m After passing through convolutional layers, activation functions, and linear layers, we obtain Y. n*m .

[0105] S12, Acquiring and X n*m Position codes of the same size, and the position code is compared with Y. n*m Add them together to get HO n*m and HO n*m As input to the first sub-layer.

[0106] S13, HO n*m After passing through a normalization layer and a multi-head attention layer, J is obtained. n*m .

[0107] S14, J n*m with HO n*m The summation result is then passed through a normalization layer and a feedforward neural network layer to obtain L. n*m .

[0108] S15, L n*m With J n*m Adding them together gives H1 n*m H1 n*m As the input to the second sub-layer, it undergoes the same processing as the sub-layer to obtain H2. n*m After passing through the third sub-layer to the Nth e The processing of each sub-layer ultimately yields HN. e n*m .

[0109] S16, HN e n*m After the normalization layer, Z is obtained. n*m Z n*m This is the output of the encoder, which is the hidden layer representation of the sample speech signal.

[0110] The decoder takes the hidden layer representation and sentence beginning identifier of the sample speech signal as input and outputs the speech recognition text and predicted label of the sample speech signal as output.

[0111] The decoder's processing procedure is as follows:

[0112] S21. First, use the sentence beginning identifier as the input to the decoder.

[0113] Specifically, for example, the sentence beginning is marked with Q1 n*m .

[0114] S22, Acquisition and Q1 n*m Position codes of the same size, and the position code is compared with Q1. n*m Add them together to get HP n*m and HP n*m As input to the first sub-layer.

[0115] S23, HP n*m After passing through a normalization layer and a multi-head attention layer, S is obtained. n*m .

[0116] S24, S n*m with HP n*m The sum is then normalized to obtain A. n*m .

[0117] S25, A n*m Hidden layer representation Z of sample speech signal n*m As input to the multi-head attention layer, the resulting output of the multi-head attention layer is the same as A. n*m Adding them together gives B1 n*m .

[0118] S26, B1 n*m As the input to the second sub-layer, it undergoes the same processing as the sub-layer to obtain B2. n*m After passing through the third sub-layer to the Nth d The processing of each sub-layer ultimately yields BN. d n*m .

[0119] S27, BN d n*m After normalization layer, linear layer and loss function, E1 is obtained. n*m E1 n*m This is the output of the decoder, which is the speech recognition text and predicted label of the sample speech signal of the training samples.

[0120] For example, the speech recognition model in this embodiment includes an encoder and a decoder, both of which are recurrent neural networks. In this embodiment, the encoder and decoder need to support variable-length inputs. Common recurrent neural networks include Long Short-Term Memory (LSTM) units or Gate Recurrent Units (GRUs). The processing procedure of recurrent neural networks will not be described in detail here.

[0121] Figure 5 A flowchart of a speech recognition method provided in an embodiment of this application is shown below. Figure 5 As shown, the method includes the following steps:

[0122] S301. Acquire the continuous speech signal to be recognized.

[0123] S302. Input the first speech segment of the continuous speech signal to be recognized into the speech recognition model to obtain the first recognized text of the first speech segment and the label of the first recognized text. The label is used to indicate whether the first recognized text is a complete sentence.

[0124] Specifically, the speech recognition model is based on Figure 2 The method shown is used for training.

[0125] S303. If it is determined that the label of the first recognized text indicates that the first recognized text is not a complete sentence, then the target speech segment after concatenating the second speech segment with the first speech segment is re-inputted into the speech recognition model to obtain the target recognized text and the label of the target recognized text of the target speech segment. The second speech segment is the next speech segment of the first speech segment.

[0126] S304. If it is determined that the label of the target recognition text indicates that the target recognition text is a complete sentence, output the target recognition text.

[0127] As one possible implementation, the method in this embodiment may further include:

[0128] S305. If the label of the first identified text indicates that the first identified text is a complete sentence, then output the first identified text.

[0129] Furthermore, in one implementable embodiment, the method after S304 may further include:

[0130] S306. Discard the first and second speech segments, and input the third speech segment into the speech recognition model for speech recognition.

[0131] In one implementable manner, the speech recognition model includes an encoder and a decoder. Accordingly, in S302, a first speech segment of the continuous speech signal to be recognized is input into the speech recognition model to obtain a first recognized text of the first speech segment and a label for the first recognized text. Specifically, this can be:

[0132] S3021. Using the acoustic features of the first speech segment as the input of the encoder, output the hidden layer representation of the first speech segment.

[0133] S3022, Using the hidden layer representation of the first speech segment as the input of the decoder, output the first recognized text and the label of the first recognized text.

[0134] Specifically, the hidden layer representation of the first speech segment is used as the input to the decoder, and the output is the first recognized text and the label of the first recognized text, which can be:

[0135] S30221. Using the hidden layer representation and sentence beginning identifier of the first speech segment as input to the decoder, output the first recognized character of the first speech segment.

[0136] S30222: Using the hidden layer representation of the first speech segment and the first recognized character as the input to the decoder, output the second recognized character of the first speech segment.

[0137] S30223, Using the hidden layer representation of the first speech segment and the second recognized character as input to the decoder, until the output termination identifier, the speech recognition text of the first speech segment and the label of the speech recognition text of the first speech segment.

[0138] The following is combined Figure 6 The specific process of the above continuous speech recognition is explained in detail. Figure 6 This is a schematic diagram of a speech recognition method provided in an embodiment of this application, as shown below. Figure 6 As shown, the speech recognition model includes an encoder and a decoder. Each speech segment of the continuous speech signal to be recognized is sequentially input into the encoder. First, the first speech segment of the continuous speech signal to be recognized is input into the speech recognition model to obtain the first recognized text and its label. This label indicates whether the first recognized text is a complete sentence. If the label of the first recognized text indicates that it is not a complete sentence, meaning that the text corresponding to the first speech segment has been transcribed, but semantically it is not a complete sentence, then the target speech segment, concatenated with the second speech segment and the first speech segment, is re-input into the speech recognition model for a new round of speech recognition, obtaining the target recognized text and its label. The second speech segment is the next speech segment after the first speech segment. Next, if the label of the target recognized text indicates that it is a complete sentence, the target recognized text is output. Otherwise, the third speech segment, the second speech segment, and the first speech segment are concatenated to obtain a concatenated speech segment, and a new round of speech recognition is performed. If the label of the first recognized text indicates that it is a complete sentence, then the first recognized text is output. After outputting the recognized text, the recognized speech segments are discarded, and subsequent speech segments are recognized. It should be noted that when concatenating the second speech segment with the first speech segment, they are concatenated in the order of input.

[0139] The speech recognition method provided in this embodiment acquires a continuous speech signal to be recognized, inputs a first speech segment of the continuous speech signal to a speech recognition model, and obtains a first recognized text and a label for the first recognized text. If it is determined that the label of the first recognized text indicates that the first recognized text is not a complete sentence, then the target speech segment, which is a concatenation of the second speech segment and the first speech segment, is re-inputted into the speech recognition model to obtain the target recognized text and a label for the target recognized text. If it is determined that the label of the target recognized text indicates that the target recognized text is a complete sentence, the target recognized text is output. Since the speech recognition model only outputs the first recognized text when it determines that the first recognized text of the first speech segment is a complete sentence, and re-inputs the target speech segment, which is a concatenation of the second speech segment and the first speech segment, into the speech recognition model for recognition when it is not a complete sentence, until a complete sentence is recognized, the accurate punctuation position can be determined based on the semantics of the recognized text during continuous speech recognition, and punctuation can be automatically performed, avoiding misjudgment that causes semantic incoherence and improving the accuracy of continuous speech recognition.

[0140] Moreover, the speech recognition method provided in this embodiment does not require an additional voice activity detection (VAD) system for initial speech segmentation, reducing computing resources and improving system real-time performance.

[0141] Figure 7 This is a schematic diagram of the structure of a speech recognition model training device provided in an embodiment of this application, as shown below. Figure 7 As shown, the device may include an acquisition module 11 and a model training module 12.

[0142] The acquisition module 11 is used to acquire a training sample set during any iteration. The training sample set includes multiple training samples. Each training sample includes a sample speech signal, the text corresponding to the sample speech signal, and a label for the text. The label is used to indicate whether the text is a complete sentence.

[0143] The model training module 12 is used to: for each training sample in the training sample set, take the acoustic features of the sample speech signal in the training sample as the input of the speech recognition model, and output the speech recognition text and predicted label of the sample speech signal.

[0144] Based on the speech recognition text and predicted labels of the sample speech signals obtained in each iteration, as well as the text corresponding to the sample speech signals and the labels of the text, the parameters of the speech recognition model are adjusted until the training stop condition is met.

[0145] The speech recognition model determined by the iterative process that satisfies the stopping training condition is defined as the trained speech recognition model.

[0146] In one embodiment, the model training module 12 is used to: take the acoustic features of the sample speech signals in the training samples as the input of the encoder, and output the hidden layer representation of the sample speech signals.

[0147] The hidden layer representation of the sample speech signal is used as the input to the decoder, and the output is the speech recognition text and predicted label of the sample speech signal.

[0148] In one embodiment, the model training module 12 is specifically used to: take the hidden layer representation and sentence beginning identifier of the sample speech signal as input to the decoder, and output the first recognized character of the sample speech signal;

[0149] The decoder takes the hidden layer representation of the sample speech signal and the first recognized character as input and outputs the second recognized character of the sample speech signal.

[0150] The hidden layer representation of the sample speech signal and the second recognized character are used as inputs to the decoder until the output termination marker, the speech recognition text of the sample speech signal, and the predicted label are obtained.

[0151] In one embodiment, the model training module 12 is used to: construct a loss function based on the speech recognition text and predicted labels of the sample speech signal, as well as the text corresponding to the sample speech signal and the labels of the text;

[0152] The parameters of the speech recognition model are adjusted using backpropagation based on the loss function.

[0153] In one embodiment, the acquisition module 11 is used to:

[0154] A training sample set is formed by obtaining a predetermined number of training samples from the target training sample set.

[0155] The target training sample set is obtained in the following way:

[0156] For each speech data in the original speech dataset, a sub-training sample set of speech data is obtained. Each training sample in the sub-training sample set is determined based on the segmentation of the speech data.

[0157] The target training sample set is formed by combining the subsets of training samples for each speech data in the original speech dataset.

[0158] In one embodiment, the acquisition module 11 is specifically used for:

[0159] Obtain the duration range of each character in the text corresponding to the speech data;

[0160] Based on the number of text characters m in the voice data, the text of the voice data is truncated to obtain m first texts, and a label is added to each of the m first texts;

[0161] For each of the m first texts, the speech data is segmented according to the duration range of the speech corresponding to each character in the first text and the speech data to obtain the sample speech signal corresponding to the first text;

[0162] Based on m first texts, the label of each first text, and the sample speech signal corresponding to each first text, a sub-training sample set of speech data is obtained, wherein the first text, the label of the first text, and the sample speech signal corresponding to the first text constitute a training sample.

[0163] Figure 8 This is a schematic diagram of the structure of a voice recognition device provided in an embodiment of this application, as shown below. Figure 8 As shown, the device may include an acquisition module 21 and a processing module 22.

[0164] The acquisition module 21 is used to acquire the continuous speech signal to be recognized.

[0165] Processing module 22 is used to input the first speech segment of the continuous speech signal to be recognized into the speech recognition model, and obtain the first recognized text of the first speech segment and the label of the first recognized text. The label is used to indicate whether the first recognized text is a complete sentence. The speech recognition model then... Figure 2 The method shown is used for training.

[0166] The processing module 22 is also used to: if it is determined that the label of the first recognized text indicates that the first recognized text is not a complete sentence, then re-input the target speech segment after concatenating the second speech segment with the first speech segment into the speech recognition model to obtain the target recognized text and the label of the target recognized text of the target speech segment, wherein the second speech segment is the next speech segment of the first speech segment;

[0167] If the label of the target-recognized text indicates that the target-recognized text is a complete sentence, output the target-recognized text.

[0168] In one embodiment, the processing module 22 is further configured to: output the first identified text if it is determined that the label of the first identified text indicates that the first identified text is a complete sentence.

[0169] In one embodiment, the processing module 22 is further configured to: discard the first speech segment and the second speech segment, and input the third speech segment into the speech recognition model for speech recognition.

[0170] In one embodiment, the speech recognition model includes an encoder and a decoder, and the processing module 22 is used for:

[0171] The encoder takes the acoustic features of the first speech segment as input and outputs the hidden layer representation of the first speech segment. The decoder takes the hidden layer representation of the first speech segment as input and outputs the first recognized text and the label of the first recognized text.

[0172] It should be understood that the device embodiments and method embodiments can correspond to each other, and similar descriptions can be referred to the method embodiments. To avoid repetition, further details will not be provided here. Specifically, Figure 7 The speech recognition model training device shown and Figure 8 The speech recognition device shown can execute the method embodiment corresponding to the computer device, and the foregoing and other operations and / or functions of each module in the device are respectively for implementing the method embodiment corresponding to the computer device, which will not be described in detail here for the sake of brevity.

[0173] The speech recognition model training device and speech recognition device of this application embodiments have been described above from the perspective of functional modules, with reference to the accompanying drawings. It should be understood that these functional modules can be implemented in hardware, in software instructions, or in a combination of hardware and software modules. Specifically, the steps of the method embodiments in this application can be completed by the integrated logic circuits in the processor's hardware and / or by software instructions. The steps of the methods disclosed in this application embodiments can be directly manifested as being executed by a hardware decoding processor, or by a combination of hardware and software modules in the decoding processor. Optionally, the software module can be located in a mature storage medium in the art, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, etc. This storage medium is located in memory, and the processor reads information from the memory and, in conjunction with its hardware, completes the steps in the above method embodiments.

[0174] Figure 9 This is a schematic block diagram of the computer device 300 provided in the embodiments of this application.

[0175] like Figure 9 As shown, the computer device 300 may include:

[0176] The system includes a memory 310 and a processor 320. The memory 310 stores computer programs and transfers the program code to the processor 320. In other words, the processor 320 can retrieve and run the computer program from the memory 310 to implement the methods described in the embodiments of this application.

[0177] For example, the processor 320 can be used to execute the above-described method embodiments according to instructions in the computer program.

[0178] In some embodiments of this application, the processor 320 may include, but is not limited to:

[0179] General-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc.

[0180] In some embodiments of this application, the memory 310 includes, but is not limited to:

[0181] Volatile memory and / or non-volatile memory. Non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. Volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of RAM are available, such as Static RAM (SRAM), Dynamic RAM (DRAM), Synchronous DRAM (SDRAM), Double Data Rate SDRAM (DDR SDRAM), Enhanced Synchronous DRAM (ESDRAM), Synchronous Link DRAM (SLDRAM), and Direct Rambus RAM (DR RAM).

[0182] In some embodiments of this application, the computer program may be divided into one or more modules, which are stored in the memory 310 and executed by the processor 320 to perform the method provided in this application. The one or more modules may be a series of computer program instruction segments capable of performing specific functions, which describe the execution process of the computer program in the computer device.

[0183] like Figure 9 As shown, the computer device may further include:

[0184] Transceiver 330, which can be connected to processor 320 or memory 310.

[0185] The processor 320 can control the transceiver 330 to communicate with other devices; specifically, it can send information or data to other devices or receive information or data sent by other devices. The transceiver 330 may include a transmitter and a receiver. The transceiver 330 may further include antennas, and the number of antennas may be one or more.

[0186] It should be understood that the various components in the computer device are connected through a bus system, which includes, in addition to the data bus, a power bus, a control bus, and a status signal bus.

[0187] This application also provides a computer storage medium storing a computer program thereon, which, when executed by a computer, enables the computer to perform the methods of the above-described method embodiments. Alternatively, embodiments of this application also provide a computer program product containing instructions that, when executed by a computer, cause the computer to perform the methods of the above-described method embodiments.

[0188] When implemented using software, it can be implemented entirely or partially as a computer program product. This computer program product includes one or more computer instructions. When these computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., digital video disc (DVD)), or a semiconductor medium (e.g., solid-state disk (SSD)).

[0189] Those skilled in the art will recognize that the modules and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0190] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple modules or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or modules may be electrical, mechanical, or other forms.

[0191] The modules described as separate components may or may not be physically separate. The components shown as modules may or may not be physical modules; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. For example, the functional modules in the various embodiments of this application may be integrated into one processing module, or each module may exist physically separately, or two or more modules may be integrated into one module.

[0192] The above are merely specific embodiments of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.< / eos> < / mos> < / eos> < / mos> < / mos>

Claims

1. A method for training a speech recognition model, characterized in that, include: During any iteration, a training sample set is obtained, which includes multiple training samples. Each training sample includes a sample speech signal, the text corresponding to the sample speech signal, and a label for the text. The label is used to indicate whether the text is a complete sentence. For each training sample in the training sample set, the acoustic features of the sample speech signal in the training sample are used as the input of the speech recognition model, and the speech recognition text and predicted label of the sample speech signal are output. Based on the speech recognition text and predicted label of the sample speech signal obtained in each iteration, as well as the text corresponding to the sample speech signal and the label of the text, the parameters of the speech recognition model are adjusted until the training stop condition is met. The speech recognition model determined by the iterative process that satisfies the aforementioned stopping training condition is defined as the trained speech recognition model.

2. The method according to claim 1, characterized in that, The speech recognition model includes an encoder and a decoder. The input of the speech recognition model is the acoustic features of the sample speech signals in the training samples, and the output is the speech recognition text and predicted labels of the sample speech signals, including: The encoder takes the acoustic features of the sample speech signals in the training samples as input and outputs the hidden layer representation of the sample speech signals. The hidden layer representation of the sample speech signal is used as the input to the decoder, and the speech recognition text of the sample speech signal and the predicted label are output.

3. The method according to claim 2, characterized in that, The step of using the hidden layer representation of the sample speech signal as input to the decoder and outputting the speech recognition text and the predicted label of the sample speech signal includes: Using the hidden layer representation and sentence beginning identifier of the sample speech signal as input to the decoder, the first recognized character of the sample speech signal is output. Using the hidden layer representation of the sample speech signal and the first recognized character as the input to the decoder, the second recognized character of the sample speech signal is output. The decoder takes the hidden layer representation of the sample speech signal and the second recognized character as inputs until it outputs a termination identifier, the speech recognition text of the sample speech signal, and the predicted label.

4. The method according to claim 1, characterized in that, The step of adjusting the parameters of the speech recognition model based on the speech recognition text and predicted labels of the sample speech signals obtained in each iteration, as well as the text corresponding to the sample speech signals and the labels of the text, includes: A loss function is constructed based on the speech recognition text and predicted labels of the sample speech signals, as well as the text corresponding to the sample speech signals and the labels of the text. The parameters of the speech recognition model are adjusted by backpropagation based on the loss function.

5. The method according to claim 1, characterized in that, The acquisition of the training sample set includes: The training sample set is composed of a predetermined number of training samples obtained from the target training sample set. The target training sample set is obtained in the following way: For each piece of speech data in the original speech dataset, a sub-training sample set of the speech data is obtained, and each training sample in the sub-training sample set is determined according to the segmentation of the speech data. The target training sample set is formed by combining the subsets of training samples for each speech data in the original speech dataset.

6. The method according to claim 5, characterized in that, The sub-training sample set for obtaining the speech data includes: Obtain the duration range of the speech corresponding to each character in the text of the speech data; Based on the number of text characters m in the voice data, the text of the voice data is truncated to obtain m first texts, and a tag is added to each of the m first texts; For each of the m first texts, the speech data is segmented according to the duration range of the speech corresponding to each character in the first text and the speech data to obtain the sample speech signal corresponding to the first text; Based on the m first texts, the labels of each first text, and the sample speech signals corresponding to each first text, a sub-training sample set of the speech data is obtained, wherein the first text, the labels of the first text, and the sample speech signals corresponding to the first text constitute a training sample.

7. A speech recognition method, characterized in that, include: Acquire the continuous speech signal to be recognized; The first speech segment of the continuous speech signal to be recognized is input into the speech recognition model to obtain the first recognized text of the first speech segment and the label of the first recognized text. The label is used to indicate whether the recognized text is a complete sentence. The speech recognition model is trained according to the method of any one of claims 1-6. If it is determined that the label of the first recognized text indicates that the first recognized text is not a complete sentence, then the target speech segment after concatenating the second speech segment with the first speech segment is re-inputted into the speech recognition model to obtain the target recognized text of the target speech segment and the label of the target recognized text. The second speech segment is the next speech segment of the first speech segment. If it is determined that the label of the target recognition text indicates that the target recognition text is a complete sentence, the target recognition text is output.

8. The method according to claim 7, characterized in that, The method further includes: If it is determined that the label of the first identified text indicates that the first identified text is a complete sentence, then the first identified text is output.

9. The method according to claim 7, characterized in that, After outputting the target-recognized text, the method further includes: The first and second speech segments are discarded, and the third speech segment is input into the speech recognition model for speech recognition.

10. The method according to claim 7, characterized in that, The speech recognition model includes an encoder and a decoder. The step of inputting a first speech segment of the continuous speech signal to be recognized into the speech recognition model to obtain a first recognized text of the first speech segment and a label for the first recognized text includes: Using the acoustic features of the first speech segment as the input of the encoder, the hidden layer representation of the first speech segment is output. Using the hidden layer representation of the first speech segment as input to the decoder, the decoder outputs the first recognized text and the label of the first recognized text.

11. A speech recognition model training device, characterized in that, include: The acquisition module is used to acquire a training sample set during any iteration. The training sample set includes multiple training samples. Each training sample includes a sample speech signal, the text corresponding to the sample speech signal, and a label for the text. The label is used to indicate whether the text is a complete sentence. The model training module is used to: for each training sample in the training sample set, take the acoustic features of the sample speech signal in the training sample as the input of the speech recognition model, and output the speech recognition text and predicted label of the sample speech signal; Based on the speech recognition text and predicted label of the sample speech signal obtained in each iteration, as well as the text corresponding to the sample speech signal and the label of the text, the parameters of the speech recognition model are adjusted until the training stop condition is met. The speech recognition model determined by the iterative process that satisfies the aforementioned stopping training condition is defined as the trained speech recognition model.

12. A voice recognition device, characterized in that, include: The acquisition module is used to acquire the continuous speech signal to be recognized; The processing module is configured to input a first speech segment of the continuous speech signal to be recognized into a speech recognition model to obtain a first recognized text of the first speech segment and a label of the first recognized text. The label is used to indicate whether the recognized text is a complete sentence. The speech recognition model is trained according to the method described in any one of claims 1-6. The processing module is further configured to: if it is determined that the label of the first recognized text indicates that the first recognized text is not a complete sentence, then re-input the target speech segment after concatenating the second speech segment with the first speech segment into the speech recognition model to obtain the target recognized text of the target speech segment and the label of the target recognized text, wherein the second speech segment is the next speech segment of the first speech segment; If it is determined that the label of the target recognition text indicates that the target recognition text is a complete sentence, the target recognition text is output.

13. A computer device, characterized in that, include: A processor and a memory, the memory for storing a computer program, the processor for calling and running the computer program stored in the memory to perform the method of any one of claims 1 to 6 or 7-10.

14. A computer-readable storage medium, characterized in that, Used to store a computer program that causes a computer to perform the method as described in any one of claims 1 to 6 or 7-10.

15. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6 or 7-10.

Images