Streaming speech recognition methods and devices, electronic devices, and storage media

Abstract

This application relates to the field of speech recognition technology, specifically to a streaming speech recognition method and apparatus, electronic device, and storage medium. The method includes: acquiring a speech block to be recognized; performing object recognition processing on the speech block based on a connectionist temporal classification model to obtain an object recognition result, and determining the number of objects in the speech block based on the object recognition result; determining the recognition count based on the number of objects in the speech block, and performing speech recognition processing on the speech block for the specified number of recognition counts based on an attention model to obtain a speech recognition result corresponding to the speech block. The method uses a connectionist temporal classification model to predict the number of recognition objects contained in the current speech block as the speech recognition count for the attention model. By performing the corresponding number of recognition counts on the speech block using the attention model, speech can be recognized more accurately and efficiently and converted into corresponding text and other information.

Description

Technical Field

[0001] This application relates to the field of speech recognition technology, specifically to a streaming speech recognition method and apparatus, electronic device, and storage medium. Background Technology

[0002] Speech recognition, also known as Automatic Speech Recognition (ASR), aims to enable machines to convert received speech signals into text for output through recognition and understanding. Speech recognition is an important branch of modern artificial intelligence development.

[0003] Traditional speech recognition technology relies on Hidden Markov Models (HMMs), Gaussian Mixture Models (GMMs), and Deep Neural Networks-HMMs to build acoustic models. This method, which uses networks composed of language, acoustic, and dictionary models, requires training each model separately before fusing them together using decoders such as Weighted Finite State Transition Machines (WFSTs). Training or designing each model requires specialized knowledge and technical expertise, and the process of training and recognizing each model is cumbersome, inefficient, inaccurate, and involves significant latency. Therefore, it is necessary to provide a new speech recognition technology to address these problems in existing technologies. Summary of the Invention

[0004] The purpose of this application is to provide a streaming speech recognition method and apparatus.

[0005] According to one aspect of this application, a streaming speech recognition method is provided, comprising the following steps:

[0006] Obtain the speech block to be recognized;

[0007] The speech block is processed for object recognition based on the connectionist temporal classification model to obtain object recognition results, and the number of objects in the speech block is determined based on the object recognition results.

[0008] The number of recognition attempts is determined based on the number of objects in the speech block, and the speech block is processed according to the number of recognition attempts based on the attention model to obtain the speech recognition result corresponding to the speech block.

[0009] In an exemplary embodiment of this application, the step of performing object recognition processing on the speech block based on the connectionist temporal classification model to obtain an object recognition result, and determining the number of objects in the speech block based on the object recognition result, includes:

[0010] Based on the connectionist temporal classification model, the speech block is subjected to object recognition processing to obtain at least one set of object recognition processing results and corresponding accuracy.

[0011] The set of object recognition processing results with the highest accuracy is determined as the object recognition processing result;

[0012] The number of objects in the speech block is determined based on the object recognition results.

[0013] In an exemplary embodiment of this application, the step of performing object recognition processing on the speech block based on the connectionist temporal classification model to obtain the object recognition result includes:

[0014] The speech block is encoded to obtain the feature sequence of the speech block;

[0015] The object recognition result is obtained by performing object recognition processing on the feature sequence of the speech block based on the connectionist temporal classification model.

[0016] In an exemplary embodiment of this application, the step of determining the number of recognition attempts based on the number of objects in the speech block, and performing speech recognition processing on the speech block based on the number of recognition attempts using an attention model to obtain a speech recognition result corresponding to the speech block includes:

[0017] The number of recognition attempts is determined based on the number of objects in the speech block, and the speech block is processed according to the number of recognition attempts based on the attention model to obtain at least one set of speech recognition processing results and corresponding accuracy.

[0018] The results of the speech recognition processing with the highest accuracy are determined as the speech recognition results corresponding to the speech block.

[0019] In an exemplary embodiment of this application, obtaining the speech block to be recognized includes:

[0020] If an unrecognized object is detected in the previous speech block to be recognized, object recognition processing is performed on the previous speech block to be recognized.

[0021] In an exemplary embodiment of this application, the streaming speech recognition method further includes:

[0022] If the accuracy of the results of at least one set of speech recognition processing is less than the threshold, the speech block is reprocessed for object recognition or speech recognition.

[0023] In an exemplary embodiment of this application, obtaining the speech block to be recognized includes:

[0024] The speech block is extracted from the speech signal to be recognized according to a specified time range or a specified speech block size.

[0025] According to another aspect of this application, a streaming speech recognition device is provided, comprising:

[0026] The acquisition module is used to acquire the speech blocks to be recognized;

[0027] The first recognition module is used to perform object recognition processing on the speech block based on the connectionist temporal classification model to obtain the object recognition result, and to determine the number of objects in the speech block based on the object recognition result;

[0028] The second recognition module is used to determine the number of recognition attempts based on the number of objects in the speech block, and to perform speech recognition processing on the speech block based on the number of recognition attempts using an attention model to obtain a speech recognition result corresponding to the speech block.

[0029] According to another aspect of this application, an electronic device is provided, comprising:

[0030] At least one processor; and,

[0031] A memory communicatively connected to the at least one processor; wherein,

[0032] The memory stores instructions that can be executed by the at least one processor, which, when executed by the at least one processor, enables the at least one processor to perform the above-described streaming speech recognition method.

[0033] According to another aspect of this application, a computer-readable storage medium is provided, which stores a computer program that, when executed by a processor, implements the above-described streaming speech recognition method.

[0034] This application provides a streaming speech recognition method, device, electronic device, and storage medium. It uses a connectionist temporal classification model to predict the number of recognition objects contained in the current speech block as the speech recognition count of the attention model. The attention model performs the corresponding number of recognitions on the speech block, thereby obtaining the final speech recognition result more accurately and efficiently. Attached Figure Description

[0035] Figure 1 This is a flowchart illustrating a speech recognition method in related technologies;

[0036] Figure 2 This is a flowchart illustrating another speech recognition method in related technologies;

[0037] Figure 3 This is a flowchart illustrating a streaming speech recognition method according to an embodiment of this application;

[0038] Figure 4 This is a schematic diagram of the structure of a streaming speech recognition device in an embodiment of this application. Detailed Implementation

[0039] To make the objectives, features, and advantages of this application more apparent and understandable, the technical solutions of the embodiments and examples of this application will be clearly and completely described below in conjunction with the accompanying drawings. However, the exemplary embodiments and examples can be implemented in various forms and should not be construed as limited to the examples set forth herein; rather, these embodiments and examples are provided to make this application more comprehensive and complete, and to fully convey the concept of the exemplary embodiments and examples to those skilled in the art. The features, structures, or characteristics described in this application can be combined in any suitable manner in one or more embodiments and examples. In the following description, numerous specific details are provided to give a full understanding of the embodiments and examples of this application. However, those skilled in the art will recognize that the technical solutions of this application can be practiced by omitting one or more of the specific details, or other methods, components, apparatus, steps, etc. can be used. In other cases, well-known technical solutions are not shown or described in detail to avoid obscuring various aspects of this application.

[0040] Furthermore, the accompanying drawings are merely illustrative of this application and are not necessarily drawn to scale. The same reference numerals in the drawings denote the same or similar parts, and therefore, repeated descriptions of them will be omitted. Although the steps of the method in this application are described in a specific order in the drawings, this does not require or imply that these steps must be performed in that specific order, or that all the steps shown must be performed to achieve the desired result. The flowcharts shown in the drawings are merely illustrative and do not necessarily include all steps. For example, some steps may be decomposed, while others may be combined or partially combined; therefore, the actual order of execution may change depending on the circumstances. Some block diagrams shown in the drawings are functional entities and do not necessarily correspond to physically or logically independent entities. These functional entities may be implemented in software, in one or more hardware modules or integrated circuits, or in different network and / or processor devices and / or microcontroller devices.

[0041] Speech recognition, also known as Automatic Speech Recognition (ASR), aims to enable machines to recognize and understand received speech signals and convert them into text for output. Speech recognition is an important branch of modern artificial intelligence. Traditional speech recognition technology is based on Hidden Markov Models (HMMs), Gaussian Mixture Models (GMMs), and Deep Neural Network-Hidden Markov Models (DNLMs) to build acoustic models. This method, which uses a network composed of language models, acoustic models, and dictionary models, requires separate training for each model. Then, decoders such as WFST are used to fuse multiple models together. The training or design of each model requires specialized knowledge and technical expertise, making the training and recognition process quite cumbersome. With the development and application of related technologies, traditional speech recognition technology, due to its low efficiency, low accuracy, and significant latency, has gradually been replaced by more advanced end-to-end streaming ASR. End-to-end (E2E) systems require only the speech features of the input end and the text information of the output end to directly convert input speech into output text. Streaming speech recognition, unlike non-streaming speech recognition which returns results only after processing a complete sentence of audio, can return recognition results in real time while processing the audio stream. Streaming speech recognition is better suited for scenarios requiring real-time recognition results, such as live streaming captions, real-time conference recording, voice input, and voice wake-up.

[0042] One related technology is a streaming speech recognition based on a connectionist temporal classification model, such as... Figure 1 As shown, this recognition method involves encoding the streamed speech to extract its feature information, then inputting it into a connectionist temporal classification model for prefix bundle search recognition. After recognition, a pre-selected recognition result group is output. Next, the pre-selected recognition result group is decoded and sorted to obtain another pre-selected recognition result group. Finally, the two recognition result groups are weighted and summed according to their scores, and the pre-selected result with the highest score is taken as the final recognition result. Another streaming speech recognition method based on a connectionist temporal classification model is as follows... Figure 2 As shown, in the Figure 1The same method is used to weight and sum the two sets of recognition results to obtain the current recognition result. This result is then input into the connectionist temporal classification model to optimize it, thereby achieving more accurate recognition results in speech recognition. Both of the above schemes are based on the connectionist temporal classification model for speech recognition. However, because the connectionist temporal classification model itself has strong independence, it recognizes based on the pronunciation features of each word and cannot consider the relationships between words, resulting in lower accuracy. Furthermore, both methods require complex weighted summation and decoding sorting operations on the recognition results during the recognition process, leading to significant latency and reduced synchronization of the output recognition results.

[0043] In view of the aforementioned problems in related technologies, this application specifically proposes a streaming speech recognition method, apparatus, electronic device, and storage medium. This method and apparatus are mainly applied to end-to-end streaming speech recognition scenarios, where end-to-end refers to the process from the input end to the output end of speech recognition. Common input features for end-to-end speech recognition are fbank (Filter Bank) features, and the processing involves pre-emphasis, framing, windowing, short-time Fourier transform (STFT), Mel (Mel) filtering, and mean removal of the speech signal. The output end can be recognized objects such as letters, subwords, and words. The method includes: acquiring a speech block to be recognized; performing object recognition processing on the speech block based on a connectionist temporal classification model to obtain object recognition results, and determining the number of objects in the speech block based on the object recognition results; determining the number of recognition attempts based on the number of objects in the speech block, and performing speech recognition processing on the speech block for the number of recognition attempts based on an attention model to obtain a speech recognition result corresponding to the speech block. The method uses a connectionist temporal classification model to predict the number of recognition objects contained in the current speech block as the number of speech recognition attempts of the attention model. The attention model then performs the corresponding number of recognition attempts on the speech block, thereby obtaining the final speech recognition result more accurately and efficiently.

[0044] An exemplary embodiment of this application provides a streaming speech recognition method. Figure 3A flowchart illustrating a streaming speech recognition method according to an exemplary embodiment of this application is shown. The streaming speech recognition method can be implemented by a terminal device, meaning the terminal device can execute the steps of the following method. In this case, the streaming speech recognition device can be included within the terminal device. The terminal device can include, but is not limited to, mobile terminals such as mobile phones, laptops, digital radio receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), and in-vehicle terminals (e.g., in-vehicle navigation terminals), as well as fixed terminals such as digital TVs and desktop computers. Figure 3 As shown, the streaming speech recognition method includes:

[0045] Step S31: Obtain the speech block to be recognized;

[0046] In one exemplary embodiment, audio information can be collected using a device such as a microphone, and the audio information collected by the microphone can be obtained through a corresponding data transmission interface for subsequent recognition. During the process of a user inputting voice into the audio acquisition device, the audio acquisition device can perform voice activity detection on the continuous voice signal. The voice activity detection can determine the nature of the detected audio data using preset detection methods. Taking energy detection as an example, when the energy of an audio segment is greater than a preset threshold, the audio segment is determined to be speech; when the energy of an audio segment is less than or equal to the preset threshold, the audio segment is determined to be noise.

[0047] In one exemplary embodiment, speech blocks can be extracted from the speech signal to be recognized based on a specified time range or a specified speech block size. The continuous speech signal is divided into multiple speech blocks according to the detected endpoint time. For example, during continuous speech input by a user to an audio acquisition device, the audio acquisition device processes the continuously input speech into speech blocks of a specified size according to the length of a pre-set data frame. For example, the input speech can be divided into multiple speech blocks in 10ms increments, or in 10kb increments. As another example, each sentence of the speech input can be determined based on the detected endpoint time, and each sentence can be recognized as a speech block. For example, if speech detection reveals that there is a speech signal between time A and time B, no speech signal between time B and time C, and a speech signal between time C and time D, then the speech signal between time A and time B can be used as the first speech block, and the speech signal between time C and time D as the second speech block.

[0048] Step S33: Perform object recognition processing on the speech block based on the connectionist temporal classification model to obtain object recognition results, and determine the number of objects in the speech block based on the object recognition results;

[0049] Connectionist temporal classification (CTC) is an algorithm commonly used in speech recognition, text recognition, and other fields. It can solve problems where the input and output sequences have different lengths and cannot be aligned. CTC has two key characteristics: First, an additional output node is added to the network's output to represent a "blank" symbol. In speech recognition, each output node of the neural network represents an acoustic modeling factor. Depending on the modeling granularity, this factor can be a monophoneme factor or a triphoneme factor, where the network's output at each time step represents the posterior probability of each phoneme factor at that time. The "blank" symbol represents the state when the network output is uncertain. That is, when the input is an unidentifiable feature such as noise, or when the input is a borderline state between two different phonemes, the network can output a "blank" symbol instead of outputting a definite phoneme. Second, the connectionist temporal classification model training method optimizes the entire sentence input to the network, aiming to maximize the output probability of the correct text sequence of the whole sentence, rather than maximizing the output probability of each frame like cross-entropy. The connectionist temporal classification model uses a forward and backward algorithm to find all paths that may map to the correct text sequence in the network's output matrix, calculates their probabilities and network errors, and then updates the neural network parameters through error backpropagation and gradient descent algorithms.

[0050] In an exemplary embodiment, the step of performing object recognition processing on the speech block based on the connectionist temporal classification model to obtain object recognition results, and determining the number of objects in the speech block based on the object recognition results, may include: performing object recognition processing on the speech block based on the connectionist temporal classification model to obtain at least one set of object recognition processing results and corresponding accuracy; determining the set of object recognition processing results with the highest accuracy as the object recognition processing result; and determining the number of objects in the speech block based on the object recognition results. Specifically, in the process of performing object recognition processing on the speech block based on the connectionist temporal classification model, on the one hand, due to the accuracy problem of the connectionist temporal classification model, and on the other hand, due to the characteristic that the recognition results are not unique, the obtained recognition results are not a uniquely determined set of results. Therefore, when multiple sets of object recognition processing results may be obtained, the accuracy of multiple sets of recognition results can be judged, and the higher the accuracy, the greater the probability that it is the object recognition processing result. Therefore, when more than one set of object recognition processing results is obtained, it is necessary to first determine the most likely set of recognition results, and then determine the number of recognized objects in the speech block based on this set of recognition results.

[0051] In an exemplary embodiment, obtaining an object recognition result by performing object recognition processing on the speech block based on the connectionist temporal classification model may include: encoding the speech block to obtain a feature sequence of the speech block; and performing object recognition processing on the feature sequence of the speech block based on the connectionist temporal classification model to obtain an object recognition result. The connectionist temporal classification model can calculate a loss function for a given input that corresponds to a probability distribution of all possible outputs. This probability distribution can then be used to predict the probability of the output with the highest probability or a specific output. Therefore, the first step in speech block recognition in a speech recognition method is to encode or extract features from the speech block to be recognized to obtain its corresponding feature sequence as input to the connectionist temporal classification model; that is, to extract the identifiable features from the audio signal as input to the speech block to be recognized. For example, in the connectionist temporal classification model recognition, each 10 milliseconds is typically considered as a speech block. Taking the speech block "I love you China" as an example, the feature sequence of this speech block is obtained through feature extraction methods such as MFCC or LPC and input into the connectionist temporal classification model. The connectionist temporal classification model then decodes and recognizes the pronunciation methods contained therein, converts them into text, and outputs them. Next, it determines the number of recognized objects contained in the speech block based on the output results.

[0052] In one exemplary embodiment, a feature extraction method is MFCC (Mel Frequency Cepstral Coefficients) feature extraction. This method passes the speech spectrum through a set of triangular filters and performs a Discrete Cosine Transform (DCT) to obtain MFCC coefficients, which represent the distribution of energy in the signal spectrum across different frequency ranges. Specifically, by setting filters, the spectral energy of the corresponding frequency range can be obtained; for example, 26 triangular filters can be set to obtain 26 MFCC coefficients, and then the lower-order coefficients are taken to obtain feature information representing the vocal tract.

[0053] Another feature extraction method is Linear Predictive Coding (LPC), which is typically used to encode aspects of speech such as pitch, formants, and short-time spectra. This feature extraction method can accurately estimate speech parameters and is easy to compute. Specifically, a speech signal can be modeled as the output of a linear time-varying system, where the input excitation signal is either a periodic pulse (during voiced sounds) or random noise (during unvoiced sounds). The difference equation of the speech signal indicates that the sampling of the speech signal can be approximated by linear fitting of past samples. Then, by locally minimizing the sum of squared differences between the actual samples and the linearly predicted samples, a set of prediction coefficients can be obtained. Linear predictive analysis can be implemented in modeling speech signal problems using various methods, such as the covariance method, autocorrelation formula, lattice method, inverse filter, spectral estimation formula, maximum likelihood formula, and inner product formula.

[0054] Step S35: Determine the number of recognition attempts based on the number of objects in the speech block, and perform speech recognition processing on the speech block based on the number of recognition attempts using an attention model to obtain the speech recognition result corresponding to the speech block.

[0055] Attention mechanisms are used to improve the performance of encoders and decoders based on recurrent neural networks (RNNs) (or long short-term memory artificial neural networks, GRUs). They are widely used in machine translation, speech recognition, image annotation, and other fields. Attention mechanisms have the ability to distinguish between different objects being recognized. For example, in machine translation and speech recognition applications, attention mechanisms assign different weights to each word in a sentence, making the learning of the neural network model more flexible. Attention mechanisms can also serve as an alignment mechanism to explain the alignment between the input and output sentences and the content learned. Attention mechanisms can assign different weights to each part of the input speech signal, extracting key and important information, enabling the model to make more accurate judgments, while the entire recognition process does not require a large amount of computation or excessive memory. In attention mechanisms, the encoder encodes the input speech signal into a vector sequence, while the decoder selectively selects a subset of the vector sequence for further processing during decoding. Therefore, attention mechanisms can fully utilize the information carried by the input sequence when generating each output.

[0056] The attention model for speech recognition not only focuses on the global semantic encoding feature vector but also adds an "attention range" to indicate which parts of the input sequence the output needs to focus on. It then generates the next output based on the region of focus, processing one object to be recognized in each recognition block. Therefore, in the streaming speech recognition method of this embodiment, the number of objects to be recognized in a speech block is first determined using a connectionist temporal classification model, and then the attention model performs the recognition a corresponding number of times to determine all the recognition results for that speech block.

[0057] In an exemplary embodiment, obtaining the speech block to be identified may further include: if an unidentified object is detected in the previous speech block to be identified, performing object recognition processing on the previous speech block to be identified. Specifically, this exemplary embodiment utilizes the characteristic of the attention model that it can identify specific objects by combining global data. During its recognition process, a specific delay can be set. For example, if the current speech block contains five objects to be identified, the attention model may only perform four recognitions on the speech block in this current recognition, delaying the recognition of the last speech block near the boundary until the recognition of the next speech block. This embodiment can solve the problem that the attention model cannot identify objects at the boundary of a speech block based on global data and the preceding and following relationships.

[0058] Furthermore, the streaming speech recognition method of this embodiment also includes an evaluation system for the accuracy of the recognition results. In an exemplary embodiment, determining the number of recognition attempts based on the number of objects in the speech block, and performing speech recognition processing on the speech block for the number of recognition attempts based on an attention model to obtain a speech recognition result corresponding to the speech block, includes: determining the number of recognition attempts based on the number of objects in the speech block, and performing speech recognition processing on the speech block for the number of recognition attempts based on an attention model to obtain at least one set of speech recognition processing results and corresponding accuracy; determining the set of speech recognition processing results with the highest accuracy as the speech recognition result corresponding to the speech block. The recognition result mentioned in this embodiment is a set of sentences composed of multiple recognition objects. In some cases, speech recognition does not only determine a unique recognition object. Therefore, multiple sets of recognition results may be obtained for a speech block. Speech recognition based on an attention model may result in a non-unique recognition result. Therefore, when more than one set of recognition results is obtained, the accuracy of each set of recognition results can be used as an evaluation criterion to determine the most likely set of recognition results. In one embodiment, a score evaluation standard can be preset to screen the accuracy of the recognition object. If the score is lower than the preset threshold, it can be considered that the accuracy is too low and the recognition result is not adopted. Alternatively, if the score is lower than the threshold, the speech block can be re-recognized to obtain a more accurate recognition result.

[0059] Another exemplary embodiment of this application provides a streaming speech recognition device. Figure 4 This is a schematic diagram of the structure of a streaming speech recognition device according to an embodiment of this application. Figure 4 As shown, the streaming speech recognition device 40 includes:

[0060] Acquisition module 42 is used to acquire the speech blocks to be recognized;

[0061] The first recognition module 44 is used to perform object recognition processing on the speech block based on the connectionist temporal classification model to obtain object recognition results, and to determine the number of recognized objects in the speech block based on the optimal object recognition processing results obtained by the object recognition processing.

[0062] The second recognition module 46 is used to determine the number of recognitions based on the number of objects in the recognized speech block, and to perform speech recognition processing on the speech block based on the number of recognitions using an attention model to obtain the target speech recognition result corresponding to the optimal speech block.

[0063] The specific details of each module / unit in the above-described device have been described in detail in the corresponding method sections and will not be repeated here. It should be noted that although several modules or units of the device for executing actions have been mentioned in the detailed description above, this division is not mandatory. In fact, according to the embodiments of this application, the features and functions of two or more modules or units described above can be embodied in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided and embodied by multiple modules or units.

[0064] In addition to the methods and apparatus described above, embodiments of this application may also be computer program products, which include computer program instructions that, when executed by a processor, cause the processor to perform the steps in the methods according to various embodiments of this application described in the "Exemplary Methods" section above.

[0065] The computer program product can be written in any combination of one or more programming languages ​​to perform the operations of the embodiments of this application. The programming languages ​​include object-oriented programming languages ​​such as Java and C++, as well as conventional procedural programming languages ​​such as C or similar languages. The program code can be executed entirely on the user's computing device, partially on the user's computing device, as a standalone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0066] Another embodiment of this application provides an electronic device that can be used to perform all or part of the steps of the method or network control method described in this exemplary embodiment. The device includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the steps of the methods according to various embodiments of this application described in the "Exemplary Methods" section of this specification.

[0067] Another embodiment of this application provides a computer-readable storage medium having computer program instructions stored thereon, which, when executed by a processor, cause the processor to perform the steps of the methods according to various embodiments of this application described in the "Exemplary Methods" section of this specification.

[0068] The computer-readable storage medium may be any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may, for example, include, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatuses, or devices, or any combination thereof. More specific examples of readable storage media (a non-exhaustive list) include: electrical connections having one or more wires, portable disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0069] The basic principles of this application have been described above with reference to specific embodiments. However, it should be noted that the advantages, benefits, and effects mentioned in this application are merely examples and not limitations, and should not be considered as essential features of each embodiment of this application. Furthermore, the specific details disclosed above are for illustrative and facilitative purposes only, and are not limitations. These details do not limit the application to the necessity of employing the aforementioned specific details for implementation.

[0070] The block diagrams of devices, apparatuses, devices, and systems involved in this application are merely illustrative examples and are not intended to require or imply that they must be connected, arranged, or configured in the manner shown in the block diagrams. As those skilled in the art will recognize, these devices, apparatuses, devices, and systems can be connected, arranged, and configured in any manner. Words such as “comprising,” “including,” “having,” etc., are open-ended terms meaning “including but not limited to,” and are used interchangeably with them. The terms “or” and “and” as used herein refer to the terms “and / or,” and are used interchangeably with them unless the context clearly indicates otherwise. The term “such as” as used herein refers to the phrase “such as but not limited to,” and is used interchangeably with it.

[0071] Other embodiments of this application will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this application that follow the general principles of this application and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this application are indicated by the following claims.

[0072] It should be understood that this application is not limited to the precise structure described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this application is limited only by the appended claims.

Claims

1. A streaming speech recognition method, characterized in that, Includes the following steps: Obtain the speech block to be recognized; Based on the connectionist temporal classification model, object recognition processing is performed on the speech block to obtain at least one set of object recognition processing results and corresponding accuracy. The number of objects in the speech block is determined by determining the set of object recognition processing results with the highest accuracy. The number of recognition attempts is determined based on the number of objects in the speech block, and the speech block is processed according to the number of recognition attempts based on the attention model to obtain the speech recognition result corresponding to the speech block.

2. The streaming speech recognition method according to claim 1, characterized in that, The object recognition result obtained by performing object recognition processing on the speech block based on the connectionist temporal classification model includes: The speech block is encoded to obtain the feature sequence of the speech block; The object recognition result is obtained by performing object recognition processing on the feature sequence of the speech block based on the connectionist temporal classification model.

3. The streaming speech recognition method according to claim 1, characterized in that, The step of determining the number of recognition attempts based on the number of objects in the speech block, and performing speech recognition processing on the speech block based on the number of recognition attempts using an attention model to obtain a speech recognition result corresponding to the speech block includes: The number of recognition attempts is determined based on the number of objects in the speech block, and the speech block is processed according to the number of recognition attempts based on the attention model to obtain at least one set of speech recognition processing results and corresponding accuracy. The results of the speech recognition processing with the highest accuracy are determined as the speech recognition results corresponding to the speech block.

4. The streaming speech recognition method according to claim 1, characterized in that, The process of obtaining the speech block to be recognized includes: If an unrecognized object is detected in the previous speech block to be recognized, object recognition processing is performed on the previous speech block to be recognized.

5. The streaming speech recognition method according to claim 3, characterized in that, Also includes: If the accuracy of the results of at least one set of speech recognition processing is less than the threshold, the speech block is reprocessed for object recognition or speech recognition.

6. The streaming speech recognition method according to claim 1, characterized in that, The process of obtaining the speech block to be recognized includes: The speech block is extracted from the speech signal to be recognized according to a specified time range or a specified speech block size.

7. A streaming speech recognition device, characterized in that, include: The acquisition module is used to acquire the speech blocks to be recognized; The first recognition module is used to perform object recognition processing on the speech block based on the connectionist temporal classification model to obtain at least one set of object recognition processing results and corresponding accuracy, and to determine the number of objects in the speech block by determining the set of object recognition processing results with the highest accuracy. The second recognition module is used to determine the number of recognition attempts based on the number of objects in the speech block, and to perform speech recognition processing on the speech block based on the number of recognition attempts using an attention model to obtain a speech recognition result corresponding to the speech block.

8. An electronic device, characterized in that, include: At least one processor; as well as, A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the streaming speech recognition method as described in claims 1-6.

9. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the streaming speech recognition method as described in claims 1-6.

Images