Speech recognition method, speech recognition system and storage medium

Abstract

The present invention provides a speech recognition method, a speech recognition system and a storage medium. The speech recognition method comprises: a speech stream processing step, receiving a speech stream and dividing the speech stream into speech frames; a speech frame processing step, performing a silence judgment on the speech frames; and a speech recognition step, exchanging messages with a speech recognition engine according to the result of the silence judgment.

Description

Technical Field

[0001] The present invention generally relates to the field of automatic processing of speech, and more specifically to a speech recognition method, a speech recognition system and a storage medium. Background Art

[0002] Speech recognition technology has been widely used in all aspects of production and life. For example, in the call scenario, the main application scenarios of real-time speech recognition technology include but are not limited to real-time speech recognition scenarios of intelligent applications of call centers such as intelligent agent assistants and real-time quality inspection. The basis for realizing these business scenarios is to use the speech recognition engine to recognize the caller and the called voice in the call as text as the input of the subsequent business processing module. In addition, these scenarios have relatively high requirements for the real-time performance of speech recognition, and basically what is said is what is received. The call scenario is a two-person dialogue scenario, which is divided into the caller and the called. The current implementation method is generally that one call occupies the concurrent capabilities of two speech recognition engines, including one caller voice and one called voice. Under this mechanism, one engine supports one voice. Converted into the number of calls, the maximum number of calls supported by the speech recognition engine = the number of concurrent voice recognition engines / 2.

[0003] As the number of calls increases rapidly, the concurrency requirements for the voice recognition engine are also higher, which places higher demands on both software and hardware resources. Summary of the invention

[0004] A brief overview of the invention is given below in order to provide a basic understanding of some aspects of the invention. However, it should be understood that this overview is not an exhaustive overview of the invention. It is not intended to identify the key or important parts of the invention, nor is it intended to limit the scope of the invention. Its purpose is merely to give some concepts of the invention in a simplified form as a prelude to a more detailed description that will be given later.

[0005] When people speak, there are two scenarios: speaking and pausing, and the corresponding speech is divided into two parts: speech and silence. In a two-person conversation scenario, when one party speaks, the other party is often in a listening state. The call scenario is also a two-person conversation scenario, which is manifested in that when the caller speaks, the called party is silent, and when the called party speaks, the caller is silent. In the current recognition scenario, if a call lasts 30 seconds, the speech recognition engine will occupy 30 seconds of the caller and called party's speech recognition time. However, the single-track speech in the conversation (either the caller or the called party) generally has a long silent segment, and the recognition of the silent segment is actually a waste of the speech recognition engine's capabilities.

[0006] In view of this problem, the present invention proposes a speech recognition method, a speech recognition system and a storage medium, which can realize that only valid speech frames are transmitted to the speech recognition engine, thus saving engine recognition resources for silent segments.

[0007] According to one aspect of the present invention, there is provided a speech recognition method, comprising: a speech stream processing step of receiving a speech stream and dividing the speech stream into speech frames; a speech frame processing step of performing silence judgment on the speech frames; and a speech recognition step of interacting with a speech recognition engine based on the result of the silence judgment.

[0008] In an embodiment of the present invention, the method further includes: a recognition result processing step, after obtaining the recognition result from the speech recognition engine, calculating the time position of the recognition result in the original speech stream.

[0009] In an embodiment of the present invention, in the recognition result processing step, the time position of the recognition result in the original voice stream is calculated according to the time position returned by the voice recognition engine and the stored silence duration information.

[0010] In an embodiment of the present invention, in the speech recognition step, exchanging messages with the speech recognition engine includes creating a new speech recognition session, sending a speech frame to be recognized, obtaining a recognition result, and ending the speech recognition session.

[0011] In the embodiment of the present invention, the session information of the newly created speech recognition session includes: a session identifier, a speech identifier, call path information of the speech, and a processing position of the speech.

[0012] In an embodiment of the present invention, in the voice frame processing step, silence determination is performed on the voice frame through voice endpoint detection.

[0013] In an embodiment of the present invention, if the speech frame is not a silent frame and the previous speech frame is a silent frame, a new speech recognition session is created; if the speech frame is not a silent frame and the previous speech frame is not a silent frame, the speech recognition is continued; if the speech frame is a silent frame and the previous speech frame is not a silent frame, the speech recognition session is terminated; if the speech frame is a silent frame and the previous speech frame is a silent frame, the duration of the silent segment is calculated.

[0014] In an embodiment of the present invention, when the voice recognition session is newly created, the current concurrency number of the voice recognition engine is increased, and when the voice recognition session is ended, the current concurrency number of the voice recognition engine is decreased.

[0015] In an embodiment of the present invention, if the concurrency of the speech recognition engine reaches an upper limit, the new speech frames are cached and the earlier speech frames are discarded.

[0016] In an embodiment of the present invention, the early speech frames exceeding the maximum duration of the speech frame buffer are discarded.

[0017] In an embodiment of the present invention, the speech frame includes a first frame, an intermediate frame, and a last frame, and the first frame is processed as follows: if the first frame is a silent frame, the silent time is accumulated; if the first frame is a non-silent frame, it is determined whether the speech recognition engine has spare recognition capacity, and if so, the speech recognition is performed; if the recognition capacity is full, the speech frame is cached or discarded according to the cache strategy, and the intermediate frame is processed as follows: if the intermediate frame is a silent frame, it is determined whether the previous frame of speech is a silent frame, and if the previous frame of speech is silent, the silent time is accumulated, and if the previous frame of speech is If the first frame is not silent, the speech recognition is terminated, the final recognition result is obtained, and the time position of the recognition result is calculated; if the intermediate frame is a non-silent frame, it is determined whether the previous frame of speech is a silent frame. If the previous frame of speech is silent, the same processing as the case where the first frame is a non-silent frame is performed; if the previous frame of speech is not silent, the speech recognition is continued to be cached or continued according to the session situation, and the speech frame processing is performed on the last frame as follows: if the previous frame of speech is silent, the cache is cleared; if the previous frame of speech is not silent, the speech recognition is terminated, the final recognition result is obtained, and the time position of the recognition result is calculated.

[0018] According to another aspect of the present invention, there is provided a speech recognition system, comprising: a speech stream processing module, which receives a speech stream and divides the speech stream into speech frames; a speech frame processing module, which performs silence judgment on the speech frames; and a speech recognition module, which interacts with a speech recognition engine based on the result of the silence judgment.

[0019] According to another aspect of the present invention, there is provided a speech recognition system, comprising: a memory on which instructions are stored; and a processor configured to execute the instructions stored in the memory to perform the above-mentioned speech recognition method.

[0020] According to another aspect of the present invention, there is provided a computer-readable storage medium, comprising computer-executable instructions, which, when executed by one or more processors, enable the one or more processors to perform the above-mentioned speech recognition method.

[0021] According to the embodiment of the present invention, only valid speech frames are transmitted to the speech recognition engine, saving engine recognition resources for silent segments, which can effectively reduce the number of concurrency required by the speech recognition engine. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] The accompanying drawings, which constitute a part of the specification, illustrate embodiments of the present invention and, together with the description, serve to explain the principles of the present invention.

[0023] The present invention may be more clearly understood from the following detailed description with reference to the accompanying drawings, in which:

[0024] Figure 1 is an exemplary flow chart of a speech recognition method according to an embodiment of the present invention.

[0025] Figure 2 According to an embodiment of the present invention Figure 1 A detailed exemplary flowchart of the steps for processing speech frames in a speech recognition method.

[0026] Figure 3 An exemplary configuration of a computing device in which embodiments according to the present invention may be implemented is shown. DETAILED DESCRIPTION

[0027] The following detailed description is made with reference to the accompanying drawings, and the following detailed description is provided to assist in a comprehensive understanding of various exemplary embodiments of the present invention. The following description includes various details to assist in understanding, but these details are considered to be examples only and are not intended to limit the present invention, which is defined by the appended claims and their equivalents. The words and phrases used in the following description are only used to enable a clear and consistent understanding of the present invention. In addition, for the sake of clarity and brevity, descriptions of well-known structures, functions, and configurations may be omitted. Those of ordinary skill in the art will recognize that various changes and modifications may be made to the examples described herein without departing from the spirit and scope of the present invention.

[0028] Figure 1 FIG. 1 is an exemplary flow chart of a speech recognition method according to an embodiment of the present invention. The speech recognition method according to an embodiment of the present invention may include steps S101 to S103.

[0029] like Figure 1 As shown, step S101 is a voice stream processing step, receiving a voice stream and dividing the voice stream into voice frames.

[0030] After the voice stream is acquired, the voice stream is divided into frames according to a preset length. For example, a real-time voice stream is received, the number of bytes is calculated according to the audio sampling rate, number of channels, bit depth, and frame length, and the real-time voice stream is divided into N frames.

[0031] like Figure 1 As shown, step S102 is a speech frame processing step, in which silence determination is performed on the speech frame.

[0032] In some embodiments, in the voice frame processing step, silence determination may be performed on the voice frame through voice endpoint detection.

[0033] For example, speech endpoint detection refers to finding the silent segment from a speech, that is, finding the starting point and ending point of the non-silent segment from the speech signal.

[0034] In some embodiments, it may be that: if the speech frame is not a silent frame and the previous speech frame is a silent frame, a new speech recognition session is created; if the speech frame is not a silent frame and the previous speech frame is not a silent frame, speech recognition is continued; if the speech frame is a silent frame and the previous speech frame is not a silent frame, the speech recognition session is ended; if the speech frame is a silent frame and the previous speech frame is a silent frame, the duration of the silent segment is calculated.

[0035] The detection method is to make a silence judgment on the voice frame. If it is not a silence frame, the state of the previous voice frame is used to decide whether to create a new recognition session or continue the recognition. If it is silence, the previous voice frame is used to decide whether to end a recognition session. After the current session is ended, another voice stream that needs to be recognized will have the right to create a new session. For example, if the voice frame is a silence frame and the previous frame is a silence frame, it is considered to be a continuous silence segment, and only the silence segment time needs to be accumulated. If the previous frame is not a silence frame, it is considered to be the end endpoint of the non-silent segment, and the voice recognition session is ended.

[0036] like Figure 1 As shown, step S103 is a speech recognition step, and according to the result of the silence judgment, a message is exchanged with the speech recognition engine. For example, if it is judged to be a silence frame, the frame may not be sent to the speech recognition engine. Only the frame judged to be non-silent is sent to the speech recognition engine for recognition. This can save the resources of the speech recognition engine.

[0037] For example, the following algorithm is used for silence judgment. The embodiment of the present invention mainly uses the speech energy value as the speech feature for analysis and processing, and adopts the "short-time energy" method. The so-called "short-time energy" means that the speech signal is first framed. The embodiment of the present invention also simplifies the speech frame processing, uses a fixed time length for fast speech framing, and calculates the energy of each frame after framing. Suppose the nth frame speech signal (given by the amplitude value x n The short-time energy of (m) is represented by En, and its calculation formula is shown in the following formula:

[0038]

[0039] En is a function that measures the change in the amplitude of the voice signal. It uses the square of the amplitude of the signal, so it is more sensitive to large signals. The distinction between speech and silent noise can be reflected in its energy. The signal-to-noise ratio of the voice signal in the call scenario is relatively high. As long as the short-time energy is used, the voice segment (non-silent segment) and the noise segment (silent segment) can be distinguished. For example, a frame with a short-time energy less than a preset threshold can be considered a silent segment. m=0…N-1 respectively represents the N sampling points in the nth frame.

[0040] For example, assuming that the silent segment accounts for 50% of the voice, the present invention only recognizes the non-silent segment, then, in the 30-second call time, the voice engine is occupied for only 15 seconds. The saved 15 seconds of engine recognition capability can be used for another call.

[0041] The speech recognition method of the embodiment of the present invention may further include step S104.

[0042] like Figure 1 As shown, step S104 is a recognition result processing step. After obtaining the recognition result from the speech recognition engine, the time position of the recognition result in the original speech stream is calculated.

[0043] For example, in the recognition result processing step, the time position of the recognition result in the original voice stream is calculated based on the time position returned by the voice recognition engine and the stored silence duration information.

[0044] The time position returned by the speech engine may be the time position of the recognized result text in the recognized speech segment.

[0045] That is, although the silent time period does not need to be identified, its duration information can be recorded to accurately reproduce the temporal correlation (eg, interval time, etc.) of each non-silent time period.

[0046] For example, while performing real-time speech recognition, the recognition result is obtained, which includes the recognized text and the front end and the back end of the text. The front end is added to the silent time stored in the cache to obtain the position of the recognized text result in the original audio.

[0047] Figure 2 According to an embodiment of the present invention Figure 1 A detailed exemplary flow chart of the steps of processing a speech frame in a speech recognition method of Figure 1 An example of more detailed processing in steps S102 to S104 is shown.

[0048] For example, first, in Figure 1 In step S101, the byte stream is converted into a frame with a duration of one second. The sampling rate of the voice stream of the call is generally 8kHz, 16 bits, and mono. The byte stream after frame division is 16000 bytes.

[0049] Since the call recording has less noise, this embodiment uses the short-time energy method to detect the voice endpoint. The cache is implemented using a memory database. The maximum voice cache length is 2 seconds of voice, which can ensure the real-time recognition results. The specific process of voice recognition processing is as follows: Figure 2 shown.

[0050] In some embodiments, the speech frame includes a first frame, an intermediate frame, and a last frame (for example, the first frame of speech refers to the first frame information of a speech, the intermediate frame of speech refers to the intermediate frame information of the speech, and the last frame of speech refers to the last frame information of the speech), and the first frame is processed as follows: if the first frame is a silent frame (also called a "silent point"), the silent time is accumulated (for example, the cached call audio silent time is added with a time unit, such as 1 second); if the first frame is a non-silent frame (also called a "non-silent point"), it is determined whether the speech recognition engine has space. If the recognition capacity is full, the voice frame is cached or discarded according to the cache strategy (for example, the audio is cached in the memory database and waits for other sessions to end), and the intermediate frame is processed as follows: if the intermediate frame is a silent frame, it is determined whether the previous frame of voice is a silent frame. If the previous frame of voice is silent, the silent time is accumulated (for example, the silent time in the memory database is increased by one). time unit, such as 1 second), if the previous frame of speech is not silent, then end speech recognition, obtain the last recognition result, and calculate the position of the recognition result (specifically, the position in time) (for example, obtain the recognition engine result, end the session, and read other cached audio in the memory database to start processing); if the middle frame is a non-silent frame, then determine whether the previous frame of speech is a silent frame, if the previous frame of speech is silent, perform the same processing as the case where the first frame is a non-silent frame (for example, check whether there is an available session, if so, obtain the session ID to start recognition; if not, cache the audio in the memory database and wait for other sessions to end); if the previous frame of speech is not silent, then continue to cache or continue speech recognition (for example, continue speech recognition) according to the session situation, and perform the following speech frame processing on the last frame: if the previous frame of speech is silent, clear the cache (for example, clear the cache information of the call in the memory database); if the previous frame of speech is not silent, then end speech recognition, obtain the last recognition result, and calculate the position of the recognition result (for example, obtain the recognition result, end the session, and clear the cache information of the call in the memory database).

[0051] Since the speech of the silent segment is no longer sent to the speech recognition engine, the bg (the front end point of this result) of the return result of the recognition engine will be different from the original end point time of the audio. In order to obtain the original position of the recognition result in the audio. The present invention designs a method for unifying the time end point: assuming that the length of the speech frame is one second, if the audio frame is judged to be a silent point, a one-second silent time is added to the corresponding information of the call in the cache. In this way, all processed silent points are cached as records, and the recognition engine obtains the result, and after obtaining the bg, the silent time stored in the cache is added, which can be aligned with the original time end point of the audio. For example, the first second and the second second of audio a are silent, and the silent time of a in the cache is accumulated to 2 seconds. The third and fourth seconds are not silent, and the recognition engine returns the result: "Good morning", "bg: 0", which is different from the real audio. Then, the program calculates the real bg: 2 (indicates that it starts 2 seconds after the initial position) corresponding to "good morning" based on the previous silent time accumulated to 2 seconds, which is consistent with the original result of the audio.

[0052] In some embodiments, in the speech recognition step, interacting with the speech recognition engine includes creating a new speech recognition session, sending a speech frame to be recognized, obtaining a recognition result, and ending the speech recognition session.

[0053] For example, the speech recognition step can be implemented using a speech recognition interface. The processing mechanism of the speech recognition interface is as follows: when the speaker starts speaking, the speech recognition engine creates a new session, while the speaker is speaking, the speech stream is synchronized to the speech recognition engine, and the recognition result is obtained in real time. After the speaker finishes speaking, the final recognition result is obtained and the session ends.

[0054] The message interaction of the speech recognition interface adopts the HTTP interface protocol, which is easy to expand and has high compatibility. For example, a non-blocking, event-driven model programming framework can be used to implement high-concurrency interface calls. New speech recognition session interface: Establish a new recognition session before starting speech recognition. Real-time speech stream synchronization interface: Synchronize the speech frames to be recognized to the speech recognition engine through real-time speech frame data synchronization. Real-time recognition result acquisition interface: While transmitting the real-time speech stream, obtain the real-time recognition results through the interface. End speech recognition session interface: End the speech recognition session after the speech ends.

[0055] In some embodiments, the session information of the newly created speech recognition session includes: a session identifier, a speech identifier, call path information of the speech, and a processing location of the speech.

[0056] For example, the session identifier is a unique ID of the recognition session established with the speech recognition engine; the speech identifier is a unique ID to distinguish the calling voice or the called voice in a call; the voice call accompanying information is information related to the call, including but not limited to: calling number, called number, call ID, calling voice or called voice; the voice processing position is the time position of the currently processed voice frame in the entire voice.

[0057] The established session information needs to be stored in the cache. The session information includes the session identifier, the voice identifier, the voice call path information, and the voice processing location.

[0058] In some embodiments, when a new speech recognition session is created, the current concurrency of the speech recognition engine is increased, and when the speech recognition session is ended, the current concurrency of the speech recognition engine is decreased.

[0059] The real-time speech recognition mechanism provided by the speech recognition engine generally adopts session-based management. After establishing a speech recognition session, real-time speech frames are transmitted as input data for speech recognition. At the same time, real-time speech recognition results are obtained through the interface. After the last frame is transmitted to the speech recognition engine, the session ends. Generally, the purchased speech recognition engines have a concurrency limit. Since the concurrency of the speech recognition engine will be frequently queried and changed during the real-time speech recognition process, a high-speed cache is used to store the concurrency. When starting a recognition session, the current concurrency needs to be increased, and when ending a recognition session, the current concurrency needs to be reduced.

[0060] In some embodiments, if the concurrency of the speech recognition engine reaches an upper limit, the new speech frames are cached and the earlier speech frames are discarded.

[0061] In some embodiments, early speech frames that exceed the maximum duration of the speech frame buffer are discarded.

[0062] Assume that the speech recognition engine supports 10 recognition sessions in total, and there are 20 voices that need to be recognized. If the silence segments of 10 of the voices correspond exactly to the speech segments of the other 10 voices, theoretically, the engine resources of the 10 silence segments can be transferred to the other 10 speech segments for execution, so that the effect of 10 recognition sessions supporting 20 voices can be achieved. However, in reality, the silence segments of the voices are unevenly distributed, the length of the speech segments is inconsistent, and the randomness of the distribution and duration of the effective voices in the voices leads to more than 10 sessions requesting the recognition capability of the speech recognition engine at the same time, resulting in some transcription requests being discarded.

[0063] In order to solve this problem, the present invention introduces a voice frame cache mechanism and a cache discard mechanism. If the number of concurrent calls of the speech recognition engine has reached the upper limit, new voice frames will be cached. Since high-speed reading is required, the storage of the voice frame cache is implemented by high-speed cache. As long as the voice frames in the cache are sent to the speech recognition engine before the end of the entire sentence, the change in the transcription response time can be achieved without perception. When a whole sentence of real-time speech of the speaker ends, if there are still many voice frames left in the cache to be recognized in the sentence, the real-time performance of speech recognition will be affected, which may be manifested as a long time after the speaker finishes speaking before the speech recognition result is obtained. The longer the audio is cached, the worse the real-time performance of speech recognition. Therefore, it is necessary to select the maximum duration of the voice frame cache according to the actual business needs. When the maximum cache duration is exceeded, the early voice frames are considered to be expired caches and will be discarded to ensure the real-time performance of speech recognition.

[0064] Other existing technologies that use voice endpoint detection for speech recognition are mostly aimed at single-sentence recognition scenarios and are mostly used in non-real-time recognition applications. They do not have the response performance required for real-time recognition.

[0065] According to an embodiment of the present invention, voice endpoint detection technology is used to distinguish valid voice frames from silent frames, and only valid voice frames are recognized. In real-time voice recognition scenarios, voice engine recognition resources for silent segments are saved. By using an innovative voice frame cache mechanism and cache discard mechanism, the real-time performance of recognition is guaranteed while the utilization rate of the voice recognition engine is improved. The response time of single-sentence voice recognition can be controlled at around 400 milliseconds, which is suitable for business scenarios with high real-time recognition requirements such as agent assistants and real-time quality inspection. At the same time, real-time recognition result position calculation technology is used to accurately restore the position of the recognition result text in the original voice.

[0066] According to the embodiments of the present invention, the problem of greatly improving the utilization rate of the speech recognition engine and reducing the procurement cost of the speech recognition engine in the real-time speech recognition scenario is solved without affecting the original recognition quality and response time. Under the existing technology, the maximum number of calls supported by the speech recognition in the call scenario = the number of concurrent calls of the speech recognition engine / 2; using the speech recognition method of the embodiments of the present invention, the utilization rate of the speech engine can be doubled, that is, the maximum number of calls supported by the speech recognition in the call scenario = the number of concurrent calls of the speech recognition engine.

[0067] The present invention also provides a speech recognition system, comprising: a speech stream processing module, which receives a speech stream and divides the speech stream into speech frames; a speech frame processing module, which performs silence judgment on the speech frames; and a speech recognition module, which interacts with a speech recognition engine based on the result of the silence judgment.

[0068] The embodiments of the present invention pre-process the real-time voice stream and utilize technologies such as voice endpoint detection, engine concurrency management, voice session management, voice frame caching, and result endpoint calculation to achieve the goal of only transmitting valid voice frames to the voice recognition engine in real time, thereby saving engine recognition resources for silent segments.

[0069] The embodiments of the present invention are applied to real-time speech recognition scenarios of intelligent applications of call centers such as agent assistants and real-time quality inspection. The agent assistant displays the conversation text in real time when the operator answers the call. The conversation text can quickly locate the user's intention after being processed by AI semantic understanding, which is convenient for the operator to carry out subsequent business processing. Real-time quality inspection monitors the service quality of the operator by real-time identification of keywords and other information in the transcribed text. The above-mentioned businesses all have extremely high requirements for the real-time performance of speech recognition. After adopting speech preprocessing, the present invention can ensure that the real-time performance of speech recognition is not reduced, and the single sentence transcription delay is about 400 milliseconds, which meets the requirements of the above-mentioned business scenarios.

[0070] By implementing and using the present invention, the utilization rate of the real-time speech recognition engine can be increased by nearly one-fold, and the investment cost of the speech recognition engine can be greatly reduced.

[0071] Figure 3 An exemplary configuration of a computing device 300 is shown in which embodiments according to the present invention may be implemented.

[0072] The computing device 300 is an example of a hardware device to which the above aspects of the present invention can be applied. The computing device 300 can be any machine configured to perform processing and / or calculations. The computing device 300 can be, but is not limited to, a workstation, a server, a desktop computer, a laptop computer, a tablet computer, a personal data assistant (PDA), a smart phone, a vehicle-mounted computer, or a combination thereof.

[0073] like Figure 3As shown, the computing device 300 may include one or more components that may be connected or communicated with the bus 302 via one or more interfaces. The bus 302 may include, but is not limited to, an Industry Standard Architecture (ISA) bus, a Micro Channel Architecture (MCA) bus, an Enhanced ISA (EISA) bus, a Video Electronics Standards Association (VESA) local bus, and a Peripheral Component Interconnect (PCI) bus. The computing device 300 may include, for example, one or more processors 304, one or more input devices 306, and one or more output devices 308. The one or more processors 304 may be any type of processor and may include, but is not limited to, one or more general-purpose processors or special-purpose processors (such as special-purpose processing chips). The processor 302 may, for example, be configured to execute instructions stored on the memory to perform Figure 1 The speech recognition method described above. Alternatively, the processor 302 can also implement the functions of the above-mentioned voice stream processing module, voice frame processing module, and speech recognition module. The input device 306 can be any type of input device capable of inputting information to the computing device, and can include but is not limited to a mouse, a keyboard, a touch screen, a microphone and / or a remote controller. The output device 308 can be any type of device capable of presenting information, and can include but is not limited to a display, a speaker, a video / audio output terminal, a vibrator and / or a printer.

[0074] The computing device 300 may also include or be connected to a non-transitory storage device 314, which may be any non-transitory storage device that can implement data storage, and may include, but is not limited to, a disk drive, an optical storage device, a solid-state memory, a floppy disk, a flexible disk, a hard disk, a tape, or any other magnetic medium, a compressed disk or any other optical medium, a cache memory, and / or any other storage chip or module, and / or any other medium from which a computer can read data, instructions, and / or code. The computing device 300 may also include a random access memory (RAM) 310 and a read-only memory (ROM) 312. The ROM 312 may store programs, utilities, or processes to be executed in a non-volatile manner. The RAM 310 may provide volatile data storage and store instructions related to the operation of the computing device 300. The computing device 300 may also include a network / bus interface 316 coupled to a data link 318. The network / bus interface 316 can be any type of device or system capable of enabling communication with external devices and / or networks, and can include but is not limited to modems, network cards, infrared communication devices, wireless communication devices and / or chipsets (such as Bluetooth™ devices, 802.11 devices, WiFi devices, WiMax devices, cellular communication facilities, etc.).

[0075] The present invention may be implemented as any combination of a device, a system, an integrated circuit, and a computer program on a non-transitory computer-readable storage medium. One or more processors may be implemented as an integrated circuit (IC), an application specific integrated circuit (ASIC), or a large scale integrated circuit (LSI), a system LSI, a super LSI, or a super LSI component that performs some or all of the functions described in the present invention.

[0076] The present invention includes the use of software, applications, computer programs or algorithms. The software, applications, computer programs or algorithms can be stored on a non-transitory computer-readable storage medium to enable a computer such as one or more processors to perform the above steps and the steps described in the accompanying drawings. For example, one or more memories store the software or algorithm in executable instructions, and one or more processors can associate a set of instructions to execute the software or algorithm to provide various functions according to the embodiments described in the present invention.

[0077] Software and computer programs (which may also be referred to as programs, software applications, applications, components or code) include machine instructions for a programmable processor and may be implemented in a high-level procedural language, an object-oriented programming language, a functional programming language, a logic programming language, or an assembly language or machine language. The term "computer-readable storage medium" refers to any computer program product, apparatus or device for providing machine instructions or data to a programmable data processor, such as magnetic disks, optical disks, solid-state storage devices, memories and programmable logic devices (PLDs), including computer-readable storage media that receive machine instructions as computer-readable signals.

[0078] For example, computer readable storage media may include dynamic random access memory (DRAM), random access memory (RAM), read-only memory (ROM), electrically erasable read-only memory (EEPROM), compact disk read-only memory (CD-ROM) or other optical disk storage devices, magnetic disk storage devices or other magnetic storage devices, or any other medium that can be used to carry or store the desired computer readable program code in the form of instructions or data structures and can be accessed by a general or special-purpose computer or a general or special-purpose processor. As used in the present invention, disks or discs include compact disks (CDs), laser discs, optical discs, digital versatile discs (DVDs), floppy disks, and Blu-ray discs, where disks typically reproduce data magnetically, while discs reproduce data optically by lasers. Combinations of the above are also included in the scope of computer readable storage media.

[0079] The subject matter of the present invention is provided as an example of a method, system, and computer-readable storage medium for performing the features described in the present invention. However, in addition to the above features, other features or variations may also be expected. It is expected that any emerging technology that may replace any of the above-mentioned technologies can be used to complete the implementation of the components and functions of the present invention.

[0080] In addition, the above description provides examples without limiting the scope, applicability or configuration set forth in the claims. Without departing from the spirit and scope of the present invention, the functions and arrangements of the elements discussed may be changed. Various embodiments may appropriately omit, replace or add various processes or components. For example, features described with respect to certain embodiments may be combined in other embodiments.

[0081] Similarly, while operations are depicted in a particular order in the drawings, this should not be understood as requiring that such operations be performed in the particular order shown or in sequential order, or that all illustrated operations be performed to achieve desired results. In certain circumstances, multitasking and parallel processing can be advantageous.

Claims

1. A speech recognition method, comprising: A voice stream processing step, receiving a voice stream and dividing the voice stream into voice frames; A voice frame processing step, performing a silence judgment on the voice frame; as well as A speech recognition step, exchanging messages with a speech recognition engine according to the result of the mute judgment; A recognition result processing step, after obtaining the recognition result from the speech recognition engine, calculating the time position of the recognition result in the original speech stream; The speech frame includes a first frame, a middle frame, and a last frame. The first frame is subjected to the following speech frame processing: If the first frame is a silent frame, the silent time is accumulated; If the first frame is a non-silent frame, it is determined whether the speech recognition engine has spare recognition capacity. If so, the speech recognition is performed; if the recognition capacity is full, the speech frame is cached or discarded according to the cache strategy. The intermediate frame is subjected to the following speech frame processing: If the intermediate frame is a silent frame, determine whether the previous frame of speech is a silent frame. If the previous frame of speech is silent, accumulate the silent time. If the previous frame of speech is not silent, end the speech recognition, obtain the final recognition result, and calculate the time position of the recognition result. If the intermediate frame is a non-silent frame, determine whether the previous frame of voice is a silent frame. If the previous frame of voice is silent, perform the same processing as the case where the first frame is a non-silent frame; if the previous frame of voice is non-silent, continue caching or continue the voice recognition according to the session situation.

2. The speech recognition method according to claim 1, wherein: In the recognition result processing step, the time position of the recognition result in the original voice stream is calculated according to the time position returned by the voice recognition engine and the stored silence duration information.

3. The speech recognition method according to claim 2, wherein: In the speech recognition step, exchanging messages with the speech recognition engine includes creating a new speech recognition session, sending a speech frame to be recognized, obtaining a recognition result, and ending the speech recognition session.

4. The speech recognition method according to claim 3, wherein: The newly created session information of the speech recognition session includes: a session identifier, a speech identifier, call path information of the speech, and a processing position of the speech.

5. The speech recognition method according to claim 1, wherein: In the speech frame processing step, silence determination is performed on the speech frame through speech endpoint detection.

6. The speech recognition method according to claim 5, wherein: If the speech frame is not a silent frame and the previous speech frame is a silent frame, a new speech recognition session is created. If the speech frame is not a silent frame and the previous speech frame is not a silent frame, continue the speech recognition; If the speech frame is a silent frame and the previous speech frame is not a silent frame, the speech recognition session is terminated. If the speech frame is a silent frame and the previous speech frame is a silent frame, the duration of the silent segment is calculated.

7. The speech recognition method according to claim 4, wherein: When the speech recognition session is newly established, the current concurrency number of the speech recognition engine is increased, and when the speech recognition session is ended, the current concurrency number of the speech recognition engine is decreased.

8. The speech recognition method according to claim 7, wherein: If the concurrency number of the speech recognition engine reaches the upper limit, the new speech frames are cached and the earlier speech frames are discarded.

9. The speech recognition method according to claim 8, wherein: The early speech frames exceeding the maximum duration of the speech frame buffer are discarded.

10. The speech recognition method according to claim 1, wherein: The last frame is subjected to the following speech frame processing: If the previous frame of speech is silent, the cache is cleared; if the previous frame of speech is not silent, the speech recognition is terminated, the last recognition result is obtained, and the time position of the recognition result is calculated.

11. A speech recognition system, comprising: A voice stream processing module receives a voice stream and divides the voice stream into voice frames; A voice frame processing module performs silence judgment on the voice frame; as well as A speech recognition module, which interacts with a speech recognition engine according to the result of the mute judgment; After the speech recognition system obtains the recognition result from the speech recognition engine, it calculates the time position of the recognition result in the original speech stream; The speech frame includes a first frame, a middle frame, and a last frame. The first frame is subjected to the following speech frame processing: If the first frame is a silent frame, the silent time is accumulated; If the first frame is a non-silent frame, it is determined whether the speech recognition engine has spare recognition capacity. If so, the speech recognition is performed; if the recognition capacity is full, the speech frame is cached or discarded according to the cache strategy. The intermediate frame is subjected to the following speech frame processing: If the intermediate frame is a silent frame, determine whether the previous frame of speech is a silent frame. If the previous frame of speech is silent, accumulate the silent time. If the previous frame of speech is not silent, end the speech recognition, obtain the final recognition result, and calculate the time position of the recognition result. If the intermediate frame is a non-silent frame, determine whether the previous frame of voice is a silent frame. If the previous frame of voice is silent, perform the same processing as the case where the first frame is a non-silent frame; if the previous frame of voice is non-silent, continue caching or continue the voice recognition according to the session situation.

12. A speech recognition system, comprising: a memory having instructions stored therein; as well as A processor is configured to execute instructions stored in the memory to perform the speech recognition method according to any one of claims 1 to 10.

13. A computer-readable storage medium comprising computer-executable instructions, which, when executed by one or more processors, cause the one or more processors to perform the speech recognition method according to any one of claims 1 to 10.

Images