A speech recognition processing method, system, device and storage medium

Abstract

The present application relates to a kind of speech recognition processing method, system, device and storage medium, it is related to cache speech processing field.The present application carries out offline speech recognition to voice information, obtains command while sending voice information to cloud speech recognition server and carries out online speech recognition to obtain the text form of voice information, and text form is sent to semantic understanding server and carries out semantic analysis and understanding to obtain command;According to network off-line situation, in the process of online speech processing, stage one weak network signal, stage one stop interaction signal, stage two weak network signal, stage two stop interaction signal, receive online command and offline command The situation is arbitrated to control the final command obtained in the process of online speech processing and offline speech processing.Dynamically real-time decision online speech recognition semantic processing and offline speech recognition semantic understanding switching, cooperation, both guarantee recognition accuracy, also improve the speech recognition processing efficiency under weak network.

Description

Technical Field

[0001] This invention relates to the field of speech recognition processing technology, and in particular to a speech recognition processing method, system, device and storage medium. Background Technology

[0002] With the continuous development of IoT, AI, and voice technologies, voice recognition has become a key technology for the intelligence of terminal devices. Traditional voice recognition processing methods are divided into two types: the first involves inputting voice data into a cloud server for online voice recognition and semantic understanding; the second involves using a local recognition engine for offline voice recognition. Both methods have their drawbacks. The first method cannot achieve voice control of devices when not connected to the internet; the second method, due to memory limitations, cannot achieve the flexibility of voice commands and can only recognize a limited number of offline words, resulting in low applicability and failing to meet the needs of intelligent chat communication. To avoid the shortcomings of both methods, existing technologies select between online and offline voice recognition based on network connectivity, such as 202111659197.6. However, for intelligent terminal devices such as cars and home appliances, unstable or congested networks can easily occur, resulting in weak network environments. In such cases, voice response times are very long, and even unresponsiveness can occur when network failures occur between the device and the server. How to enable devices to use different speech recognition methods under different network conditions, including good network, no network, and weak network scenarios, is an urgent problem to be solved, so as to improve the accuracy and response speed of speech recognition in dynamically changing network scenarios. Summary of the Invention

[0003] To solve the above-mentioned technical problems, or at least partially solve them, the present invention provides a speech recognition processing method, system, device, and storage medium.

[0004] In a first aspect, the present invention provides a speech recognition processing method, comprising: acquiring speech information; performing offline speech recognition and semantic understanding on the speech information to obtain a command, while simultaneously sending the speech information to a cloud speech recognition server for online speech recognition to obtain a text form of the speech information, and sending the text form to a semantic understanding server for semantic analysis and understanding to obtain a command; arbitrating the reception of online and offline commands based on network conditions, including stage one weak network signal, stage one stop interaction signal, stage two weak network signal, and stage two stop interaction signal during online speech processing, in order to control the online and offline speech processing processes to obtain the final command.

[0005] Furthermore, based on network online / offline status, during the online voice processing process, arbitration is performed on the reception of online and offline commands, considering the weak network signal in Phase 1, the stop interaction signal in Phase 1, the weak network signal in Phase 2, and the stop interaction signal in Phase 2, to control the online and offline voice processing processes to obtain the final commands, including:

[0006] If the command output receives an offline command, and the network is online, the offline command is temporarily stored; if the network is offline, it continues to determine whether it is in a wake-up state or VAD processing state. If it is, the offline command is executed; otherwise, the current state is maintained.

[0007] If the command output receives a weak network signal in phase one, it determines whether an offline command has been received. If so, the offline command is executed and the system enters an idle state; otherwise, the current state is maintained.

[0008] If the command output receives a Phase 1 interaction stop signal, it determines whether it is in online speech recognition state. If so, it enters idle state; otherwise, it maintains the current state.

[0009] If the command output receives an online command, it determines whether it is in an online semantic understanding state. If so, it executes the online command and then enters an idle state.

[0010] If the command output receives a weak network signal in stage two, it determines whether an offline command has been received. If so, the offline command is executed and the system enters an idle state; otherwise, the current state is maintained.

[0011] If the command output receives a Phase 2 interaction stop signal, it determines whether it is in an online semantic understanding state. If so, it enters an idle state; otherwise, it maintains the current state.

[0012] Furthermore, if the command output is in the form of text received from online speech recognition, the command can be obtained through offline speech understanding using the text of the speech information, and then replaced or obtained as an offline command.

[0013] Furthermore, a network flag is preset to determine the network's online / offline status, and an offline identification flag is preset to determine the generation status of offline commands. The system checks whether the network is online; if it is online, the network flag is set to 1, otherwise it is set to 0. The system checks whether an offline command is obtained from offline semantic understanding of the text obtained from offline speech recognition or online speech recognition; if so, the offline identification flag is set to 1, otherwise it is set to 0.

[0014] Furthermore, when performing online speech recognition, the system detects whether the network request time for online speech recognition initiated to the cloud speech recognition server is less than the second request time threshold and greater than or equal to the first request time threshold. If so, a weak network signal for stage one is generated.

[0015] When performing online speech recognition, the system detects whether the network request time for online speech recognition initiated to the cloud speech recognition server is greater than or equal to the second request time threshold. If so, a phase one interaction stop signal is generated.

[0016] When performing online semantic understanding, the network request time for online semantic analysis and understanding initiated to the semantic understanding server is detected to be less than the second request time threshold and greater than or equal to the first request time threshold. If so, a stage two weak network signal is generated.

[0017] When performing online semantic understanding, the system detects whether the network request time for online semantic analysis and understanding initiated to the semantic understanding server is greater than or equal to the second request time threshold. If so, a phase two interaction stop signal is generated.

[0018] Furthermore, the wake-up state, VAD processing state, online speech recognition state, online semantic understanding state, and idle state are derived from a finite state machine, which includes:

[0019] The device is in the initial state when it is first powered on, and then initialization is performed.

[0020] Once the device initialization is complete, it transitions from the initial state to the idle state.

[0021] After a user wakes up the device via a wake-up event, the device transitions from an idle state to a wake-up state.

[0022] Upon entering the wake-up state, voice triggering enters the VAD processing state. The VAD processing state determines the start and end of the voice information and times the voice process. When the VAD detects the end of the voice information, it enters both online speech recognition and offline speech recognition / understanding states. In the online speech recognition state, the voice information is sent to a cloud-based speech recognition server for online recognition. Once the text form of the voice information is obtained, it is sent to the command output while simultaneously entering the online semantic understanding state. In this state, the text form of the voice information is sent to a semantic understanding server to obtain online commands for semantic analysis and understanding, which are then sent to the command output. In the offline speech recognition / understanding state, the voice information is used for speech recognition and semantic understanding to obtain offline commands, which are then sent to the command output. The semantic understanding process in the offline speech recognition / understanding state supports analyzing and understanding the text of the voice information obtained in the online speech recognition state to obtain offline commands.

[0023] The command output status is arbitrated based on the network status (online / offline), during online voice processing (phase one weak network signal, phase one stop interaction signal, phase two weak network signal, phase two stop interaction signal), and the received online and offline commands, in order to control the online and offline voice processing processes to obtain the final command.

[0024] Furthermore, if the VAD detects that the start time of the human voice exceeds the first threshold, it assumes that no human voice was heard after waking up and enters an idle state. If the VAD detects that the end time of the human voice exceeds the second threshold, it assumes that the VAD has made a processing error and enters an idle state.

[0025] In a second aspect, the present invention provides a speech recognition processing system, comprising: a speech input module, an online speech recognition module, an online semantic understanding module, a network status module, an offline speech recognition and understanding module, and a command output module; wherein,

[0026] The voice input module is used to acquire the user's voice information, including wake-up function and VAD detection function;

[0027] The offline speech recognition and understanding module is used to recognize and understand speech information offline or to understand the text of the speech information provided by the online speech recognition module through the command output module to obtain offline commands.

[0028] The online speech recognition module is used to send speech information to the cloud speech recognition server to obtain the text form of the speech information;

[0029] The online semantic understanding module is used to send the text form of the voice information to the semantic understanding server for semantic analysis and understanding to obtain online commands;

[0030] The network status module generates a phase one weak network signal, a phase one stop interaction signal, a phase two weak network signal, and a phase two stop interaction signal based on the network data request time of online speech recognition and online semantic understanding.

[0031] The command output module is used to arbitrate the online and offline commands received during online voice processing based on network conditions, including weak network signal in phase one, stop interaction signal in phase one, weak network signal in phase two, and stop interaction signal in phase two, in order to control the online and offline voice processing processes to obtain the final command.

[0032] Thirdly, the present invention provides a speech recognition processing apparatus, comprising: at least one processing unit, the processing unit being connected to a storage unit via a bus unit, the storage unit storing a computer program, and the computer program being executed by the processing unit to implement the speech recognition processing method.

[0033] Fourthly, the present invention provides a computer-readable storage medium storing a computer program, characterized in that, when the computer program is executed by a processor, it implements the speech recognition processing method.

[0034] The technical solutions provided in the embodiments of the present invention have the following advantages compared with the prior art:

[0035] This invention performs offline speech recognition and semantic understanding to obtain commands, while simultaneously sending the speech information to a cloud-based speech recognition server for online speech recognition to obtain the text form of the speech information. The text form is then sent to a semantic understanding server for semantic analysis and understanding to obtain commands. Based on network online / offline conditions, during online speech processing, arbitration is performed on the received online and offline commands to control the online and offline speech processing processes and obtain the final command. By dynamically and in real-time deciding on the switching and coordination between online speech recognition and semantic processing and offline speech recognition and semantic understanding based on network online / offline conditions, different network weakness conditions, and the interruption of interaction, both recognition accuracy and efficiency under weak network conditions are ensured.

[0036] This application supports offline semantic understanding to obtain offline commands by using text obtained from online speech recognition. While ensuring timeliness, the accuracy of offline commands is guaranteed, thus ensuring the overall recognition accuracy.

[0037] This application uses a finite state machine to control the orderly execution of the speech recognition processing method, thereby avoiding conflicts. Attached Figure Description

[0038] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.

[0039] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0040] Figure 1 This is a schematic diagram of a finite state machine used to implement a speech recognition processing method according to an embodiment of the present invention;

[0041] Figure 2 A flowchart of a speech recognition processing method provided in an embodiment of the present invention;

[0042] Figure 3The flowchart provided in this embodiment of the invention arbitrates the reception of online and offline commands based on the network status during online voice processing, including stage one weak network signal, stage one stop interaction signal, stage two weak network signal, and stage two stop interaction signal, in order to control the online and offline voice processing processes to obtain the final command.

[0043] Figure 4 A flowchart illustrating the generation of the Phase 1 weak network signal, the Phase 1 stop interaction signal, the Phase 2 weak network signal, and the Phase 2 stop interaction signal, provided in an embodiment of the present invention.

[0044] Figure 5 This is a schematic diagram of a speech recognition processing system provided in an embodiment of the present invention;

[0045] Figure 6 This is a schematic diagram of a speech recognition processing device provided in an embodiment of the present invention. Detailed Implementation

[0046] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0047] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0048] Example 1

[0049] This invention provides a speech recognition processing method. To better execute the speech recognition processing method, this invention constructs a finite state machine to implement the speech recognition processing method. (See attached document.) Figure 1 As shown, the finite state machine includes:

[0050] Initially, the speech recognition processing device is in this initial state when it is first powered on, and then initializes. After initialization, the microphone of the speech recognition device is turned on, the processor runs, and the finite state machine transitions from the initial state to the idle state. When the user wakes up the device via a wake-up event, it transitions from the idle state to the wake-up state; in specific implementations, the speech processing state is also triggered by a wake-up event to enter the wake-up state, and upon entering the wake-up state, the offline recognition flag is reset to 0. Upon entering the wake-up state, a wake-up prompt tone is played.

[0051] Upon entering the wake-up state, voice triggering enters the VAD processing state. The VAD processing state determines the start and end of the voice information and times the voice process. Specifically, in the VAD processing state, the start of human voice is detected within the sound intensity range, thus determining the start of the voice information, and the entire voice recording process is timed. If the VAD detects that the start time of human voice exceeds a first threshold, it considers that no human voice was heard after wake-up and enters an idle state. If the VAD detects that the end time of human voice exceeds a second threshold, it considers that the VAD processing has failed and enters an idle state.

[0052] After VAD detects the end of the voice information, it enters two states: online speech recognition and offline speech recognition and understanding. In the online speech recognition state, the voice information is sent to a cloud-based speech recognition server for online recognition to obtain the text form of the voice information. The online speech recognition state times the network request for online speech recognition. Simultaneously, the text is sent to the command output, and the system enters the online semantic understanding state. In the online semantic understanding state, the text form of the voice information is sent to a semantic understanding server to obtain a semantically analyzed and understood online command. The online semantic understanding state also times the network request for online semantic understanding. The online command is then sent to the command output. In the offline speech recognition and understanding state, the voice information is used for speech recognition and semantic understanding to obtain an offline command, which is then sent to the command output. The semantic understanding process in the offline speech recognition and understanding state supports analyzing and understanding the text of the voice information obtained in the online speech recognition state to obtain the offline command. When generating the offline command, the offline recognition flag is set to 1.

[0053] In the speech recognition processing method provided in this application, the command output arbitrates the received online and offline commands based on network online / offline conditions, including weak network signals in stage one, stop interaction signals in stage one, weak network signals in stage two, and stop interaction signals in stage two during online speech processing, to control the online and offline speech processing processes to obtain the final command. For details, please refer to... Figure 2 As shown, a speech recognition processing method includes:

[0054] Acquire voice information; receive user voice through the microphone of the voice recognition processing device and acquire voice information through the VAD function.

[0055] Offline speech recognition and semantic understanding are performed on voice information to obtain offline commands. In one embodiment, the corresponding offline recognized text is searched in the local text database of the speech recognition processing device, and the offline recognized text is parsed to obtain the offline command. In another embodiment, due to limitations in the storage and processor of the speech recognition processing device, a miniaturized speech recognition and understanding model is deployed, and local offline speech recognition and semantic understanding are performed using the miniaturized speech recognition and understanding model.

[0056] The voice information is sent to a cloud-based speech recognition server for online speech recognition, resulting in the text form of the voice information.

[0057] The text is sent to a semantic understanding server for semantic analysis and understanding to obtain online commands.

[0058] Based on the network status (online / offline), during the online voice processing, arbitration is performed on the reception of online and offline commands, considering the weak network signal in Phase 1, the stop interaction signal in Phase 1, the weak network signal in Phase 2, and the stop interaction signal in Phase 2, in order to control the online and offline voice processing processes to obtain the final command.

[0059] In the specific implementation process, please refer to Figure 3 As shown, based on network online / offline status, during the online voice processing process, the reception of online and offline commands is arbitrated according to the weak network signal in Phase 1, the stop interaction signal in Phase 1, the weak network signal in Phase 2, and the stop interaction signal in Phase 2, in order to control the online and offline voice processing processes to obtain the final command, which includes:

[0060] The system presets a network flag to determine network online / offline status and an offline recognition flag to determine offline command generation status. It checks whether the network is online; if online, it sets the network flag to 1, otherwise it sets the network flag to 0. It checks whether an offline command is obtained based on offline speech recognition or offline semantic understanding of text obtained from online speech recognition; if so, it sets the offline recognition flag to 1, otherwise it sets the offline recognition flag to 0. When entering the wake-up state, the offline recognition flag is set to 0.

[0061] When an offline command is received in the command output, if the network is online (e.g., the network flag is 1), the offline command is temporarily stored. However, due to limitations in the performance of the speech recognition processing device, offline commands obtained from offline speech recognition are less effective than online commands obtained from online speech recognition semantic understanding. If the network is online, the offline command is temporarily stored, and an attempt is made to retrieve an online command. If the network is offline, the system continues to check if the device is in a wake-up state or VAD processing state. If so, the offline command is executed; otherwise, the current state is maintained. If the network is offline and an online command cannot be retrieved, the offline command is executed.

[0062] Perform online speech recognition when the network is online:

[0063] During online speech recognition, if the network is normal, the command output is the text form of the received speech information. Offline semantic parsing is then performed on this text to obtain an offline command, which replaces the previously stored offline command. Because online speech recognition is more effective, offline semantic parsing is performed on the text obtained from online speech recognition to obtain a more accurate offline command.

[0064] During online speech recognition, if a weak network signal is received at stage one during command output, it is determined whether an offline command has been received. If so, the offline command is executed, and then the system enters an idle state; otherwise, the current state is maintained. For example, Figure 4 As shown, during online speech recognition, the system checks whether the network request time for online speech recognition initiated to the cloud speech recognition server is less than the second request time threshold and greater than or equal to the first request time threshold. If so, a Stage 1 weak network signal is generated. The generation of the Stage 1 weak signal indicates that online speech recognition has not been achieved after the first request time threshold has been exceeded. To ensure efficiency and avoid lag, the offline command is executed immediately if it is ready.

[0065] During online speech recognition, if the command output receives a Phase 1 interaction stop signal, it determines whether it is in online speech recognition mode. If so, it enters idle mode; otherwise, it maintains the current mode. For example, Figure 4 As shown, during online speech recognition, the system checks whether the network request time initiated to the cloud speech recognition server is greater than or equal to the second request time threshold. If so, a Phase 1 interaction stop signal is generated. The generation of the Phase 1 interaction stop signal indicates that a Phase 1 weak network signal has occurred. If no offline command is generated during the period from the Phase 1 weak network signal to the Phase 1 interaction stop signal, time is given to compete between generating offline and online commands. If the second request time threshold is exceeded, online speech recognition stops and enters an idle state. If an offline command is generated, the online command generation process ends to avoid repeated execution.

[0066] During online speech recognition and semantic understanding, if the network is normal, the command output receives online commands, then determines whether it is in online semantic understanding state. If so, it executes the online command and then enters idle state.

[0067] During online semantic understanding, if the command output receives a weak network signal in stage two, it checks whether an offline command has been received. If so, the offline command is executed, and the system enters an idle state; otherwise, the current state is maintained. For example, Figure 4As shown, during online semantic understanding, the system detects whether the network request time for online semantic analysis and understanding initiated to the semantic understanding server is less than the second request time threshold and greater than or equal to the first request time threshold. If so, a stage two weak network signal is generated. The stage two weak network signal indicates that online semantic understanding has not been achieved after the first request time threshold has been exceeded. To ensure efficiency and avoid lag, the offline command is executed immediately if it is ready.

[0068] During online semantic understanding, if the command output receives a phase two interaction stop signal, it determines whether it is in an online semantic understanding state. If so, it enters an idle state; otherwise, it maintains the current state. For example, Figure 4 As shown, during online semantic understanding, the system detects whether the network request time for online semantic analysis and understanding initiated to the semantic understanding server is greater than or equal to the second request time threshold. If so, a phase two interaction stop signal is generated. The generation of the phase two interaction stop signal indicates that a phase two weak network signal has occurred. If no offline command is generated during the period from the phase two weak network signal to the phase two interaction stop signal, time is given to compete between generating offline and online commands. If the second request time threshold is exceeded, online speech recognition stops and enters an idle state. If an offline command is generated, the online command generation process ends to avoid repeated execution.

[0069] In some examples, the speech recognition processing device using the method of this application serves as the control center for home appliances. For instance, if the command output module of the speech recognition processing device ultimately obtains an air conditioner control command, it sends the command to the air conditioner to execute the air conditioner control; if the command output module of the speech recognition processing device obtains a voice broadcast command, it sends the broadcast content to the speech synthesis module for playback and execution.

[0070] Example 2

[0071] See Figure 5 As shown, this embodiment of the invention provides a speech recognition processing system, including: a speech input module, an online speech recognition module, an online semantic understanding module, a network status module, an offline speech recognition and understanding module, and a command output module; wherein,

[0072] The voice input module is used to acquire the user's voice information, including wake-up function and VAD detection function;

[0073] The offline speech recognition and understanding module is used to recognize and understand speech information offline or to understand the text of the speech information provided by the online speech recognition module through the command output module to obtain offline commands.

[0074] The online speech recognition module is used to send speech information to the cloud speech recognition server to obtain the text form of the speech information;

[0075] The online semantic understanding module is used to send the text form of the voice information to the semantic understanding server for semantic analysis and understanding to obtain online commands;

[0076] The network status module generates a phase one weak network signal, a phase one stop interaction signal, a phase two weak network signal, and a phase two stop interaction signal based on the network data request time of online speech recognition and online semantic understanding.

[0077] The command output module is used to arbitrate the online and offline commands received during online voice processing based on network conditions, including weak network signal in phase one, stop interaction signal in phase one, weak network signal in phase two, and stop interaction signal in phase two, in order to control the online and offline voice processing processes to obtain the final command. Specifically: If the command output module receives an offline command, and the network is online, the offline command is temporarily stored; if the network is offline, it continues to determine whether it is in a wake-up state or VAD processing state. If so, the offline command is executed; otherwise, the current state is maintained. If the command output receives the text form of the voice information obtained from online speech recognition, it uses the text of the voice information for offline speech understanding to obtain the command, which is then used to replace or obtain the offline command. If the command output module receives a weak network signal in Phase 1, it determines whether an offline command has been received. If so, the offline command is executed, and then the module enters an idle state; otherwise, the current state is maintained. If the command output module receives an interaction stop signal in Phase 1, it determines whether it is in an online speech recognition state. If so, the module enters an idle state; otherwise, the current state is maintained. If the command output module receives an online command, it determines whether it is in an online semantic understanding state. If so, the online command is executed, and then the module enters an idle state. If the command output module receives a weak network signal in Phase 2, it determines whether an offline command has been received. If so, the offline command is executed, and then the module enters an idle state; otherwise, the current state is maintained. If the command output module receives an interaction stop signal in Phase 2, it determines whether it is in an online semantic understanding state. If so, the module enters an idle state; otherwise, the current state is maintained.

[0078] Example 3

[0079] See Figure 6As shown, this embodiment of the invention provides a speech recognition processing device, including: at least one processing unit, the processing unit being connected to a storage unit via a bus unit, the storage unit serving as a computer-readable storage medium, which can be used to store software programs, computer-executable programs, and modules, such as the software program, computer-executable program, and modules (speech input module, online speech recognition module, online semantic understanding module, network status module, offline speech recognition and understanding module, and command output module) corresponding to a speech recognition processing method in this embodiment of the invention. The processing unit implements the aforementioned speech recognition processing method by running the software program, computer-executable program, and modules stored in the storage unit, including:

[0080] Acquire voice information; perform offline speech recognition and semantic understanding to obtain commands, and simultaneously send the voice information to a cloud speech recognition server for online speech recognition to obtain the text form of the voice information, and send the text form to a semantic understanding server for semantic analysis and understanding to obtain commands; based on the network status, during the online voice processing, arbitration is performed on the reception of online and offline commands in stages one (weak network signal), stage one (stop interaction signal), stage two (weak network signal), and stage two (stop interaction signal) to control the online and offline voice processing processes to obtain the final command.

[0081] Furthermore, the storage unit may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other non-volatile solid-state storage device. In some instances, the storage unit may further include memory remotely located relative to the processing unit, and these remote memories can be connected to the electronic device via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0082] Of course, the computer program stored in the storage unit of the speech recognition processing device provided in the embodiments of the present invention is not limited to the method operation described above, but can also execute related operations in a speech recognition processing method provided in any embodiment of the present invention.

[0083] Example 4

[0084] This invention provides a computer-readable storage medium storing a computer program. When the computer program is executed, it implements the speech recognition processing method, the method comprising:

[0085] Acquire voice information; perform offline speech recognition and semantic understanding to obtain commands, and simultaneously send the voice information to a cloud speech recognition server for online speech recognition to obtain the text form of the voice information, and send the text form to a semantic understanding server for semantic analysis and understanding to obtain commands; based on the network status, during the online voice processing, arbitration is performed on the reception of online and offline commands in stages one (weak network signal), stage one (stop interaction signal), stage two (weak network signal), and stage two (stop interaction signal) to control the online and offline voice processing processes to obtain the final command.

[0086] Of course, the computer program stored in the computer-readable storage medium provided in the embodiments of the present invention is not limited to the method operation described above, but can also execute related operations in a speech recognition processing method provided in any embodiment of the present invention.

[0087] In the embodiments provided by this invention, it should be understood that the disclosed structures and methods can be implemented in other ways. For example, the structural embodiments described above are merely illustrative. For instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection through some interfaces, structures, or units, and may be electrical, mechanical, or other forms.

[0088] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0089] Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0090] The above description is merely a specific embodiment of the present invention, enabling those skilled in the art to understand or implement the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the present invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features claimed herein.

Claims

1. A speech recognition processing method, characterized in that, include: Acquire voice information; perform offline speech recognition and semantic understanding to obtain commands, and send the voice information to the cloud speech recognition server for online speech recognition to obtain the text form of the voice information, and send the text form to the semantic understanding server for semantic analysis and understanding to obtain commands; Based on network online / offline status, during online voice processing, arbitration is performed on the reception of online and offline commands, considering the weak network signal in Phase 1, the stop interaction signal in Phase 1, the weak network signal in Phase 2, and the stop interaction signal in Phase 2, to control the online and offline voice processing processes to obtain the final command, including: If the command output receives an offline command, and the network is online, the offline command is temporarily stored; if the network is offline, it continues to determine whether it is in a wake-up state or VAD processing state. If it is, the offline command is executed; otherwise, the current state is maintained. If the command output receives a weak network signal in phase one, it determines whether an offline command has been received. If so, the offline command is executed and the system enters an idle state; otherwise, the current state is maintained. If the command output receives a Phase 1 interaction stop signal, it determines whether it is in online speech recognition state. If so, it enters idle state; otherwise, it maintains the current state. If the command output receives an online command, it determines whether it is in an online semantic understanding state. If so, it executes the online command and then enters an idle state. If the command output receives a weak network signal in stage two, it determines whether an offline command has been received. If so, the offline command is executed and the system enters an idle state; otherwise, the current state is maintained. If the command output receives the Phase 2 interaction stop signal, it determines whether it is in the online semantic understanding state. If so, it enters the idle state; otherwise, it maintains the current state. Specifically, during online speech recognition, the system detects whether the network request time for online speech recognition initiated to the cloud speech recognition server is less than a second request time threshold and greater than or equal to a first request time threshold; if so, a stage one weak network signal is generated. Similarly, during online speech recognition, the system detects whether the network request time for online speech recognition initiated to the cloud speech recognition server is greater than or equal to a second request time threshold; if so, a stage one interaction stop signal is generated. During online semantic understanding, the system detects whether the network request time for online semantic analysis and understanding initiated to the semantic understanding server is less than a second request time threshold and greater than or equal to a first request time threshold; if so, a stage two weak network signal is generated. Finally, during online semantic understanding, the system detects whether the network request time for online semantic analysis and understanding initiated to the semantic understanding server is greater than or equal to a second request time threshold; if so, a stage two interaction stop signal is generated.

2. The speech recognition processing method according to claim 1, characterized in that, If the command output is the text form of the voice information obtained from online speech recognition, the command is obtained by offline speech understanding using the text of the voice information, and the command is replaced or obtained offline.

3. The speech recognition processing method according to claim 1, characterized in that, The system presets a network flag to determine network online / offline status and an offline identification flag to determine offline command generation status. It checks whether the network is online; if online, it sets the network flag to 1, otherwise it sets the network flag to 0. It also checks whether an offline command is obtained from offline semantic understanding of text obtained from offline speech recognition or online speech recognition; if so, it sets the offline identification flag to 1, otherwise it sets the offline identification flag to 0.

4. The speech recognition processing method according to claim 1, characterized in that, The wake-up state, VAD processing state, online speech recognition state, online semantic understanding state, and idle state are derived from a finite state machine, which includes: The device is in the initial state when it is first powered on, and then initialization is performed. Once the device initialization is complete, it transitions from the initial state to the idle state. After a user wakes up the device via a wake-up event, the device transitions from an idle state to a wake-up state. Upon entering the wake-up state, voice triggering enters the VAD processing state. The VAD processing state determines the start and end of the voice information and times the voice process. When the VAD detects the end of the voice information, it enters both online speech recognition and offline speech recognition / understanding states. In the online speech recognition state, the voice information is sent to a cloud-based speech recognition server for online recognition. Once the text form of the voice information is obtained, it is sent to the command output while simultaneously entering the online semantic understanding state. In this state, the text form of the voice information is sent to a semantic understanding server to obtain online commands for semantic analysis and understanding, which are then sent to the command output. In the offline speech recognition / understanding state, the voice information is used for speech recognition and semantic understanding to obtain offline commands, which are then sent to the command output. The semantic understanding process in the offline speech recognition / understanding state supports analyzing and understanding the text of the voice information obtained in the online speech recognition state to obtain offline commands. The command output status is arbitrated based on the network status (online / offline), during online voice processing (phase one weak network signal, phase one stop interaction signal, phase two weak network signal, phase two stop interaction signal), and the received online and offline commands, in order to control the online and offline voice processing processes to obtain the final command.

5. The speech recognition processing method according to claim 4, characterized in that, If the VAD detects that the start time of a human voice exceeds the first threshold, it assumes that no human voice was heard after waking up and enters an idle state. If the VAD detects that the end time of a human voice exceeds the second threshold, it assumes that the VAD has made a processing error and enters an idle state.

6. A speech recognition processing system, characterized in that, include: The system includes a voice input module, an online speech recognition module, an online semantic understanding module, a network status module, an offline speech recognition and understanding module, and a command output module; among which, The voice input module is used to acquire the user's voice information, including wake-up function and VAD detection function; The offline speech recognition and understanding module is used to recognize and understand speech information offline or to understand the text of the speech information provided by the online speech recognition module through the command output module to obtain offline commands. The online speech recognition module is used to send speech information to the cloud speech recognition server to obtain the text form of the speech information; The online semantic understanding module is used to send the text form of the voice information to the semantic understanding server for semantic analysis and understanding to obtain online commands; The network status module generates a Phase 1 weak network signal, a Phase 1 stop-interaction signal, a Phase 2 weak network signal, and a Phase 2 stop-interaction signal based on the network data request time for online speech recognition and online semantic understanding. Specifically, during online speech recognition, it checks whether the network request time initiated to the cloud speech recognition server is less than a second request time threshold and greater than or equal to a first request time threshold; if so, a Phase 1 weak network signal is generated. Similarly, during online speech recognition, it checks whether the network request time initiated to the cloud speech recognition server is greater than or equal to a second request time threshold; if so, a Phase 1 stop-interaction signal is generated. During online semantic understanding, it checks whether the network request time initiated to the semantic understanding server for online semantic analysis and understanding is less than a second request time threshold and greater than or equal to a first request time threshold; if so, a Phase 2 weak network signal is generated. Finally, during online semantic understanding, it checks whether the network request time initiated to the semantic understanding server for online semantic analysis and understanding is greater than or equal to a second request time threshold; if so, a Phase 2 stop-interaction signal is generated. The command output module is used to arbitrate the received online and offline commands based on network online / offline conditions, including weak network signals in stage one, stop interaction signals in stage one, weak network signals in stage two, and stop interaction signals in stage two during online voice processing. This arbitration aims to control the online and offline voice processing processes to obtain the final command, including: If the command output receives an offline command, and the network is online, the offline command is temporarily stored; if the network is offline, it continues to determine whether it is in a wake-up state or VAD processing state. If it is, the offline command is executed; otherwise, the current state is maintained. If the command output receives a weak network signal in phase one, it determines whether an offline command has been received. If so, the offline command is executed and the system enters an idle state; otherwise, the current state is maintained. If the command output receives a Phase 1 interaction stop signal, it determines whether it is in online speech recognition state. If so, it enters idle state; otherwise, it maintains the current state. If the command output receives an online command, it determines whether it is in an online semantic understanding state. If so, it executes the online command and then enters an idle state. If the command output receives a weak network signal in stage two, it determines whether an offline command has been received. If so, the offline command is executed and the system enters an idle state; otherwise, the current state is maintained. If the command output receives a Phase 2 interaction stop signal, it determines whether it is in an online semantic understanding state. If so, it enters an idle state; otherwise, it maintains the current state.

7. A speech recognition processing device, characterized in that, include: At least one processing unit is provided, the processing unit is connected to a storage unit via a bus unit, the storage unit stores a computer program, and when the computer program is executed by the processing unit, it implements the speech recognition processing method as described in any one of claims 1-5.

8. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the speech recognition processing method as described in any one of claims 1-5.

Images