Speech processing method, speech processing device and computer program product

Abstract

The invention provides a voice processing method and device, and the method comprises the steps: receiving a user voice instruction, calling a corresponding acoustic front-end processing engine according to the user voice instruction, carrying out the preprocessing of the user voice instruction, and outputting the preprocessed audio data; calling a corresponding wake-up engine to process the preprocessed audio data according to the preprocessed audio data, and outputting a wake-up signal; responding to the wake-up signal, calling a corresponding voice recognition engine to process the preprocessed audio data according to the preprocessed audio data, and outputting a voice text; according to the voice text, calling a corresponding semantic understanding engine to process the voice text, and outputting semantic understanding information; wherein the semantic comprehension information is used for driving an upper layer service to execute the user voice instruction according to the semantic comprehension information. According to the invention, the overall performance and user experience of the voice system can be improved.

Description

Technical Field

[0001] The present application relates to the field of voice interaction technology, and in particular to a voice processing method, device, and computer program product. Background Art

[0002] With the advancement of vehicle intelligence and electrification, the smart cockpit has become a core competitive advantage and selling point for major automakers. In-vehicle intelligent voice control, as a key method of user interaction, is becoming increasingly important. Users can use voice commands to control vehicle functions, access information, entertainment, and other services, thereby enhancing the driving experience and safety.

[0003] Currently, most car companies usually choose a voice technology company to provide a complete voice solution, which is fully developed and mass-produced by a voice technology company. The algorithm modules in the entire voice chain, from speech recognition, semantic understanding to speech synthesis, are developed and integrated by the same company. The various algorithm modules in the voice chain are deeply coupled and tightly integrated. There is a strong dependency between the various algorithm modules. Optimizing one module may require adjusting other modules.

[0004] Different voice technology companies may have different expertise and strengths in different algorithm modules. For example, some are better at speech recognition, while others are more advantageous in semantic understanding. Furthermore, the performance of each algorithm module may vary in different scenarios. Therefore, a voice system solution with a single voice technology company providing a complete set of voice link products may not meet user and market requirements in terms of overall performance and user experience. Summary of the Invention

[0005] The purpose of this application is to propose a speech processing method and apparatus, and a computer program product thereof, to improve the overall performance of the speech system and user experience.

[0006] To achieve the above objectives, according to a first aspect of the present application, a speech processing method is provided, the method comprising:

[0007] Receive user voice commands, call the corresponding acoustic front-end processing engine according to the user voice commands to pre-process the user voice commands, and output pre-processed audio data;

[0008] Calling a corresponding wake-up engine according to the pre-processed audio data to process the pre-processed audio data and output a wake-up signal;

[0009] In response to the wake-up signal, calling a corresponding speech recognition engine according to the pre-processed audio data to process the pre-processed audio data and output a speech text;

[0010] According to the voice text, a corresponding semantic understanding engine is called to process the voice text and output semantic understanding information; wherein, the semantic understanding information is used to drive upper-layer services to execute the user voice instructions according to the semantic understanding information.

[0011] According to a second aspect of the present application, a speech processing device is provided, comprising a module for executing the above method.

[0012] According to a third aspect of the present application, a computer program product is provided, comprising computer program instructions, wherein the computer program instructions instruct a computer device to perform operations corresponding to the method described in the first aspect.

[0013] The speech processing method, device, and computer program product proposed in this application have the following beneficial effects:

[0014] By decoupling the various algorithm modules in the voice link (such as acoustic front-end processing, wake-up, speech recognition, and semantic understanding), the algorithm modules of the optimal technology suppliers can be statically or dynamically selected according to the performance requirements of different algorithm modules. For example, a speech recognition engine provided by a company that excels in speech recognition can be selected, while a semantic understanding engine provided by another company with advantages in semantic understanding can be selected. Static selection means that the specific implementation of the optimal algorithm modules for each stage has been determined during the system design phase and will not change during the operation of the system. Dynamic selection means that during the operation of the system, an optimal algorithm module is dynamically selected from multiple algorithm modules to adapt to the current voice processing requirements. Through modular design, the entire voice system can integrate the best technologies in various fields, significantly improving the overall performance and user experience of the voice system. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0016] Figure 1 This is a flow chart of a speech processing method in an embodiment of the present application.

[0017] Figure 2 This is a schematic diagram of voice link division and voice interaction in an embodiment of the present application. DETAILED DESCRIPTION

[0018] The detailed description of the accompanying drawings is intended to serve as an illustration of the current embodiment of the present application and is not intended to represent the only form in which the present application can be implemented. It should be understood that the same or equivalent functions can be accomplished by different embodiments intended to be included in the spirit and scope of the present application.

[0019] See Figures 1-2 One embodiment of the present application provides a voice processing method that divides a voice link into multiple modules / links, and calls a suitable engine in each module / environment to process audio-related data. The method includes the following steps:

[0020] Step S10: receiving a user voice command, calling a corresponding acoustic front-end processing engine according to the user voice command to pre-process the user voice command, and outputting pre-processed audio data;

[0021] Specifically, the user's voice commands can be collected through a microphone or other voice input device and then input into the voice system that executes the method of this embodiment; calling the corresponding acoustic front-end processing engine to pre-process the user's voice commands is mainly to improve the quality of the user's audio data to facilitate subsequent recognition processing.

[0022] Step S20, calling a corresponding wake-up engine according to the pre-processed audio data to process the pre-processed audio data and output a wake-up signal;

[0023] Specifically, based on the preprocessed audio data, a specific wake-up word (such as "Hello, Xiao Ming") is detected to determine whether the voice system needs to be activated. The function of the wake-up engine is mainly to extract key features from the audio data and identify whether the key features match the preset wake-up word. If they match, a wake-up signal is output; in addition, since the module algorithms provided by different suppliers may be incompatible, after determining the wake-up engine to be called, if the acoustic front-end processing engine called in step S10 is incompatible with the wake-up engine called in step S20, the preprocessed audio data needs to be converted into a format that the wake-up engine can process. If the acoustic front-end processing engine called in step S10 is compatible with the wake-up engine called in step S20, no format conversion is required, for example, the two engines are provided by the same supplier.

[0024] Step S30, in response to the wake-up signal, calling a corresponding speech recognition engine according to the pre-processed audio data to process the pre-processed audio data and output a speech text;

[0025] Specifically, the speech recognition engine is mainly used to recognize the speech text of the preprocessed audio data, convert the audio signal into phonemes or word sequences, and thus convert the user's voice commands into understandable text form; in addition, since the module algorithms provided by different suppliers may be incompatible, after determining the speech recognition engine to be called, if the speech recognition engine called in step S30 is incompatible with the acoustic front-end processing engine called in step S10 and the wake-up engine called in step S20, it is also necessary to convert the preprocessed audio data and the wake-up signal into a format that can be processed by the speech recognition engine. If the speech recognition engine called in step S30 is compatible with the acoustic front-end processing engine called in step S10 and the wake-up engine called in step S20, no format conversion is required, for example, the two engines are provided by the same supplier.

[0026] Step S40: Call the corresponding semantic understanding engine according to the voice text to process the voice text and output semantic understanding information; wherein the semantic understanding information is used to drive the upper-layer business to execute the user voice command according to the semantic understanding information.

[0027] Specifically, the semantic understanding engine is used to perform semantic analysis on the voice text, understand the user's intention, domain and word slot information. The intention expresses the user's core needs, the domain defines the application scope of the user's instructions, and the word slot information provides the specific parameters required to realize the intention. The user says "turn on the air conditioner and adjust the temperature to 26 degrees", then the intention is to control the air conditioner, the domain is vehicle environment control, and the word slot information is temperature (26 degrees). In addition, since the module algorithms provided by different suppliers may be incompatible, after determining the speech recognition engine to be called, if the semantic understanding engine called in step S40 is incompatible with the speech recognition engine called in step S30, the voice text needs to be converted into a format that can be processed by the speech understanding engine. If the semantic understanding engine called in step S40 is compatible with the speech recognition engine called in step S30, no format conversion is required, for example, if the two engines are provided by the same supplier.

[0028] The method of this embodiment decouples the various algorithm modules in the voice link (such as acoustic front-end processing, wake-up, voice recognition, and semantic understanding). It can statically or dynamically select the algorithm modules (acoustic front-end processing engine, wake-up engine, voice recognition engine, and semantic understanding engine) of the optimal technology supplier according to the performance requirements of different algorithm modules. For example, a voice recognition engine provided by a company that performs well in voice recognition can be selected, while a semantic understanding engine provided by another company with advantages in semantic understanding can be selected. Static selection means that the specific implementation of the optimal algorithm module for each stage has been determined during the system design phase and will not change during the operation of the system. Dynamic selection means that during the operation of the system, an optimal algorithm module is dynamically selected from multiple algorithm modules to adapt to the current voice processing requirements. Through modular design, the entire voice system can integrate the best technologies in various fields, significantly improving the overall performance and user experience of the voice system.

[0029] In some embodiments, the method comprises:

[0030] When the semantic understanding engine obtains the semantic understanding information based on the offline recognition mode, it calls the corresponding dialogue management engine according to the semantic understanding information to process the semantic understanding information, and outputs the semantic understanding information carrying decision information to the upper-layer business, so as to drive the upper-layer business to execute the user voice command according to the semantic understanding information;

[0031] When the semantic understanding engine obtains the semantic understanding information based on the online recognition mode, the semantic understanding information is directly output to the upper-layer business to drive the upper-layer business to execute the user voice instruction according to the semantic understanding information.

[0032] Specifically, in offline recognition mode, the semantic understanding engine runs on the local device and does not rely on a real-time network connection. It converts the received voice text into semantic understanding information, and then calls the dialogue management engine for further processing; the dialogue management engine makes decisions based on the semantic understanding information, and passes the semantic understanding information carrying the decision information to the upper-level business to drive the upper-level business to execute user instructions; for example, the user says to the smart cockpit in the car: "I want to listen to someone's song", the speech recognition engine converts the speech into text: "I want to listen to someone's song", the semantic understanding engine (offline mode) analyzes the text, recognizes the intention as "play music", the domain is "music", and the word slot information is "singer: someone"; the dialogue management engine makes decisions based on the semantic understanding information, for example, determines whether the user is in the music application, whether further confirmation is needed, etc. The dialogue management engine passes the semantic understanding information carrying the decision information (such as "confirm to play someone's song") to the upper-level business, and the upper-level business performs operations based on the decision information, for example, calling the music playback application to play someone's song.

[0033] In online recognition mode, the semantic understanding engine sends the received speech text to the cloud server for processing. After the cloud server returns the semantic understanding information, the system directly passes it to the upper-layer business to drive the upper-layer business to execute the user's instructions. This method relies on a real-time network connection. For example, a user in the car says to the smart cockpit: "What's the weather like in Beijing today?"

[0034] The speech recognition engine converts speech into text: "What's the weather like in Beijing today?" The semantic understanding engine (in online mode) sends the text to the cloud server for processing. The cloud server analyzes the text and identifies the intent as "querying the weather," the domain as "weather," and the word slot information as "city: Beijing." The cloud server returns the semantic understanding information (such as "Today's weather in Beijing is sunny turning cloudy") directly to the device. The device passes the semantic understanding information to the upper-level business, which performs operations based on the semantic understanding information. For example, weather information can be broadcast to the user through speech synthesis technology.

[0035] In some embodiments, the method comprises:

[0036] Receive the reply voice text output by the upper-layer business, call the corresponding synthesis engine according to the reply voice text to process the reply voice text, and output the synthesized audio data to the voice player so that the voice player plays the synthesized audio data.

[0037] Specifically, the reply voice text received from the upper-layer business is usually a response to the user's instructions or information about the execution results. For example, when a user asks about the weather, the reply voice text output by the upper-layer business may be "Today the weather in Beijing is sunny and then cloudy"; based on the received reply voice text, the corresponding speech synthesis engine (Text-to-Speech, TTS) is called. The function of the speech synthesis engine is to convert text information into natural and fluent speech audio data. After the speech synthesis engine completes the processing, it outputs the synthesized audio data, which is sent to the speech player (such as speakers and headphones). The speech player converts the digital audio signal into an analog signal and plays it through a physical device. The user can then hear the voice reply generated by the speech system; for example, the user asks: "What will the weather be like in Beijing tomorrow?" The speech system receives the user's voice command, and the speech recognition engine converts the user's speech into an analog signal. The speech is converted into text: "What will the weather be like in Beijing tomorrow?" The semantic understanding engine analyzes the text and recognizes that the intention is "query weather", the domain is "weather", and the word slot information is "city: Beijing, date: tomorrow". The dialogue management engine makes decisions based on the semantic understanding information and calls the weather information service to obtain tomorrow's weather information. The upper-layer business (such as the weather information service) returns tomorrow's weather information, for example: "The weather in Beijing tomorrow is cloudy". The speech synthesis module receives the reply speech text output by the upper-layer business: "The weather in Beijing tomorrow is cloudy", and calls the speech synthesis engine for processing, converting the text "The weather in Beijing tomorrow is cloudy" into natural and fluent speech audio data. The synthesized audio data is sent to the speaker, and the user hears the voice reply: "The weather in Beijing tomorrow is cloudy".

[0038] In some embodiments, calling a corresponding acoustic front-end processing engine according to the user voice instruction to pre-process the user voice instruction includes:

[0039] In response to the user voice command, the current scene or user preference is obtained, an acoustic front-end processing engine is selected from at least two acoustic front-end processing engines according to the current scene or user preference, and the one acoustic front-end processing engine is called to preprocess the user voice command, wherein the preprocessing includes at least audio noise reduction.

[0040] Specifically, in the in-vehicle voice system, the main purpose of acoustic front-end processing is to pre-process user voice commands to eliminate the ambient noise in the car (such as engine noise, wind noise, road noise, etc.) and improve the accuracy of voice recognition. During the preprocessing process, the system will select the most suitable engine from multiple acoustic front-end processing engines for audio noise reduction based on the current scene or user preferences; when the user issues a voice command in the car, such as "turn on the air conditioner", the system receives the voice signal through the microphone in the car, and the system obtains the current scene (such as vehicle speed, noise level in the car) or user preferences (such as the need for voice clarity) through sensors or user settings. For example, when the vehicle speed is high, the ambient noise is louder; and the user preference may be set to "high definition"; the system selects the most suitable engine from at least two pre-configured acoustic front-end processing engines. For example, acoustic front-end processing engine A may be optimized for wind noise and road noise at high speeds, while acoustic front-end processing engine B may be more suitable for the in-vehicle environment at low speeds or when parked. Based on the current scenario where the vehicle speed is high and the noise is louder, the system selects acoustic front-end processing engine A. The selected acoustic front-end processing engine A pre-processes the user's voice commands, including audio noise reduction, echo cancellation, and reverberation suppression. Among them, audio noise reduction is used to eliminate engine noise, wind noise, and road noise, echo cancellation is used to reduce the echo caused by the space inside the vehicle, and reverberation suppression is used to optimize the clarity of the voice signal. The pre-processed audio data is clearer and easier to be processed by subsequent modules (such as speech recognition).

[0041] In in-vehicle voice systems, acoustic front-end processing is a key link in ensuring the accuracy of voice recognition. By dynamically selecting the optimal acoustic front-end processing engine based on the current scenario or user preferences, the system can more effectively eliminate in-vehicle noise and improve the clarity of voice signals, thereby enhancing the overall voice interaction experience.

[0042] In some embodiments, calling a corresponding wake-up engine to process the preprocessed audio data according to the preprocessed audio data includes:

[0043] The current scenario or user preference is obtained, a wake-up engine is selected from at least two wake-up engines according to the current scenario or user preference, and the one wake-up engine is called to process the preprocessed audio data.

[0044] Specifically, in the in-vehicle voice system, the role of the wake-up engine is to detect whether the user has issued a specific wake-up word (such as "Hello, Xiao A"), thereby activating the voice interaction system. In order to improve the accuracy and adaptability of the wake-up, the system will select the most suitable engine from multiple wake-up engines for processing according to the current scene or user preferences; the system obtains the current scene (such as vehicle speed, in-vehicle noise level) or user preferences (such as sensitivity requirements for wake-up words) through sensors or user settings. For example, when the vehicle speed is high, the ambient noise is large; and the user preference can be set to high wake-up sensitivity. The system selects the most suitable engine from at least two pre-configured wake-up engines. Wake-up engine C may be targeted at The noise environment during high-speed driving is optimized and has stronger noise resistance. Wake-up engine D may be more suitable for quiet environments at low speeds or when parking, and is more sensitive to wake-up word detection. According to the current scenario with high vehicle speed and high noise, the system selects wake-up engine C. The selected wake-up engine processes the pre-processed audio data to detect whether it contains specific wake-up words. For example, when the user says the wake-up word "Hello, Xiao A" while driving at high speed, wake-up engine C accurately detects the wake-up word and activates the system through noise reduction and anti-interference technology. If the wake-up word is detected, the wake-up engine outputs a wake-up signal and the system enters the voice command receiving state. If the wake-up word is not detected, the system continues to remain in sleep or standby state.

[0045] This embodiment uses multiple wake-up engines to support main wake-up word wake-up, interruption wake-up, scene word wake-up, custom wake-up word wake-up, two-word wake-up, etc. If it is a oneshot wake-up method, a pre-audio segment will be taken out from the wake-up engine and sent to the speech recognition engine first. The speech recognition engine continuously performs speech recognition and continuously outputs the text results of speech recognition.

[0046] In in-vehicle voice systems, the selection and processing of wake-up engines is critical to ensuring that the system responds to user commands in a timely manner. By dynamically selecting the optimal wake-up engine based on the current scenario or user preferences, the system can more effectively adapt to different in-vehicle environments, improve the accuracy and reliability of wake-up word detection, and thus enhance the overall voice interaction experience.

[0047] In some embodiments, calling a corresponding speech recognition engine to process the preprocessed audio data according to the preprocessed audio data includes:

[0048] The current scenario or user preference is obtained, a speech recognition engine is selected from at least two speech recognition engines according to the current scenario or user preference, and the speech recognition engine is called to process the preprocessed audio data.

[0049] Specifically, the system can obtain information through sensors (such as GPS, vehicle speed sensor, and in-car microphone array to analyze noise types). For example, it can determine whether the vehicle is currently driving at high speed, in urban congestion, or parked, or detect that the noise inside the vehicle is mainly wind noise, engine noise... Recognition engine C, the system can store the user's personalized settings. For example, the user may prefer an engine that recognizes a specific accent more accurately, or have higher requirements for recognition speed; the system combines the characteristics of different speech recognition engines (such as noise resistance, accuracy in recognizing specific accents / dialects, recognition speed, and degree of optimization for specific vocabulary / fields) based on the acquired scenarios or preference information. The system then makes a decision. For example, Engine X may excel in noisy environments, Engine Y may be more accurate in quiet environments or when recognizing specific terms, and Engine Z is known for its extremely fast recognition speed. Based on the current scenario (such as strong wind noise) and / or user preferences (if the user has not specified a specific preference), the system decides to select Engine X. The system then passes the preprocessed audio data to the selected speech recognition engine (such as Engine X). Engine X executes its internal speech recognition algorithm and converts the audio stream into a text string. Recognition Engine X then outputs the converted text result (such as "Turn on the air conditioner") to the next processing module (the semantic understanding engine).

[0050] In some embodiments, the invoking a corresponding semantic understanding engine according to the voice text to process the voice text includes:

[0051] The current scenario or user preference is obtained, a speech understanding engine is selected from at least two speech understanding engines according to the current scenario or user preference, and the speech understanding engine is called to process the speech text.

[0052] In some embodiments, calling a corresponding dialog management engine to process the semantic understanding information according to the semantic understanding information includes:

[0053] The current scenario or user preference is obtained, a dialogue management engine is selected from at least two dialogue management engines according to the current scenario or user preference, and the one dialogue management engine is called to process the semantic understanding information.

[0054] Specifically, after receiving the speech text converted by the speech recognition engine, the system selects the optimal engine from multiple available semantic understanding engines based on the current scenario or user preferences. The selected engine is then invoked to process the speech text, extracting its intent and key information (such as entities and attributes), and generating structured semantic understanding information so that subsequent upper-level business modules can accurately execute user instructions. The system makes decisions based on the acquired scenario or preference information and the characteristics of different semantic understanding engines (such as their understanding of specific domains / industries, their ability to resolve complex sentences / ambiguities, their depth of contextual understanding, and their ability to leverage user personalized information). For example, engine P may be very powerful in handling navigation and local search, while engine R may be particularly adept at understanding user personalized settings and habits. The system sends the speech recognition output text and related contextual information (optional) to the selected semantic understanding engine. The invoked semantic understanding engine analyzes the text, understands the user's true intent, extracts key information (such as intent type, location, time, person, object, and other entities), and generates structured semantic understanding information (typically a data set containing intent, domain, and word slot information).

[0055] For example, in an in-vehicle intelligent voice system, when a user says the command "navigate to the company", the system first determines that the vehicle is in motion through the vehicle speed sensor, and at the same time identifies from the user settings that the user prefers to use "AutoNavi Map" for navigation, and locates the current location in Shanghai through GPS. Subsequently, the system evaluates multiple available semantic understanding engines and finds that Engine P has the most comprehensive functions and the highest accuracy in navigation, so it selects Engine P to process the command. The system sends the text "Navigate to the company" output by voice recognition and context information such as the current city "Shanghai" to Engine P. Engine P combines this information to accurately understand the user's navigation intention, and combines the data of AutoNavi Map to plan the best route from the current location in Shanghai to the company, and outputs the semantic understanding information to the upper-level business, driving the navigation application to perform the corresponding navigation operation, thereby achieving the purpose of convenient navigation by the user through voice commands. Another embodiment of the present application provides a voice processing device, including a module for executing the voice processing method described in the above embodiment, and the module can be implemented based on software, hardware, or a combination of software and hardware.

[0056] It should be noted that the device provided in this embodiment can be used to execute the method described in the above embodiment. Therefore, the content not described in detail in this embodiment can be obtained by referring to the content of the method of the above embodiment, so it will not be repeated here.

[0057] Another embodiment of the present application provides a speech processing device, comprising:

[0058] Communication interface, used for communicating with other electronic devices;

[0059] a memory for storing computer program instructions;

[0060] The processor is used to execute the computer program instructions to support the device of this embodiment to implement the method described in the above embodiment.

[0061] In this embodiment, the memory mainly includes a program storage area and a data storage area, wherein the program storage area can store operating devices, at least one application required for a function, etc., and the data storage area can store related data, etc. In addition, the memory can be a high-speed random access memory, or a non-volatile memory, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, and a flash card, etc., or the memory can also be other volatile solid-state memory devices.

[0062] The processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor, or the processor may be any conventional processor. The processor is the control center of the device of this embodiment, and uses various interfaces and lines to connect the various parts of the device of this embodiment.

[0063] Another embodiment of the present application provides a computer program product, including computer program instructions, wherein the computer program instructions instruct a computer device to perform operations corresponding to the method described in the above embodiment.

[0064] Specifically, the computer program product includes a series of computer program instructions. These computer program instructions are codes written in a computer program that define how to perform specific operations. These computer program instructions are designed to be loaded onto a computer device and instruct the device to perform specific operations, which are the steps in the speech processing method described in the above embodiment. In this way, the computer program product of this embodiment provides a complete software solution that can be run on various computer devices to implement the speech processing method of the above embodiment.

[0065] While various embodiments of the present application have been described above, the above descriptions are illustrative, non-exhaustive, and not intended to be limiting of the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is selected to best explain the principles of the embodiments, their practical applications, or technological improvements in the marketplace, or to enable others skilled in the art to understand the embodiments disclosed herein.

Claims

1. A speech processing method, characterized in that: The method comprises: Receive user voice commands, call the corresponding acoustic front-end processing engine according to the user voice commands to pre-process the user voice commands, and output pre-processed audio data; Calling a corresponding wake-up engine according to the pre-processed audio data to process the pre-processed audio data and output a wake-up signal; In response to the wake-up signal, calling a corresponding speech recognition engine according to the pre-processed audio data to process the pre-processed audio data and output a speech text; According to the voice text, a corresponding semantic understanding engine is called to process the voice text and output semantic understanding information; wherein, the semantic understanding information is used to drive upper-layer services to execute the user voice instructions according to the semantic understanding information.

2. The method according to claim 1, characterized in that The method comprises: When the semantic understanding engine obtains the semantic understanding information based on the offline recognition mode, it calls the corresponding dialogue management engine according to the semantic understanding information to process the semantic understanding information, and outputs the semantic understanding information carrying decision information to the upper-layer business, so as to drive the upper-layer business to execute the user voice command according to the semantic understanding information; When the semantic understanding engine obtains the semantic understanding information based on the online recognition mode, the semantic understanding information is directly output to the upper-layer business to drive the upper-layer business to execute the user voice instruction according to the semantic understanding information.

3. The method according to claim 2, characterized in that The method comprises: Receive the reply voice text output by the upper-layer business, call the corresponding synthesis engine according to the reply voice text to process the reply voice text, and output the synthesized audio data to the voice player so that the voice player plays the synthesized audio data.

4. The method according to claim 1, wherein The calling of a corresponding acoustic front-end processing engine according to the user voice instruction to pre-process the user voice instruction includes: In response to the user voice command, the current scene or user preference is obtained, an acoustic front-end processing engine is selected from at least two acoustic front-end processing engines according to the current scene or user preference, and the one acoustic front-end processing engine is called to preprocess the user voice command, wherein the preprocessing includes at least audio noise reduction.

5. The method according to claim 1, characterized in that The calling a corresponding wake-up engine according to the pre-processed audio data to process the pre-processed audio data includes: The current scenario or user preference is obtained, a wake-up engine is selected from at least two wake-up engines according to the current scenario or user preference, and the one wake-up engine is called to process the preprocessed audio data.

6. The method according to claim 1, characterized in that The calling of a corresponding speech recognition engine to process the preprocessed audio data according to the preprocessed audio data includes: The current scenario or user preference is obtained, a speech recognition engine is selected from at least two speech recognition engines according to the current scenario or user preference, and the speech recognition engine is called to process the preprocessed audio data.

7. The method according to claim 1, characterized in that The calling of a corresponding semantic understanding engine according to the voice text to process the voice text includes: The current scenario or user preference is obtained, a speech understanding engine is selected from at least two speech understanding engines according to the current scenario or user preference, and the speech understanding engine is called to process the speech text.

8. The method according to claim 2, characterized in that The calling a corresponding dialogue management engine according to the semantic understanding information to process the semantic understanding information includes: The current scenario or user preference is obtained, a dialogue management engine is selected from at least two dialogue management engines according to the current scenario or user preference, and the one dialogue management engine is called to process the semantic understanding information.

9. A speech processing device, characterized in that: Comprising modules for executing the method according to any one of claims 1 to 8.

10. A computer program product, characterized in that The method comprises computer program instructions, wherein the computer program instructions instruct a computer device to execute operations corresponding to the method according to any one of claims 1 to 8.

Images