Voice interaction method and device, vehicle, storage medium and program product
By identifying user intent switching strategies and combining chat mode marking and end-to-end models, the problem of diverse user interaction needs in existing technologies is solved, achieving an efficient and accurate voice interaction experience.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- XIAOMI TECH (WUHAN) CO LTD
- Filing Date
- 2026-03-10
- Publication Date
- 2026-06-05
AI Technical Summary
Existing technical solutions are unable to meet the diverse voice interaction needs of users, resulting in a poor user experience.
By recognizing the user intent represented by the voice signal, the processing strategy can be flexibly switched. The first processing strategy is used to generate control commands, or the second processing strategy is used to generate human-like response voice. Combined with chat mode marking and end-to-end voice generation model, the voice interaction process is optimized.
It enables efficient and accurate voice interaction in both control and chat scenarios, improving user experience and usability, reducing latency, and maintaining the consistency and human-like nature of voice responses.
Smart Images
Figure CN122157653A_ABST
Abstract
Description
Technical Field
[0001] This disclosure relates to the field of artificial intelligence, and more particularly to a voice interaction method, device, vehicle, storage medium, and program product. Background Technology
[0002] With the popularization of intelligent voice interaction technology, users' demands for voice interaction experience are increasing. Existing technical solutions usually rely on a single processing strategy to handle various user voice requests. This approach often fails to meet the diverse needs of users, resulting in a poor user experience. Summary of the Invention
[0003] To overcome the problems existing in related technologies, this disclosure provides a voice interaction method, device, vehicle, storage medium, and program product.
[0004] According to a first aspect of the present disclosure, a voice interaction method is provided, comprising: Acquire the user's voice input signal; Using a processing strategy corresponding to the user intent represented by the voice signal, a response voice corresponding to the voice signal is obtained; Output the reply voice; The processing strategy is either a first processing strategy or a second processing strategy. The first processing strategy is used to generate control commands based on the user's intent so that the controlled device can perform corresponding operations. The response voice generated by the second processing strategy has a higher degree of anthropomorphism than that of the first processing strategy.
[0005] In the above technical solution, during voice interaction, by recognizing the user's intent represented by the voice signal, the corresponding processing strategy can be matched and invoked. When the first processing strategy is adopted, control commands can be accurately generated to ensure that the device effectively executes the operation. When the second processing strategy is adopted, the human-likeness and naturalness of the response can be effectively ensured. In this way, the response voice obtained by using the processing strategy corresponding to the user's intent can meet the diverse needs of users and improve the practicality and user experience of voice interaction.
[0006] In some possible implementations, the second processing strategy has a lower response speech generation latency than the first processing strategy.
[0007] The above technical solutions can achieve low-latency responses, further improving the user experience.
[0008] In some possible implementations, obtaining the response speech corresponding to the speech signal using a processing strategy corresponding to the user intent represented by the speech signal includes: If the user has the intention to control, then the first processing strategy is used to obtain the response voice. If the user does not have the intention to control, the second processing strategy is used to obtain the response voice.
[0009] In the above technical solution, the processing strategy can be flexibly selected according to whether the user has the intention to control. When there is a need for control, precise command control can be performed, and when there is no need for control, a human-like response can be quickly given to improve the interactive experience.
[0010] In some possible implementations, the method further includes: If the preset conditions are met, it is determined that the user does not have the intention to control. If the preset conditions are not met, it is determined that the user has the intention to control. The preset conditions include: the voice signal is identified as carrying a chat mode marker, and the semantic recognition result of the voice signal indicates that the user does not have the intention to control.
[0011] In the above technical solution, the user's control intent can be accurately determined by preset conditions, so as to accurately match the corresponding processing strategy and obtain a response voice that meets the user's needs.
[0012] In some possible implementations, obtaining the response speech corresponding to the speech signal using a processing strategy corresponding to the user intent represented by the speech signal includes: If the voice signal is identified as carrying a chat mode marker, the second processing strategy is activated to process the voice signal to generate a chat response voice, and the natural language understanding function is invoked to perform semantic recognition on the voice signal; If the semantic recognition result indicates that the user has a control intention, then the speech synthesis function in the first processing strategy is invoked to generate a control response speech, and the control response speech is identified as the reply speech; If the semantic recognition result indicates that the user does not have the intention to control, then the chat reply voice is identified as the reply voice.
[0013] In the above technical solution, the processing strategy can be flexibly switched according to the semantic recognition results, which can take into account both chat and control scenarios. It can ensure that the chat reply is timely and human-like, and that the control command is executed accurately.
[0014] In some possible implementations, the first processing strategy is achieved through a cascaded processing chain consisting of automatic speech recognition, natural language understanding, and speech synthesis executed sequentially. The natural language understanding is used to parse user intent and generate control commands to operate the controlled device; the speech synthesis is used to synthesize speech carrying corresponding information based on the result of the natural language understanding or the execution result of the control commands.
[0015] In the above technical solution, the first processing strategy can accurately parse user intent, generate instructions and synthesize speech, ensuring efficient and accurate control and interaction.
[0016] In some possible implementations, the second processing strategy is implemented through an end-to-end speech generation model, which generates speech carrying conversational response content based on the input speech signal.
[0017] The above technical solution can reduce the response delay of the second processing strategy and better maintain the coherence and human-like expression of the voice response.
[0018] In some possible implementations, the speech synthesis function in the first processing strategy is implemented by calling or reusing the speech synthesis module in the end-to-end speech generation model of the second processing strategy.
[0019] The above technical solution can achieve uniformity in voice output style and timbre among different strategies, improve the consistency of user experience, avoid redundant development, and reduce the redundancy of the voice interaction system.
[0020] In some possible implementations, in the second processing strategy, the end-to-end speech generation model is used to generate speech associated with the current dialogue context based on the speech signal and historical dialogue content.
[0021] In the above technical solution, by integrating historical dialogue information, the contextual relevance of the response and the coherence of the dialogue can be ensured, thereby providing an interactive experience that is closer to real-life communication.
[0022] In some possible implementations, in the first processing strategy, when generating control commands, the voice interaction method further includes: The control command is compared and verified with a preset set of prohibition commands; If the control command belongs to the set of prohibited commands, a response voice is generated to indicate that the operation is restricted; If the control instruction does not belong to the set of prohibited instructions, then the control instruction is executed.
[0023] In the above technical solution, by comparing and verifying the set of control commands and prohibition commands, illegal operations can be effectively avoided and equipment safety can be ensured. At the same time, reasonable feedback prompts can optimize the user interaction experience.
[0024] In some possible implementations, the voice interaction method further includes: If the voice signal indicates that the user intends to enter chat mode, then a first control command is generated; The first control command is issued, wherein the first control command is used to instruct the device to add a chat mode marker to the subsequently uploaded voice signal.
[0025] In the above technical solution, users can proactively and conveniently start an interactive experience based on chat, and the voice interaction system can optimize the processing flow in advance based on tags to improve the response speed of semantic replies in chat mode.
[0026] In some possible implementations, the voice interaction method further includes: If the voice signal indicates that the user intends to exit the chat mode, a second control command is generated; The second control command is issued, wherein the second control command is used to instruct the device to remove the chat mode flag for subsequently uploaded voice signals.
[0027] In the above technical solution, users can actively exit the chat mode, and through the issuance and recognition of commands, invalid processing can be avoided, thereby improving the accuracy and efficiency of device voice interaction.
[0028] According to a second aspect of the present disclosure, a voice interaction device is provided, comprising: The acquisition module is used to acquire the user's voice input signal; The processing module is used to obtain a response voice corresponding to the voice signal by using a processing strategy corresponding to the user intent represented by the voice signal; The output module is used to output the response voice. The processing strategy is either a first processing strategy or a second processing strategy. The first processing strategy is used to generate control commands based on the user's intent so that the controlled device can perform corresponding operations. The second processing strategy has a lower response voice generation delay than the first processing strategy, and / or the response voice generated by the second processing strategy is more human-like than that generated by the first processing strategy.
[0029] According to a third aspect of the present disclosure, a voice interaction device is provided, comprising: processor; Memory used to store processor-executable instructions; The processor is configured to execute the executable instructions in the memory to implement the steps of the voice interaction method provided in the first aspect of this disclosure.
[0030] According to a fourth aspect of the present disclosure, a vehicle is provided, comprising: The voice interaction device provided in the second aspect of this disclosure, or the voice interaction device provided in the third aspect of this disclosure.
[0031] According to a fifth aspect of the present disclosure, a computer-readable storage medium is provided, on which a computer program is stored, which, when executed by a processor, implements the steps of the voice interaction method provided in the first aspect of the present disclosure.
[0032] According to a sixth aspect of the present disclosure, a computer program product is provided, including a computer program that, when executed by a processor, comprises the steps of the voice interaction method provided in the first aspect of the present disclosure.
[0033] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description
[0034] The accompanying drawings, which are incorporated in and form a part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure.
[0035] Figure 1 This is a flowchart illustrating a voice interaction method according to an exemplary embodiment.
[0036] Figure 2 This is a schematic diagram illustrating a voice interaction system according to an exemplary embodiment.
[0037] Figure 3 This is a flowchart illustrating a voice interaction method according to an exemplary embodiment.
[0038] Figure 4 This is a block diagram illustrating a voice interaction device according to an exemplary embodiment.
[0039] Figure 5 This is a block diagram illustrating a voice interaction device according to an exemplary embodiment. Detailed Implementation
[0040] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numerals in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this disclosure. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this disclosure as detailed in the appended claims.
[0041] It should be noted that all actions involving the acquisition of signals, information, or data in this disclosure are carried out in compliance with the relevant data protection laws and policies of the country where the location is situated, and with authorization from the owner of the relevant device.
[0042] Figure 1 This is a flowchart illustrating a voice interaction method according to an exemplary embodiment. This voice interaction method can be applied to voice interaction systems, such as voice interaction systems in vehicle smart cockpits. Figure 1 As shown, the voice interaction method may include steps S101 to S103.
[0043] In step S101, the voice signal input by the user is acquired.
[0044] In one embodiment, the user's voice can be captured in real time through a microphone in the terminal device, and after preprocessing such as noise reduction, a clear voice signal can be obtained.
[0045] In step S102, a response voice corresponding to the voice signal is obtained by using a processing strategy that corresponds to the user's intent represented by the voice signal.
[0046] In step S103, a response voice is output.
[0047] The aforementioned processing strategy is either the first processing strategy or the second processing strategy.
[0048] The first processing strategy is used to generate control commands based on user intent, so that the controlled device can perform corresponding operations. For example, after recognizing the user intent to "open the car door", a door opening command can be generated and sent to the door controller to execute the door opening operation; then, a response voice can be generated to announce the operation result such as "the car door is open".
[0049] The second processing strategy generates a more human-like response than the first. For example, during casual conversation, a speech synthesis model integrating emotion parameters can be used to ensure the human-like quality of the response. In one embodiment, the second processing strategy also has a lower response generation latency than the first strategy to ensure response efficiency; for example, a lightweight speech synthesis model can be used to achieve rapid generation of the response.
[0050] In one embodiment, a Natural Language Processing (NLP) module can be used to parse the semantics of the speech signal. If a device control intent (such as "turn on the air conditioner") is identified, a first processing strategy is invoked to process the speech signal; if an interactive intent such as casual conversation or question and answer is identified, a second processing strategy is invoked to process the speech signal.
[0051] In the above technical solution, during voice interaction, by recognizing the user's intent represented by the voice signal, the corresponding processing strategy can be matched and invoked. When the first processing strategy is adopted, control commands can be accurately generated to ensure that the device effectively executes the operation. When the second processing strategy is adopted, the human-likeness and naturalness of the response can be effectively ensured. In this way, the response voice obtained by using the processing strategy corresponding to the user's intent can meet the diverse needs of users and improve the practicality and user experience of voice interaction.
[0052] In some possible implementations, step S102 can be achieved in the following way: If the user has the intention to control, the first processing strategy is used to obtain a response voice. If the user does not have the intention to control, a second processing strategy is used to obtain a response voice.
[0053] In one embodiment, a natural language understanding module can be used to determine whether the user has a control intent in order to select the corresponding processing strategy. In this technical solution, processing strategies can be flexibly selected based on whether the user has a control intent; precise command control is provided when there is a control need, while a quick, human-like response is given when there is no control need, thus improving the interactive experience.
[0054] In one embodiment, whether a user has control intent can be determined by determining whether the voice signal meets preset conditions. For example, if the preset conditions are met, it is determined that the user does not have control intent; if the preset conditions are not met, it is determined that the user has control intent. The preset conditions may include: the voice signal is identified as carrying a chat mode marker, and the semantic recognition result of the voice signal indicates that the user does not have control intent.
[0055] For example, a chat pattern marker can be a pattern marker actively triggered by the user through a specific wake word (such as "Chat with me"). This marker can be used to indicate that the user is currently in a chat-oriented interaction process during a subsequent continuous dialogue. Therefore, if a voice signal carrying a chat pattern marker is detected, it can be assumed that the current voice interaction is highly likely to continue with a non-controlling chat intention.
[0056] For example, natural language understanding technology can be used to semantically understand speech signals. By identifying key information such as the device subject and control verbs in the sentence, it can be determined whether the current speech signal contains the intention to operate the controlled device.
[0057] In this way, by setting preset conditions, the user's control intention can be accurately determined, and the corresponding processing strategy can be accurately matched to obtain a response voice that meets the user's needs.
[0058] In some possible implementations, step S102 can be achieved in the following way: If the voice signal is identified as carrying a chat mode marker, the second processing strategy is activated to process the voice signal to generate a chat response voice, and the natural language understanding function is invoked to perform semantic recognition on the voice signal. If the semantic recognition result indicates that the user has a control intention, then the speech synthesis function in the first processing strategy is invoked to generate a control response speech, and the control response speech is identified as the reply speech; If the semantic recognition result indicates that the user does not have the intention to control, then the chat reply voice is identified as a reply voice.
[0059] In one embodiment, when a voice signal is identified as carrying a chat mode marker, the voice interaction system can immediately activate a second processing strategy to generate a human-like chat response with low latency, while simultaneously performing semantic recognition on the voice signal. In this way, while ensuring the fluency and immediacy of the conversation, it can simultaneously monitor and judge the user's potential control intentions, thereby maintaining the user's chat experience while promptly responding to device control commands.
[0060] For example, if a user says "turn on the air conditioner" during a chat, semantic recognition will detect that the user has the intention to control. The voice interaction system can immediately generate a command to control the air conditioner to turn on. If the air conditioner is already on, the voice synthesis function of the first processing strategy will generate a control response voice such as "the air conditioner has been turned on for you".
[0061] For example, if a user says "Tell me a joke" during a chat, semantic recognition will detect that the user does not have the intention to control. In this case, the chat reply voice generated by the second processing strategy can be identified as the reply voice.
[0062] In the above technical solution, the processing strategy can be flexibly switched according to the semantic recognition results, which can take into account both chat and control scenarios. It can ensure that the chat reply is timely and human-like, and that the control command is executed accurately.
[0063] In some possible implementations, the second processing strategy is implemented through an end-to-end speech generation model, which generates speech carrying conversational response content based on the input speech signal.
[0064] This end-to-end speech generation model takes speech signals as direct input and directly outputs the corresponding response speech. It unifies the processes of ASR (Automatic Speech Recognition), NLP (Natural Language Processing), and TTS (Text-to-Speech) into a single large model, eliminating cumbersome modular processing steps. This end-to-end speech generation model learns the complete mapping from input speech to output response speech through a single, end-to-end trained neural network system, thereby reducing latency and better maintaining the coherence and human-like expression of the speech response.
[0065] In one embodiment, in the second processing strategy, an end-to-end speech generation model is used to generate speech associated with the current dialogue context based on the speech signal and historical dialogue content.
[0066] For example, a user engages in multiple rounds of casual conversation, such as first asking "How's the weather today?" and then later asking "What about tomorrow?". When using the second processing strategy, the end-to-end speech generation model not only receives the current "What about tomorrow?" speech signal, but also combines the weather query intent identified in the previous round of dialogue with historical context to generate a context-appropriate and natural response speech, such as "Tomorrow is expected to be sunny, and the temperature will be about two degrees higher than today."
[0067] In this way, by integrating historical dialogue information, we can ensure the contextual relevance and dialogue coherence of the response, thereby providing an interactive experience that is closer to real-life communication.
[0068] In some possible implementations, the first processing strategy is achieved through a cascaded processing chain consisting of automatic speech recognition, natural language understanding, and speech synthesis executed sequentially. Natural language understanding is used to parse user intent and generate control commands to operate controlled devices; speech synthesis is used to synthesize speech carrying corresponding information based on the results of natural language understanding or the execution results of control commands.
[0069] In one embodiment, in the first processing strategy, the ASR module can first be used to convert speech into text; then the NLP module is used to parse the controlled device and specific operation to generate control commands, which are sent to the controlled device for execution via network protocol; the TTS module can synthesize corresponding speech for broadcast based on the execution feedback returned by the device.
[0070] For example, when a user says "open the car window", the ASR module can be used to convert the speech into text; then, the NLP module parses "car window" as the controlled device and "open" as the operation intention, and generates a car window opening command accordingly; after the command is successfully executed, the TTS module synthesizes the corresponding speech based on the "car window open" status feedback returned by the device and broadcasts it, thereby completing a device control interaction.
[0071] The first processing strategy employs a cascaded processing chain, with automatic speech recognition, natural language understanding, and speech synthesis working in sequence to accurately interpret user intent, generate commands, and synthesize speech, ensuring efficient and accurate control and interaction.
[0072] In one embodiment, the speech synthesis function in the first processing strategy is implemented by calling or reusing the speech synthesis module in the end-to-end speech generation model of the second processing strategy.
[0073] For example, if the text to be used in TTS is "The air conditioning has been turned on for you," it can be input into the speech synthesis module integrated into the end-to-end speech generation model via a preset interface to directly generate a natural and fluent voice with a tone and style familiar to the user. This achieves consistency in speech output style and tone across different strategies, improves user experience consistency, avoids redundant development, and reduces redundancy in the voice interaction system.
[0074] Figure 2 This is a schematic diagram illustrating a voice interaction system according to an exemplary embodiment. The voice interaction system can be located in the cloud, with voice signals collected by the vehicle and uploaded to the cloud in real time as an audio stream. The cloud can then send the generated response voice back to the vehicle to enable voice interaction with occupants.
[0075] Figure 3 This is a flowchart illustrating a voice interaction method according to an exemplary embodiment. Through this... Figure 3 This allows for a clearer understanding of the implementation process of the voice interaction method provided in this disclosure. For example... Figure 3 As shown, the voice interaction method may include steps S301 to S308.
[0076] In step S301, the voice signal is input to the ASR module in the cascaded processing link to obtain the recognized text.
[0077] In step S302, it is determined whether the voice signal carries a chat mode marker. If yes, steps S303 and S304 are executed; otherwise, step S308 is executed.
[0078] In step S303, an end-to-end speech generation model is invoked to generate human-like chat response speech.
[0079] In step S304, the identified text is input to the NLP module in the cascaded processing chain for semantic understanding.
[0080] In step S305, it is determined whether the semantic recognition result represents that the user has a control intention. If yes, proceed to step S306; otherwise, proceed to step S307.
[0081] In step S306, the end-to-end speech generation model of the control system pauses speech generation and instead uses the TTS module in the cascaded processing link to generate a response speech based on the control intent and the result of the command execution.
[0082] In step S307, the chat response voice output by the end-to-end speech generation model is used as the reply voice.
[0083] In step S308, the recognized text is input to the NLP module in the cascaded processing link for semantic understanding, and the TTS module in the cascaded processing link directly uses the natural language understanding results given by the NLP module or the execution results of control commands generated based on user intent to generate a response voice carrying the corresponding information.
[0084] In the above technical solution, by introducing chat mode marking and a dual-path parallel processing mechanism, the user experience in both control and chat scenarios is effectively balanced. The voice interaction system can select the optimal processing path, ensuring accurate and reliable execution of control commands while providing low-latency, highly human-like responses for chat interactions, thus optimizing the user experience.
[0085] In some possible implementations, in the first processing strategy, when generating control commands, the voice interaction method provided in this disclosure further includes: The control commands are compared and verified against a preset set of prohibited commands. If the control command belongs to the prohibited command set, a response voice will be generated to indicate that the operation is restricted; If the control instruction is not in the prohibited instruction set, then the control instruction is executed.
[0086] For example, when a user issues a voice signal saying "Switch the vehicle to Sport mode and turn off all safety assists," the system can generate a corresponding control command through natural language understanding. Before execution, the command can be compared with a preset set of prohibited commands. If the prohibited command set includes "Turn off all safety assists," the vehicle will not execute the command, but will instead generate a prompt such as "For your safety, this function cannot be executed in the current state" via the TTS module. If the command does not belong to the prohibited command set, it can be determined that the vehicle can execute the command, and the command can be executed normally with feedback.
[0087] In the above technical solution, by comparing and verifying the set of control commands and prohibition commands, illegal operations can be effectively avoided and equipment safety can be ensured. At the same time, reasonable feedback prompts can optimize the user interaction experience.
[0088] In some possible implementations, the voice interaction method provided in this disclosure further includes: If the voice signal indicates that the user intends to enter chat mode, then the first control command is generated; Issue the first control command.
[0089] The first control command is used to instruct the device to add a chat mode marker to the subsequently uploaded voice signals.
[0090] In one embodiment, when a user sends a voice signal to "start chat mode", the NLP module can identify the user's intention to enter chat mode and generate a first control command to "add chat mode flag" and send it to the vehicle. After receiving the first control command, the vehicle can automatically add the flag to the audio stream or voice data packet that is subsequently collected and uploaded until a removal command is received.
[0091] In this way, users can proactively and conveniently initiate a chat-based interactive experience, and the voice interaction system can optimize the processing flow in advance based on tags, thereby improving the response speed of semantic replies in chat mode.
[0092] In some possible implementations, the voice interaction method provided in this disclosure further includes: If the voice signal indicates that the user intends to exit the chat mode, a second control command is generated; Issue the second control command.
[0093] The second control command is used to instruct the device to remove the chat mode marker for subsequently uploaded voice signals.
[0094] In one embodiment, when a user issues a voice signal to "exit chat mode," the NLP module can recognize the user's intention to exit chat mode, generate a second control command to "remove chat mode flag," and send it to the vehicle. Upon receiving this second control command, the vehicle can stop adding the flag to subsequent audio streams or voice data packets, restoring the voice interaction system to the standard mode of voice processing using the first processing strategy.
[0095] In this way, users can actively exit chat mode, and through the issuance and recognition of commands, invalid processing can be avoided, thereby improving the accuracy and efficiency of device voice interaction.
[0096] Figure 4 This is a block diagram illustrating a voice interaction device according to an exemplary embodiment. (Refer to...) Figure 4 The voice interaction device 400 includes: Acquisition module 401 is used to acquire the voice signal input by the user; Processing module 402 is used to obtain a response voice corresponding to the voice signal by using a processing strategy corresponding to the user intent represented by the voice signal; Output module 403 is used to output the reply voice; The processing strategy is either a first processing strategy or a second processing strategy. The first processing strategy is used to generate control commands based on the user's intent so that the controlled device can perform corresponding operations. The response voice generated by the second processing strategy has a higher degree of anthropomorphism than that of the first processing strategy.
[0097] In the above technical solution, during voice interaction, by recognizing the user's intent represented by the voice signal, the corresponding processing strategy can be matched and invoked. When the first processing strategy is adopted, control commands can be accurately generated to ensure that the device effectively executes the operation. When the second processing strategy is adopted, the human-likeness and naturalness of the response can be effectively ensured. In this way, the response voice obtained by using the processing strategy corresponding to the user's intent can meet the diverse needs of users and improve the practicality and user experience of voice interaction.
[0098] In some possible implementations, the second processing strategy has a lower response speech generation latency than the first processing strategy.
[0099] In some possible implementations, the processing module 402 includes: The first processing submodule is used to obtain the response voice using the first processing strategy if the user has a control intention; The second processing submodule is used to obtain the response voice by utilizing the second processing strategy if the user does not have the intention to control.
[0100] In some possible implementations, the processing module 402 further includes: The determination submodule is used to determine whether the user has control intent if a preset condition is met, and to determine whether the user has control intent if the preset condition is not met. The preset conditions include: the voice signal is identified as carrying a chat mode marker, and the semantic recognition result of the voice signal indicates that the user does not have the intention to control.
[0101] In some possible implementations, the processing module 402 is used to obtain the response voice corresponding to the voice signal in the following manner: If the voice signal is identified as carrying a chat mode marker, the second processing strategy is activated to process the voice signal to generate a chat response voice, and the natural language understanding function is invoked to perform semantic recognition on the voice signal; If the semantic recognition result indicates that the user has a control intention, then the speech synthesis function in the first processing strategy is invoked to generate a control response speech, and the control response speech is identified as the reply speech; If the semantic recognition result indicates that the user does not have the intention to control, then the chat reply voice is identified as the reply voice.
[0102] In some possible implementations, the first processing strategy is achieved through a cascaded processing chain consisting of automatic speech recognition, natural language understanding, and speech synthesis executed sequentially. The natural language understanding is used to parse user intent and generate control commands to operate the controlled device; the speech synthesis is used to synthesize speech carrying corresponding information based on the result of the natural language understanding or the execution result of the control commands.
[0103] In some possible implementations, the second processing strategy is implemented through an end-to-end speech generation model, which generates speech carrying conversational response content based on the input speech signal.
[0104] In some possible implementations, the speech synthesis function in the first processing strategy is implemented by calling or reusing the speech synthesis module in the end-to-end speech generation model of the second processing strategy.
[0105] In some possible implementations, in the second processing strategy, the end-to-end speech generation model is used to generate speech associated with the current dialogue context based on the speech signal and historical dialogue content.
[0106] In some possible implementations, in the first processing strategy, when generating control instructions, the processing module further includes: The control module is used to compare and verify the control command with a preset set of prohibited commands; if the control command belongs to the set of prohibited commands, a response voice is generated to indicate that the operation is restricted; if the control command does not belong to the set of prohibited commands, the control command is executed.
[0107] In some possible implementations, the voice interaction device 400 further includes: The first generation and distribution module is used to generate a first control command if the voice signal indicates that the user intends to enter chat mode; and to distribute the first control command, wherein the first control command is used to instruct the device to add a chat mode mark to the subsequently uploaded voice signal.
[0108] In some possible implementations, the voice interaction device 400 further includes: The second generation and delivery module is used to generate a second control command if the voice signal indicates that the user intends to exit the chat mode; and to deliver the second control command, wherein the second control command is used to instruct the device to remove the chat mode mark for subsequently uploaded voice signals.
[0109] Regarding the apparatus in the above embodiments, the specific manner in which each module performs its operation has been described in detail in the embodiments related to the method, and will not be elaborated upon here.
[0110] Figure 5 This is a block diagram illustrating a voice interaction device 1900 according to an exemplary embodiment. For example, device 1900 may be provided as a server. (Refer to...) Figure 5 The device 1900 includes a processing component 1922, which further includes one or more processors, and memory resources represented by memory 1932 for storing instructions, such as application programs, that can be executed by the processing component 1922. The application programs stored in memory 1932 may include one or more modules, each corresponding to a set of instructions. Furthermore, the processing component 1922 is configured to execute instructions to perform the aforementioned voice interaction method.
[0111] Device 1900 may also include a power supply component 1926 configured to perform power management of device 1900, a wired or wireless network interface 1950 configured to connect device 1900 to a network, and an input / output interface 1958. Device 1900 can operate on an operating system, such as Windows Server, stored in memory 1932. TM Mac OS X TM Unix TM Linux TM FreeBSD TM Or similar.
[0112] In another exemplary embodiment, this disclosure also provides a vehicle that includes means for implementing the above-described voice interaction method, or is capable of interacting with means for implementing the aforementioned voice interaction method.
[0113] In another exemplary embodiment, this disclosure also provides a computer-readable storage medium having stored thereon computer program instructions that, when executed by a processor, implement the steps of the voice interaction method provided in this disclosure.
[0114] In another exemplary embodiment, this disclosure also provides a computer program product comprising a computer program executable by a programmable device, the computer program having a code portion for performing the above-described voice interaction method when executed by the programmable device.
[0115] Those skilled in the art will also understand that the various illustrative logical blocks and steps listed in the embodiments of this application can be implemented by electronic hardware, computer software, or a combination of both. Whether such functionality is implemented through hardware or software depends on the specific application and the overall system design requirements. Those skilled in the art can implement the described functionality using various methods for each specific application, but such implementation should not be construed as exceeding the scope of protection of the embodiments of this application.
[0116] It should be understood that, unless otherwise specifically indicated, features of various embodiments of this disclosure described herein can be combined with each other. As used herein, the term “and / or” includes any one of the relevant listed items and any combination of any two or more; similarly, “at least one of…” includes any one of the relevant listed items and any combination of any two or more.
[0117] Although terms such as “first,” “second,” and “third” may be used herein to describe various components, parts, regions, layers, or sections, these components, parts, regions, layers, or sections are not limited to these terms. Rather, these terms are used only to distinguish one component, part, region, layer, or section from another. Therefore, without departing from the teachings of the examples described herein, the first component, part, region, layer, or section mentioned in the examples may also be referred to as the second component, part, region, layer, or section. Furthermore, the terms “first” and “second” are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, a feature defined as “first” or “second” may explicitly or implicitly include at least one of that feature. In the description herein, “a plurality” means at least two, such as two, three, etc., unless otherwise explicitly specified.
[0118] Furthermore, the term “exemplary” is used herein to mean serving as an example, instance, or illustration. Any aspect or design described herein as “exemplary” is not necessarily to be construed as advantageous compared to other aspects or designs. Rather, the use of the term “exemplary” is intended to present the concept in a concrete manner. As used herein, the term “or” is intended to mean an inclusive “or” rather than an exclusive “or.” That is, unless otherwise specified or clear from the context, “X applies A or B” is intended to mean any of the natural inclusive arrangements. That is, “X applies A or B” satisfies any of the foregoing instances if X applies A; X applies B; or both X applies A and B. Additionally, unless otherwise specified or clear from the context to refer to the singular form, the articles “a” and “an” as used in this application and the appended claims are generally understood to mean “one or more.”
[0119] Similarly, although this disclosure has been shown and described with respect to one or more implementations, equivalent variations and modifications will occur to those skilled in the art upon reading and understanding this specification and the accompanying drawings. This disclosure includes all such modifications and variations and is limited only by the scope of the claims. In particular, with respect to the various functions performed by the components described above (e.g., elements, resources, etc.), unless otherwise indicated, the terminology used to describe such components is intended to correspond to any component (functionally equivalent) that performs the specific function of the described component, even if structurally not equivalent to the disclosed structure. Furthermore, although specific features of this disclosure may have been disclosed with respect to only one of several implementations, such features may be combined with one or more other features of other implementations, as may be desired and advantageous to any given or particular application. Moreover, with regard to the terms “comprising,” “owning,” “having,” “having,” or variations thereof as used in the detailed description or claims, such terms are intended to be inclusive in a manner similar to the term “including.”
[0120] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the appended claims.
[0121] It should be understood that this disclosure is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this disclosure is limited only by the appended claims.
Claims
1. A voice interaction method, characterized in that, include: Acquire the user's voice input signal; Using a processing strategy corresponding to the user intent represented by the voice signal, a response voice corresponding to the voice signal is obtained; Output the reply voice; The processing strategy is either a first processing strategy or a second processing strategy. The first processing strategy is used to generate control commands based on the user's intent so that the controlled device can perform corresponding operations. The response voice generated by the second processing strategy has a higher degree of anthropomorphism than that of the first processing strategy.
2. The voice interaction method according to claim 1, characterized in that, The second processing strategy has a lower response speech generation latency than the first processing strategy.
3. The voice interaction method according to claim 1, characterized in that, The step of obtaining a response voice corresponding to the voice signal using a processing strategy corresponding to the user intent represented by the voice signal includes: If the user has the intention to control, then the first processing strategy is used to obtain the response voice. If the user does not have the intention to control, the second processing strategy is used to obtain the response voice.
4. The voice interaction method according to claim 3, characterized in that, The method further includes: If the preset conditions are met, it is determined that the user does not have the intention to control. If the preset conditions are not met, it is determined that the user has the intention to control. The preset conditions include: the voice signal is identified as carrying a chat mode marker, and the semantic recognition result of the voice signal indicates that the user does not have the intention to control.
5. The voice interaction method according to claim 1, characterized in that, The step of obtaining a response voice corresponding to the voice signal using a processing strategy corresponding to the user intent represented by the voice signal includes: If the voice signal is identified as carrying a chat mode marker, the second processing strategy is activated to process the voice signal to generate a chat response voice, and the natural language understanding function is invoked to perform semantic recognition on the voice signal; If the semantic recognition result indicates that the user has a control intention, then the speech synthesis function in the first processing strategy is invoked to generate a control response speech, and the control response speech is identified as the reply speech; If the semantic recognition result indicates that the user does not have the intention to control, then the chat reply voice is identified as the reply voice.
6. The voice interaction method according to any one of claims 1-5, characterized in that, The first processing strategy is implemented through a cascaded processing chain consisting of automatic speech recognition, natural language understanding, and speech synthesis executed sequentially. The natural language understanding is used to parse user intent and generate control commands to operate the controlled device; the speech synthesis is used to synthesize speech carrying corresponding information based on the result of the natural language understanding or the execution result of the control commands.
7. The voice interaction method according to any one of claims 1-5, characterized in that, The second processing strategy is implemented through an end-to-end speech generation model, which is used to generate speech carrying conversational response content based on the input speech signal.
8. The voice interaction method according to claim 6, characterized in that, The speech synthesis function in the first processing strategy is implemented by calling or reusing the speech synthesis module in the end-to-end speech generation model of the second processing strategy.
9. The voice interaction method according to claim 7, characterized in that, In the second processing strategy, the end-to-end speech generation model is used to generate speech associated with the current dialogue context based on the speech signal and historical dialogue content.
10. The voice interaction method according to claim 1, characterized in that, In the first processing strategy, when generating control commands, the voice interaction method further includes: The control command is compared and verified with a preset set of prohibition commands; If the control command belongs to the set of prohibited commands, a response voice is generated to indicate that the operation is restricted; If the control instruction does not belong to the set of prohibited instructions, then the control instruction is executed.
11. The voice interaction method according to claim 1, characterized in that, The voice interaction method further includes: If the voice signal indicates that the user intends to enter chat mode, then a first control command is generated; The first control command is issued, wherein the first control command is used to instruct the device to add a chat mode marker to the subsequently uploaded voice signal.
12. The voice interaction method according to claim 1, characterized in that, The voice interaction method further includes: If the voice signal indicates that the user intends to exit the chat mode, a second control command is generated; The second control command is issued, wherein the second control command is used to instruct the device to remove the chat mode flag for subsequently uploaded voice signals.
13. A voice interaction device, characterized in that, include: The acquisition module is used to acquire the user's voice input signal; The processing module is used to obtain a response voice corresponding to the voice signal by using a processing strategy corresponding to the user intent represented by the voice signal; The output module is used to output the response voice. The processing strategy is either a first processing strategy or a second processing strategy. The first processing strategy is used to generate control commands based on the user's intent so that the controlled device can perform corresponding operations. The response voice generated by the second processing strategy has a higher degree of anthropomorphism than that of the first processing strategy.
14. The voice interaction device according to claim 13, characterized in that, The second processing strategy has a lower response speech generation latency than the first processing strategy.
15. The voice interaction device according to claim 13, characterized in that, The processing module includes: The first processing submodule is used to obtain the response voice using the first processing strategy if the user has a control intention; The second processing submodule is used to obtain the response voice by utilizing the second processing strategy if the user does not have the intention to control.
16. The voice interaction device according to claim 14, characterized in that, The processing module further includes: The determination submodule is used to determine whether the user has control intent if a preset condition is met, and to determine whether the user has control intent if the preset condition is not met. The preset conditions include: the voice signal is identified as carrying a chat mode marker, and the semantic recognition result of the voice signal indicates that the user does not have the intention to control.
17. The voice interaction device according to claim 13, characterized in that, The processing module is used to obtain the response voice corresponding to the voice signal in the following manner: If the voice signal is identified as carrying a chat mode marker, the second processing strategy is activated to process the voice signal to generate a chat response voice, and the natural language understanding function is invoked to perform semantic recognition on the voice signal; If the semantic recognition result indicates that the user has a control intention, then the speech synthesis function in the first processing strategy is invoked to generate a control response speech, and the control response speech is identified as the reply speech; If the semantic recognition result indicates that the user does not have the intention to control, then the chat reply voice is identified as the reply voice.
18. The voice interaction device according to any one of claims 13-17, characterized in that, The first processing strategy is implemented through a cascaded processing chain consisting of automatic speech recognition, natural language understanding, and speech synthesis executed sequentially. The natural language understanding is used to parse user intent and generate control commands to operate the controlled device; the speech synthesis is used to synthesize speech carrying corresponding information based on the result of the natural language understanding or the execution result of the control commands.
19. The voice interaction device according to any one of claims 13-17, characterized in that, The second processing strategy is implemented through an end-to-end speech generation model, which is used to generate speech carrying conversational response content based on the input speech signal.
20. A voice interaction device, characterized in that, include: processor; Memory used to store processor-executable instructions; The processor is configured to execute the executable instructions in the memory to implement the steps of the voice interaction method according to any one of claims 1-12.
21. A vehicle, characterized in that, include: The voice interaction device according to any one of claims 13-19, or the voice interaction device according to claim 20.
22. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the voice interaction method according to any one of claims 1-12.
23. A computer program product, characterized in that, It includes a computer program that, when executed by a processor, implements the steps of the voice interaction method according to any one of claims 1-12.