Voice interaction mode control method and apparatus for in-vehicle infotainment system, electronic device, and storage medium

Abstract

A voice interaction mode control method and apparatus for an in-vehicle infotainment system, an electronic device, and a storage medium. The method comprises: in response to voice interaction wake-up voice sent by a user, acquiring a voice instruction sent by the user (S21); performing intention recognition on the voice instruction to determine a voice intention (S22); determining a control strategy for the voice interaction mode of an in-vehicle infotainment system on the basis of the voice intention (S23); and controlling the voice interaction mode of the in-vehicle infotainment system on the basis of the control strategy (S24).

Description

Voice interaction mode control methods, devices, electronic equipment and storage media for in-vehicle systems

[0001] This application claims priority to Chinese Patent Application No. 202410690572.0, filed on May 30, 2024, entitled "Voice Interaction Mode Control Method, Apparatus, Electronic Device and Storage Medium for Vehicle Infotainment System", the entire contents of which are incorporated herein by reference. Technical Field

[0002] This application relates to the field of intelligent cockpit voice interaction technology, and in particular to a method, device, electronic device and storage medium for controlling the voice interaction mode of a vehicle system. Background Technology

[0003] In recent years, the human-computer interaction experience of automotive intelligent cockpits has been greatly improved, but there are still three major pain points: the dialogue style is relatively fixed and monotonous; the interaction does not penetrate all skills and chat; and it is difficult to customize cross-scenario business, as each sentence that needs to be recognized must be pre-customized in order to respond, and it is impossible to answer whatever is asked.

[0004] With the continued popularity of Chat Generative Pre-trained Transformer (ChatGPT), a number of leading large-scale model companies have emerged, such as Spark Big Model, Wenxin Yiyan, and Pangu Big Model. How to integrate these large-scale models into the voice interaction of in-vehicle systems and provide users with an intelligent cockpit voice interaction environment with an excellent voice interaction experience has become an urgent problem to be solved. This can contribute to the development of intelligent and automated automotive technologies in the future and bring consumers more advanced, intelligent, and safer automotive products and services. Summary of the Invention

[0005] In view of this, in order to solve the above-mentioned technical problems or some of the technical problems, the embodiments of this application provide a method, device, electronic device and storage medium for controlling the voice interaction mode of a vehicle system.

[0006] In a first aspect, embodiments of this application provide a voice interaction mode control method for a vehicle infotainment system, executed by an electronic device, comprising:

[0007] It responds to user voice interaction to wake up the voice and obtains the user's voice commands;

[0008] The voice command is subjected to intent recognition to determine the voice intent;

[0009] A control strategy for determining the voice interaction mode of the vehicle system based on the stated voice intent;

[0010] The voice interaction mode of the vehicle system is controlled based on the aforementioned control strategy.

[0011] In one possible implementation, the method further includes:

[0012] Pre-train a voice command intent recognition model;

[0013] The voice command is input into the voice command intent recognition model to obtain the model output result;

[0014] The voice intent of the voice command is determined based on the output of the model.

[0015] In one possible implementation, the method further includes:

[0016] If the voice intent is determined to be a chat mode intent, then the control strategy is determined to be to enter chat mode;

[0017] If the voice intent is determined to be a vehicle-mounted system service control intent, then the control strategy is determined to be to enter the vehicle-mounted system control mode;

[0018] If the voice intent is determined to be a casual conversation intent, then the control strategy is determined to be a chat response.

[0019] In one possible implementation, the method further includes:

[0020] If the control strategy is determined to be to enter the chat mode, then the voice interaction mode of the vehicle system is controlled to enter the chat mode.

[0021] In the "Chat Mode", the system responds to the user's voice through a large model to conduct voice interaction, and keeps the voice interaction mode of the vehicle system in "Chat Mode".

[0022] In one possible implementation, the method further includes:

[0023] If the control strategy is determined to be entering the vehicle control mode, then the corresponding business control is performed on the vehicle in response to the voice command, and the chat mode is tightened.

[0024] If the control strategy is determined to be a chat response, then the voice command is replied to through a large model.

[0025] In one possible implementation, the method further includes:

[0026] In the vehicle control mode, the second round of voice commands issued by the user are acquired;

[0027] Perform intent recognition on the second round of voice commands to determine the intent of the second round of voice commands;

[0028] If it is determined that the second round of voice intent is a chat mode intent, then the voice interaction mode of the vehicle system is controlled to enter chat mode;

[0029] If it is determined that the second round of voice intent is a vehicle system business control intent, then respond to the second round of voice command to perform the corresponding business control on the vehicle system, and maintain the tightened chat mode;

[0030] If the second round of voice intent is determined to be a casual conversation intent, then do not respond to chat and maintain the tight chat mode.

[0031] In one possible implementation, the method further includes:

[0032] Acquire historical human-machine interaction voice recordings under vehicle control mode;

[0033] The historical human-machine interaction voices are analyzed to obtain the user's vehicle control habits;

[0034] Based on the aforementioned vehicle control habits, it responds to voice commands issued by the user.

[0035] Secondly, embodiments of this application provide a voice interaction mode control device for a vehicle infotainment system, comprising:

[0036] The response acquisition module is used to respond to the user's voice interaction wake-up voice and acquire the user's voice commands.

[0037] The recognition and determination module is used to perform intent recognition on the voice command and determine the voice intent.

[0038] The identification and determination module is also used to determine the control strategy of the vehicle's voice interaction mode based on the voice intent;

[0039] The control module is used to control the voice interaction mode of the vehicle system based on the control strategy.

[0040] Thirdly, embodiments of this application provide an electronic device, including: a processor and a memory, wherein the processor is configured to execute a vehicle-mounted system voice interaction mode control program stored in the memory to implement the vehicle-mounted system voice interaction mode control method described in the first aspect above.

[0041] Fourthly, embodiments of this application provide a storage medium, comprising: the storage medium storing one or more programs, the one or more programs being executable by one or more processors to implement the vehicle infotainment system voice interaction mode control method described in the first aspect above.

[0042] The vehicle infotainment system voice interaction mode control scheme provided in this application embodiment responds to user voice interaction to wake up the voice and obtain the user's voice command; performs intent recognition on the voice command to determine the voice intent; determines the control strategy for the vehicle infotainment system's voice interaction mode based on the voice intent; and controls the vehicle infotainment system's voice interaction mode based on the control strategy. Compared with existing automotive intelligent cockpits, which suffer from problems such as a single human-computer interaction dialogue style, difficulty in cross-scenario business transitions, and inability to chat smoothly, this scheme separates multiple intents of voice and controls the vehicle infotainment system's voice interaction mode according to different intents. It tightens the dialogue in different voice interaction modes, allowing the vehicle infotainment system not only to execute voice commands but also to seamlessly connect and chat with the user, thus improving the user experience. Attached Figure Description

[0043] Figure 1 is a diagram of the voice interaction mode control system architecture of a vehicle-mounted system provided in an embodiment of this application;

[0044] Figure 2 is a flowchart illustrating a voice interaction mode control method for a vehicle infotainment system provided in an embodiment of this application;

[0045] Figure 3 is a flowchart illustrating another voice interaction mode control method for a vehicle infotainment system provided in an embodiment of this application;

[0046] Figure 4 is a schematic diagram of the structure of a voice interaction mode control device for a vehicle system provided in an embodiment of this application;

[0047] Figure 5 is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0048] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are some embodiments of this application, but not all embodiments.

[0049] Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without inventive effort are within the scope of protection of this application.

[0050] To facilitate understanding of the embodiments of this application, further explanations and descriptions will be provided below with reference to the accompanying drawings. These embodiments do not constitute a limitation on the embodiments of this application.

[0051] Figure 1 is a diagram of the voice interaction mode control system architecture of a vehicle-mounted system provided in an embodiment of this application. As shown in Figure 1, this embodiment of the application is based on large-model intelligent voice chat, which will redefine the voice interaction system in the intelligent cockpit of the car. The system is divided into five aspects: multi-intent separation, multi-turn dialogue tightening logic, multi-turn semantic history inheritance, large-model chat capability, and self-learning evolution.

[0052] This system relies on large-scale model technology and existing in-vehicle voice interaction systems. Combining user operating habits and actual needs, it breaks down traditional chat barriers, creating a chat mode with cognitive levels. It integrates traditional chat with skill-based voice commands, allowing voice to not only execute instructions but also seamlessly chat with users. Coupled with self-learning and evolution, it provides users with warmth and companionship at any time. This large-scale model voice interaction system is an evolvable and customizable voice interaction system that can be customized for various OEMs.

[0053] The entire system uses user-initiated wake-up as the voice interaction entry point, then decomposes it through an intent separator, and finally enters a three-layer top-down separation logic: the first round of dialogue in the main scenario, the main scenario tightened chat dialogue, and the dialogue in the chat mode. After the user's first round of dialogue, the chat is tightened in the continuous dialogue to control accidental triggers, and the chat scenario can realize the switching between chat and business at any time.

[0054] Figure 2 is a flowchart illustrating a voice interaction mode control method for a vehicle infotainment system according to an embodiment of this application. As shown in Figure 2, the method includes:

[0055] S21. Respond to the user's voice interaction to wake up the voice and obtain the user's voice command.

[0056] In this embodiment, the vehicle system can be activated by a voice interaction wake-up voice (e.g., "Hello X") issued by a user in any of the four sound zones (driver's seat, front passenger seat, rear left seat, and rear right seat) within the car cabin, and then enter the voice interaction system.

[0057] Furthermore, after activating the vehicle's infotainment system and entering the voice interaction system, a voice prompt (e.g., "Hello, please speak") can be issued to remind the user to issue further voice commands. After hearing the voice prompt from the infotainment system, the user can then speak a voice command with an intention, such as: "Let's chat," "Enter chat mode," "Turn on the air conditioning," or "I'm in a good mood today."

[0058] S22. Perform intent recognition on the voice command to determine the voice intent.

[0059] This application embodiment can use a pre-trained voice intent recognition model to identify the intent of voice commands and determine the actual voice intent corresponding to the user's voice command. Alternatively, it can pre-store multiple voice commands under different voice intent categories, and determine the actual voice intent corresponding to the current user's voice command by comparing whether the current user's voice command is the same as or similar to the stored voice commands.

[0060] For example, if a user says "chat," the corresponding voice intent is that they want to have a chat; if a user says "turn on the air conditioner," the corresponding voice intent is that they want to control the car's system to turn on the air conditioner; if a user says "I'm in a good mood today," the corresponding voice intent is that they want to turn on some music to relax; and if a user says "enter chat mode," the corresponding voice intent is that they want to control the car's system to enter chat mode.

[0061] S23. Control strategy for determining the voice interaction mode of the vehicle system based on voice intent.

[0062] Control strategies are pre-defined to correspond to different voice intents and corresponding voice interaction modes. These voice interaction modes can be divided into three categories: casual conversation mode, vehicle system control mode, and chat mode. The control strategy determines which of these modes the voice interaction mode will enter.

[0063] "Chat Mode" refers to a relaxed, friendly, and highly interactive conversational mode switched by the in-vehicle voice interaction system. Users can engage in free and stress-free conversations with the vehicle's infotainment system in this mode. Chat Mode typically does not involve controlling in-vehicle functions but aims to provide users with an entertaining, relaxing, or fatigue-relieving conversational experience. For example, drivers can discuss the weather, share jokes, or chat about everyday matters with the infotainment system.

[0064] The vehicle-mounted control mode refers to a dialogue mode where the in-vehicle voice interaction system focuses on executing control commands issued by the driver. This helps users manage in-vehicle devices and system functions, such as navigation, audio, air conditioning, and telephone functions. In this mode, the voice interaction system primarily focuses on the vehicle's actual operational needs, responding to user-defined commands. For example, the driver can use voice commands to adjust navigation routes, play music, or make phone calls.

[0065] Chat mode is a dialogue mode in the car's voice interaction system designed for simple communication, similar to the chat mode, but with a greater emphasis on simple and casual daily conversations. Users can have short and casual exchanges with the car's system in chat mode, such as asking for information, making comments, or expressing simple emotions.

[0066] Furthermore, after determining the actual voice intent corresponding to the voice command issued by the current user, the control strategy is determined based on the determined voice intent to determine which of the above modes the voice interaction mode should enter.

[0067] S24. Control the voice interaction mode of the vehicle system based on the control strategy.

[0068] If the voice intent is determined to be the intent for the free chat mode, then the control strategy is determined to be to enter the free chat mode;

[0069] If the voice intent is determined to be a vehicle-mounted system control intent, the control strategy is to enter the vehicle-mounted system control mode; if the voice intent is determined to be a casual conversation intent, the control strategy is to enter the casual conversation mode and respond to the chat.

[0070] Optionally, in different modes, if business changes occur, the voice intent corresponding to the user's voice command can be re-evaluated at any time, and then different voice interaction modes can be re-entered to respond to the user based on the voice intent. This is described in detail in the embodiment corresponding to Figure 3.

[0071] The vehicle infotainment system voice interaction mode control method provided in this application embodiment wakes up the voice in response to the user's voice interaction, obtains the user's voice command, performs intent recognition on the voice command to determine the voice intent, determines the control strategy for the vehicle infotainment system's voice interaction mode based on the voice intent, and controls the vehicle infotainment system's voice interaction mode based on the control strategy. Compared with existing automotive intelligent cockpits, which suffer from problems such as a single human-machine interaction dialogue style, difficulty in cross-scenario business transitions, and inability to chat smoothly, this method separates multiple intents of the voice and controls the vehicle infotainment system's voice interaction mode according to different intents. It tightens the dialogue in different voice interaction modes, allowing the vehicle infotainment system not only to execute voice commands but also to seamlessly chat with the user, thus improving the user experience.

[0072] Figure 3 is a flowchart illustrating another voice interaction mode control method for a vehicle infotainment system provided in an embodiment of this application. As shown in Figure 3, the method includes:

[0073] S31. Respond to the user's voice interaction to wake up the voice and obtain the user's voice command.

[0074] In this embodiment, the details are illustrated in Figure 1. The system can be activated by a voice interaction wake-up voice (e.g., "Hello X") issued by a user in any of the four audio zones within the car cabin (driver's seat, front passenger seat, rear left seat, and rear right seat), thus entering the voice interaction system.

[0075] Furthermore, after activating the vehicle's infotainment system and entering the voice interaction system, a voice prompt (e.g., "Hello," "Please speak") can be issued to remind the user to issue further voice commands. Voice commands are specific commands or requests issued to the voice interaction system via voice input; their advantage lies in providing a convenient hands-free operation. The voice interaction system can use speech recognition technology to parse voice commands and execute corresponding operations.

[0076] As a deliberate gesture, after hearing the voice prompt from the vehicle's infotainment system, the user can give a voice command with a specific intention, such as: "Let's chat," "Enter chat mode," "Turn on the air conditioning," or "I'm in a great mood today."

[0077] S32. Pre-train a voice command intent recognition model.

[0078] In illustrative terms, a Speech Command Intent Recognition Model (SCIMM) is a model that combines Natural Language Processing (NLP) and Automatic Speech Recognition (ASR) techniques. It aims to analyze speech commands to identify the user's intent and then execute the corresponding operation. SCIMMs are constructed using at least one of various neural networks, such as Deep Neural Networks (DNNs), Convolutional Neural Networks (CNNs), and Transformers.

[0079] S33. Input the voice command into the voice command intent recognition model and obtain the model output result.

[0080] To illustrate, when inputting a voice command into a voice command intent recognition model, the input voice signal is typically preprocessed first. Preprocessing usually includes steps such as denoising and feature extraction; these steps transform the raw voice signal into a numerical feature representation that the model can understand. Then, the extracted feature representation is input into the voice command intent recognition model, whereby, through multi-layered neural network computation, the model automatically learns the latent patterns and contextual information of the voice command from the feature representation, such as recognizing key information and intent in the voice, and obtaining the model's output.

[0081] S34. Determine the voice intent of the voice command based on the model output results.

[0082] Indicatively, the model output represents the voice intent of a voice command, such as "open the music player" or "set a reminder"; so that voice interaction systems (such as mobile phones, smart home devices, and in-vehicle devices) that deploy voice command intent recognition models can perform corresponding operations based on the model's output, such as launching applications, adjusting device settings, or scheduling tasks.

[0083] By deploying a voice command intent recognition model, voice interaction systems can recognize intents through voice commands, enabling them to understand user needs more efficiently and accurately, and improving the naturalness and efficiency of human-computer interaction. In fields such as smart homes, in-vehicle systems, and intelligent customer service, this can provide users with a smoother and more convenient experience.

[0084] The following provides a unified explanation of S32 to S34:

[0085] In this embodiment, the existing voice intent recognition model can be trained by pre-collecting interactive voice sets in the smart cockpit, so that the model can effectively recognize the voice intent corresponding to the voice commands issued by the user in the smart cockpit environment.

[0086] Furthermore, the voice command issued by the current user is input into the voice command intent recognition model to obtain the model's output, thus determining the actual voice intent of the current user's voice command. For example, if the user says "chat," the corresponding voice intent is that they want to have a chat; if the user says "turn on the air conditioner," the corresponding voice intent is that they want to control the vehicle's system to turn on the air conditioner; if the user says "I'm in a good mood today," the corresponding voice intent is that they want to turn on some music to relax; and if the user says "enter chat mode," the corresponding voice intent is that they want to control the vehicle's system to enter chat mode.

[0087] S35. If the voice intent is determined to be the intent for the chat mode, then the control strategy is determined to be to enter the chat mode.

[0088] In this embodiment, the system can directly enter the chat mode only when the intention is to engage in a chat-like scenario. If the voice intention is determined to be a chat mode intention, the control strategy is set to enter chat mode. A control command is sent to the vehicle's infotainment system to switch the voice interaction mode to chat mode. In chat mode, the system responds to the user's voice through a large model for voice interaction, and maintains the vehicle's voice interaction mode as chat mode. In chat mode, the vehicle's infotainment system can continuously chat with the user, making chatting more engaging. The large model in this embodiment is applicable to all existing large models, such as the Spark Large Model, Wenxin Yiyan, and Pangu Large Model.

[0089] The above content describes the control strategy for entering the "Chat Mode" mode. Since Chat Mode signifies that the voice interaction system can achieve more natural and fluent dialogue with the user, rather than simply responding to commands, it enables easy and interactive human-computer communication by understanding the user's intent. Entering Chat Mode allows users to experience a more human-like interaction, enhancing the overall user experience. Furthermore, flexibly entering Chat Mode also makes the voice interaction system more intelligent and flexible, no longer passively performing single tasks, but actively understanding and participating in more complex conversations, thus adapting to more diverse needs and increasing user engagement and device usage frequency.

[0090] S36. If the voice intent is determined to be a vehicle-mounted system service control intent, then the control strategy is determined to be to enter the vehicle-mounted system control mode.

[0091] In this embodiment, if the voice intent is determined to be a vehicle-mounted system service control intent, the control strategy is determined to enter the vehicle-mounted system control mode. If the control strategy is determined to be entering the vehicle-mounted system control mode, the corresponding service control is performed on the vehicle-mounted system in response to the voice command, processed according to normal logic, and the chat mode is tightened.

[0092] In vehicle control mode, after the user issues a business command, the system switches to the main scenario's tightened chat mode. Compared to the initial dialogue, this mode identifies the user's intent to avoid frequent accidental chat triggers after issuing a business command. The system then acquires a second round of voice commands from the user, performs intent recognition on these commands, and determines the intent. If the second round of voice commands is determined to be a chat mode intent, the vehicle's voice interaction mode is switched to chat mode. If the second round of voice commands is determined to be a business control intent, the system responds to the second round of voice commands to perform the corresponding business control, processes the data according to normal logic, and maintains the tightened chat mode. If the second round of voice commands is determined to be a casual chat intent, the system does not respond to chat and maintains the tightened chat mode.

[0093] For example, if a user issues a voice command to "turn on the air conditioning," the vehicle's infotainment system will respond and turn on the air conditioning. If the user then says to someone else in the car, "It's so hot today," the system can determine that the voice intent is casual conversation and will not respond to that conversation.

[0094] The above content describes the process of entering the vehicle control mode under the control intent of the vehicle's infotainment system. Considering that in vehicle-to-vehicle interaction, if the driver is driving, their attention must be focused on the road; entering the vehicle control mode allows the driver to efficiently complete various functional tasks via voice commands without manual operation. This process also helps meet the specific needs of car owners in the vehicle, improves driving safety, optimizes the driving experience, and enhances operational efficiency and system intelligence, providing a more convenient and comfortable driving environment. Simultaneously, by tightening the chat mode, it also avoids mutual interference between different modes, weakening the chat mode to highlight the current vehicle control mode and improving the accuracy of mode application.

[0095] Furthermore, based on the second round of voice commands, the system can dynamically switch operating modes according to user needs. This intelligent switching allows the system to flexibly adapt to different scenarios, providing users with appropriate interaction methods whether it's for work tasks or casual conversation, thus improving the convenience and comfort of interaction. This design makes the in-vehicle system not only efficient but also entertaining and user-friendly; it also enhances driving safety and increases user engagement to a certain extent.

[0096] S37. If the voice intent is determined to be a chat-like intent, then the control strategy is determined to be responding to chat.

[0097] If the voice intent is determined to be a casual conversation intent, then the control strategy is determined to be to respond to chat, and to reply to the voice command through a large model.

[0098] It should be noted that in any mode, once the user says "enter chat mode," the system switches to chat mode. Compared to the first round of dialogue, business-related statements can be executed according to the customized functions of each OEM, while chat-like statements are responded to by the larger model, allowing for greater freedom of expression. In the third or Nth round, functions triggered by the in-vehicle voice system are also categorized at the same level. If a business-related statement requires a jump to execution, it will jump to execution and activate a tighter chat mode; if a business-related statement requires direct text-to-speech (TTS) playback, it will be played directly and chat mode will remain active; if a business-related statement requires a fallback response, a fallback response will be provided and chat mode will remain active; if it is still a casual conversation statement, it will continue to respond to casual conversation statements and maintain chat mode.

[0099] The above content describes the control strategy for responding to chat under the intent of casual conversation. Considering that casual conversation intents typically represent a user's desire for a relaxed and enjoyable conversation with the device, recognizing and responding to the user's casual conversation not only satisfies the user's entertainment needs but also allows the user to experience the device's friendliness and intelligence. This interaction makes the device more human-like and enhances the overall user experience. By providing a relaxed conversational environment, especially during long drives or waiting times, users can release stress through conversation with the device, thus becoming more relaxed and comfortable, achieving a richer and more comfortable interactive experience.

[0100] In some embodiments, the voice interaction system also has a self-learning evolution function: after entering multiple rounds of historical inheritance, the content of casual conversation and the user's multiple interactions are entered into a certain scenario. By building a negative feedback model, the system continuously learns and adjusts the user's speech and habits, injects them into a large model for historical data analysis, and ultimately reduces the user's interaction levels to achieve the user's purpose.

[0101] In a schematic manner, historical human-machine interaction voice recordings under vehicle control mode are acquired; the historical human-machine interaction voice recordings are analyzed to obtain the user's vehicle control habits; based on the vehicle control habits, the system responds to the user's voice commands.

[0102] Historical human-machine interaction voice is the voice interaction between the vehicle system and the user within a historical time period. The historical time period is a pre-selected time period, such as the most recent hour, the most recent day, the most recent week, etc. By analyzing the historical human-machine interaction voice, the user's vehicle system control habits can be determined.

[0103] In-vehicle infotainment system (IVR) control habits refer to the specific behavioral patterns, preferences, and operating habits exhibited by users when interacting with the in-vehicle system while driving. These habits are formed through long-term user interaction with the IVR system, including voice commands, manual operations, and touchscreen interactions, and are manifested in frequently used settings or preferred operating methods for specific functions. For example, users may frequently adjust the air conditioning temperature or make voice calls to specific contacts while driving; or they may have fixed operating paths for certain shortcuts on the IVR interface, such as quickly opening certain applications, adjusting screen brightness, or changing the navigation view; or, each time a user activates IVR control by voice, they first enter the main interface, then select lower-level interfaces layer by layer, finally choosing the playlist page, etc.

[0104] Optionally, after analyzing the user's historical habits, if it is determined that the user always opens the playlist page first, the system can directly jump to the playlist page the next time the user opens the vehicle control system by voice, reducing the user interaction layers and achieving an efficient response to user voice commands based on the vehicle control system's habits.

[0105] Understanding the user's in-vehicle control habits can provide a more personalized voice interaction experience, improve the smoothness and accuracy of the interaction, and allow the system to automatically adjust the response based on the user's habits without requiring the user to perform the same operation multiple times. This reduces operation steps and time consumption, significantly improves operational efficiency, and reduces the problem of misoperation. It can also enhance user stickiness and the frequency of voice interaction to a certain extent.

[0106] Through the above multi-level control, the system eventually enters the self-learning evolution stage, making significant distinctions from the source. This effectively separates multiple user intentions, ensuring that voice chat is always available, greatly reducing the probability of accidental triggering, and completely breaking down the barriers between business and chat, making it more convenient for users.

[0107] The vehicle infotainment system voice interaction mode control method provided in this application embodiment wakes up the voice in response to the user's voice interaction, obtains the user's voice command, performs intent recognition on the voice command to determine the voice intent, determines the control strategy for the vehicle infotainment system's voice interaction mode based on the voice intent, and controls the vehicle infotainment system's voice interaction mode based on the control strategy. Compared with existing automotive intelligent cockpits, which suffer from problems such as a single human-machine interaction dialogue style, difficulty in cross-scenario business transitions, and inability to chat smoothly, this method separates multiple intents of the voice and controls the vehicle infotainment system's voice interaction mode according to different intents. It tightens the dialogue in different voice interaction modes, allowing the vehicle infotainment system not only to execute voice commands but also to seamlessly chat with the user, thus improving the user experience.

[0108] Figure 4 is a structural schematic diagram of a voice interaction mode control device for a vehicle-mounted system provided in an embodiment of this application. As shown in Figure 4, it includes:

[0109] The response acquisition module 401 is used to respond to the user's voice interaction wake-up voice and acquire the user's voice command.

[0110] The recognition and determination module 402 is used to perform intent recognition on the voice command and determine the voice intent.

[0111] The identification and determination module 402 is also used to determine the control strategy of the vehicle's voice interaction mode based on the voice intent;

[0112] The control module 403 is used to control the voice interaction mode of the vehicle system based on the control strategy.

[0113] In one possible implementation, the recognition and determination module 402 is used to pre-train a voice command intent recognition model; input the voice command into the voice command intent recognition model to obtain the model output result; and determine the voice intent of the voice command based on the model output result.

[0114] In one possible implementation, the identification and determination module 402 is further configured to determine the control strategy as entering the chat mode if the voice intent is determined to be a chat mode intent; determine the control strategy as entering the vehicle control mode if the voice intent is determined to be a vehicle system service control intent; and determine the control strategy as responding to chat if the voice intent is determined to be a casual chat intent.

[0115] In one possible implementation, the control module 403 is configured to, if it is determined that the control strategy is to enter the chat mode, control the voice interaction mode of the vehicle system to enter the chat mode; in the chat mode, the vehicle system responds to the user's voice through a large model to perform voice interaction, and maintains the voice interaction mode of the vehicle system as the chat mode.

[0116] In one possible implementation, the control module 403 is further configured to, if it is determined that the control strategy is to enter the vehicle control mode, respond to the voice command to perform corresponding business control on the vehicle and tighten the chat mode; if it is determined that the control strategy is to respond to chat, reply to the voice command through a large model.

[0117] In one possible implementation, the control module 403 is further configured to: acquire a second round of voice commands issued by the user in the vehicle control mode; perform intent recognition on the second round of voice commands to determine the intent of the second round of voice commands; if the intent of the second round of voice commands is determined to be a chat mode intent, then control the voice interaction mode of the vehicle to enter the chat mode; if the intent of the second round of voice commands is determined to be a vehicle business control intent, then respond to the second round of voice commands to perform corresponding business control on the vehicle and maintain the tight chat mode; if the intent of the second round of voice commands is determined to be a casual chat intent, then do not respond to chat and maintain the tight chat mode.

[0118] In one possible implementation, the control module 403 is further configured to acquire historical human-machine interaction voices under vehicle control mode; analyze the historical human-machine interaction voices to obtain the user's vehicle control habits; and respond to the user's voice commands based on the vehicle control habits.

[0119] The vehicle infotainment system voice interaction mode control device provided in this embodiment can be the vehicle infotainment system voice interaction mode control device shown in Figure 4. It can execute all the steps of the vehicle infotainment system voice interaction mode control method shown in Figure 2-3, thereby achieving the technical effect of the vehicle infotainment system voice interaction mode control method shown in Figure 2-3. Please refer to the relevant description in Figure 2-3. For the sake of brevity, it will not be elaborated here.

[0120] Figure 5 is a schematic diagram of an electronic device provided in an embodiment of this application. The electronic device 500 shown in Figure 5 includes: at least one processor 501, a memory 502, at least one network interface 504, and other user interfaces 503. The various components in the electronic device 500 are coupled together through a bus system 505. It is understood that the bus system 505 is used to realize the connection and communication between these components. In addition to a data bus, the bus system 505 also includes a power bus, a control bus, and a status signal bus. However, for clarity, all buses are labeled as bus system 505 in Figure 5.

[0121] The user interface 503 may include a display, keyboard, or clicking device (e.g., mouse, trackball, touchpad, or touchscreen).

[0122] It is understood that the memory 502 in the embodiments of this application can be volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. The non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. The volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of RAM are available, such as Static Random Access Memory (SRAM), Dynamic Random Access Memory (DRAM), Synchronous DRAM (SDRAM), Double Data Rate SDRAM (DDR SDRAM), Enhanced Synchronous DRAM (ESDRAM), Synchlink DRAM (SLDRAM), and Direct Rambus RAM (DRRAM). The memory 502 described herein is intended to include, but is not limited to, these and any other suitable types of memory.

[0123] In some implementations, memory 502 stores elements, executable units or data structures, or subsets thereof, or extended sets thereof: operating system 5021 and application program 5022.

[0124] The operating system 5021 includes various system programs, such as the framework layer, core library layer, and driver layer, used to implement various basic business functions and handle hardware-based tasks. The application program 5022 includes various applications, such as a media player and a browser, used to implement various application functions. Programs implementing the methods of this application embodiment can be included in application program 5022.

[0125] In this embodiment of the application, by calling the program or instructions stored in the memory 502 (which may be the program or instructions stored in the application program 5022), the processor 501 is used to execute the method steps provided in each method embodiment, such as: responding to a user's voice interaction to wake up the voice, obtaining the user's voice command; performing intent recognition on the voice command to determine the voice intent; determining a control strategy for the vehicle's voice interaction mode based on the voice intent; and controlling the vehicle's voice interaction mode based on the control strategy.

[0126] In one possible implementation, a voice command intent recognition model is pre-trained; the voice command is input into the voice command intent recognition model to obtain the model output result; and the voice intent of the voice command is determined based on the model output result.

[0127] In one possible implementation, if the voice intent is determined to be a chat mode intent, then the control strategy is determined to enter chat mode; if the voice intent is determined to be a vehicle system service control intent, then the control strategy is determined to enter vehicle system control mode; if the voice intent is determined to be a casual chat intent, then the control strategy is determined to respond to chat.

[0128] In one possible implementation, if the control strategy is determined to be to enter the chat mode, then the voice interaction mode of the vehicle system is controlled to enter the chat mode; in the chat mode, the large model responds to the user's voice to perform voice interaction, and the voice interaction mode of the vehicle system is kept in the chat mode.

[0129] In one possible implementation, if the control strategy is determined to be entering the vehicle control mode, then the corresponding business control is performed on the vehicle in response to the voice command, and the chat mode is tightened; if the control strategy is determined to be responding to chat, then the voice command is replied to through a large model.

[0130] In one possible implementation, under the vehicle control mode, a second round of voice commands issued by the user is acquired; the intent of the second round of voice commands is recognized to determine the intent of the second round of voice commands; if the intent of the second round of voice commands is determined to be a chat mode intent, the voice interaction mode of the vehicle is controlled to enter the chat mode; if the intent of the second round of voice commands is determined to be a vehicle business control intent, the corresponding business control of the vehicle is performed in response to the second round of voice commands, and the chat mode is maintained; if the intent of the second round of voice commands is determined to be a casual chat intent, the chat is not responded to and the chat mode is maintained.

[0131] In one possible implementation, historical human-machine interaction voice recordings under vehicle control mode are acquired; the historical human-machine interaction voice recordings are analyzed to obtain the user's vehicle control habits; and based on the vehicle control habits, the user's voice commands are responded to.

[0132] The methods disclosed in the embodiments of this application described above can be applied to processor 501, or implemented by processor 501. Processor 501 may be an integrated circuit chip with signal processing capabilities.

[0133] In implementation, each step of the above method can be completed by the integrated logic circuits in the hardware of the processor 501 or by instructions in software form. The processor 501 can be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of this application can be directly embodied in the execution of a hardware decoding processor, or executed by a combination of hardware and software units in the decoding processor. The software units can be located in random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. This storage medium is located in memory 502, and the processor 501 reads the information in memory 502 and, in conjunction with its hardware, completes the steps of the above method.

[0134] It is understood that the embodiments described herein can be implemented in hardware, software, firmware, middleware, microcode, or a combination thereof. For hardware implementation, the processing unit can be implemented in one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), general-purpose processors, controllers, microcontrollers, microprocessors, other electronic units for performing the functions described herein, or combinations thereof.

[0135] For software implementation, the techniques described herein can be implemented by units that perform the functions described herein. The software code can be stored in memory and executed by a processor. The memory can be implemented in the processor or external to the processor.

[0136] The electronic device provided in this embodiment can be the electronic device shown in Figure 5, which can execute all the steps of the voice interaction mode control method of the vehicle system in Figure 2-3, thereby achieving the technical effect of the voice interaction mode control method of the vehicle system shown in Figure 2-3. Please refer to the relevant description in Figure 2-3. For the sake of brevity, it will not be elaborated here.

[0137] This application also provides a storage medium (computer-readable storage medium). This storage medium stores one or more programs. The storage medium may include volatile memory, such as random access memory; it may also include non-volatile memory, such as read-only memory, flash memory, hard disk, or solid-state drive; and it may also include combinations of the above types of memory.

[0138] When one or more programs in the storage medium can be executed by one or more processors to implement the above-mentioned vehicle voice interaction mode control method executed on the electronic device side.

[0139] The processor is used to execute the vehicle infotainment system's voice interaction mode control program stored in the memory to implement the following steps of the vehicle infotainment system's voice interaction mode control method executed on the electronic device side:

[0140] In response to a user's voice interaction wake-up voice, the system acquires the user's voice command; it performs intent recognition on the voice command to determine the voice intent; it determines a control strategy for the vehicle's voice interaction mode based on the voice intent; and it controls the vehicle's voice interaction mode based on the control strategy.

[0141] In one possible implementation, a voice command intent recognition model is pre-trained; the voice command is input into the voice command intent recognition model to obtain the model output result; and the voice intent of the voice command is determined based on the model output result.

[0142] In one possible implementation, if the voice intent is determined to be a chat mode intent, then the control strategy is determined to enter chat mode; if the voice intent is determined to be a vehicle system service control intent, then the control strategy is determined to enter vehicle system control mode; if the voice intent is determined to be a casual chat intent, then the control strategy is determined to respond to chat.

[0143] In one possible implementation, if the control strategy is determined to be to enter the chat mode, then the voice interaction mode of the vehicle system is controlled to enter the chat mode; in the chat mode, the large model responds to the user's voice to perform voice interaction, and the voice interaction mode of the vehicle system is kept in the chat mode.

[0144] In one possible implementation, if the control strategy is determined to be entering the vehicle control mode, then the corresponding business control is performed on the vehicle in response to the voice command, and the chat mode is tightened; if the control strategy is determined to be responding to chat, then the voice command is replied to through a large model.

[0145] In one possible implementation, under the vehicle control mode, a second round of voice commands issued by the user is acquired; the intent of the second round of voice commands is recognized to determine the intent of the second round of voice commands; if the intent of the second round of voice commands is determined to be a chat mode intent, the voice interaction mode of the vehicle is controlled to enter the chat mode; if the intent of the second round of voice commands is determined to be a vehicle business control intent, the corresponding business control of the vehicle is performed in response to the second round of voice commands, and the chat mode is maintained; if the intent of the second round of voice commands is determined to be a casual chat intent, the chat is not responded to and the chat mode is maintained.

[0146] In one possible implementation, historical human-machine interaction voice recordings under vehicle control mode are acquired; the historical human-machine interaction voice recordings are analyzed to obtain the user's vehicle control habits; and based on the vehicle control habits, the user's voice commands are responded to.

[0147] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0148] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein can be implemented in hardware, a software module executed by a processor, or a combination of both. The software module can be located in random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, compact disc read-only memory (CD-ROM), or any other form of storage medium known in the art.

[0149] The above-described embodiments further illustrate the purpose, technical solution, and beneficial effects of this application. It should be understood that the above descriptions are merely embodiments of this application and are not intended to limit the scope of protection of this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of protection of this application.

Claims

1. A method for controlling a vehicle's voice interaction mode, executed by an electronic device, comprising: It responds to user voice interaction to wake up the voice and obtains the user's voice commands; The voice command is subjected to intent recognition to determine the voice intent; A control strategy for determining the voice interaction mode of the vehicle system based on the stated voice intent; The voice interaction mode of the vehicle system is controlled based on the aforementioned control strategy.

2. The method according to claim 1, wherein, The step of performing intent recognition on the voice command and determining the voice intent includes: Pre-train a voice command intent recognition model; The voice command is input into the voice command intent recognition model to obtain the model output result; The voice intent of the voice command is determined based on the output of the model.

3. The method according to claim 1 or 2, wherein, The control strategy for determining the voice interaction mode of the vehicle system based on the voice intent includes: If the voice intent is determined to be a chat mode intent, then the control strategy is determined to be to enter chat mode; If the voice intent is determined to be a vehicle-mounted system service control intent, then the control strategy is determined to be to enter the vehicle-mounted system control mode; If the voice intent is determined to be a casual conversation intent, then the control strategy is determined to be a chat response.

4. The method according to any one of claims 1 to 3, wherein, The control of the voice interaction mode of the vehicle system based on the control strategy includes: If the control strategy is determined to be to enter the chat mode, then the voice interaction mode of the vehicle system is controlled to enter the chat mode. In the "Chat Mode", the system responds to the user's voice through a large model to conduct voice interaction, and keeps the voice interaction mode of the vehicle system in "Chat Mode".

5. The method according to any one of claims 1 to 4, wherein, The method further includes: If the control strategy is determined to be entering the vehicle control mode, then the corresponding business control is performed on the vehicle in response to the voice command, and the chat mode is tightened. If the control strategy is determined to be a chat response, then the voice command is replied to through a large model.

6. The method according to any one of claims 1 to 5, wherein, The method further includes: In the vehicle control mode, the second round of voice commands issued by the user are acquired; Perform intent recognition on the second round of voice commands to determine the intent of the second round of voice commands; If it is determined that the second round of voice intent is a chat mode intent, then the voice interaction mode of the vehicle system is controlled to enter chat mode; If it is determined that the second round of voice intent is a vehicle system business control intent, then respond to the second round of voice command to perform the corresponding business control on the vehicle system, and maintain the tightened chat mode; If the second round of voice intent is determined to be a casual conversation intent, then do not respond to chat and maintain the tight chat mode.

7. The method according to any one of claims 1 to 6, wherein, The method further includes: Acquire historical human-machine interaction voice recordings under vehicle control mode; The historical human-machine interaction voices are analyzed to obtain the user's vehicle control habits; Based on the aforementioned vehicle control habits, it responds to voice commands issued by the user.

8. A voice interaction mode control device for a vehicle infotainment system, comprising: The response acquisition module is used to respond to the user's voice interaction wake-up voice and acquire the user's voice commands. The recognition and determination module is used to perform intent recognition on the voice command and determine the voice intent. The identification and determination module is also used to determine the control strategy of the vehicle's voice interaction mode based on the voice intent; The control module is used to control the voice interaction mode of the vehicle system based on the control strategy.

9. An electronic device, comprising: A processor and a memory, the processor being configured to execute a vehicle infotainment system voice interaction mode control program stored in the memory to implement the vehicle infotainment system voice interaction mode control method according to any one of claims 1 to 7.

10. A storage medium storing one or more programs, the one or more programs being executable by one or more processors to implement the voice interaction mode control method for a vehicle infotainment system according to any one of claims 1 to 7.

Images