Voice interaction method and system based on intelligent vehicle-mounted entertainment screen, terminal and storage medium

Abstract

The invention discloses a voice interaction method and system based on an intelligent vehicle-mounted entertainment screen, a terminal and a storage medium, and the method comprises the steps: abstracting a travel service module, a cabin entertainment module and a vehicle control module into an interactive agent MCP protocol module based on an agent technology; acquiring a voice instruction of a user based on voice recognition, performing intention analysis on the voice instruction by using a large model, and recognizing a deep intention of the user; inputting the deep intention into the agent MCP protocol module, and performing task decomposition on the deep intention by the agent MCP protocol module to obtain one or more subtasks; and the agent MCP protocol module performs task execution according to one or more subtasks to obtain a task execution result. The method has the inference capability, the planning capability and the user task execution capability, can identify the deep intention of the user, improves the voice interaction experience of the user, and reduces the operation complexity.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of voice interaction, and in particular to a voice interaction method and system based on an intelligent vehicle-mounted entertainment screen, a terminal and a computer readable storage medium. BACKGROUND

[0002] With the progress of intelligent cockpit technology and the development of the times, consumers have increasingly high requirements for the intelligent cockpit system of a vehicle. In order to improve the use experience of consumers, vehicle manufacturers have added a screen with an intelligent system to the vehicle, which generally supports functions such as cockpit entertainment, travel services, vehicle control management, etc. In the case of increasingly refined and complex functions, the operation of users during driving has an increasingly large impact on driving safety.

[0003] Many intelligent systems support voice interaction functions, and the voice interaction functions can only meet one or more explicit instruction executions. Even though some current voice systems have accessed large models, they can only support more rich docking and script generation, and lack the following core defects in intelligent experience: the user's demand or instruction cannot be reasoned, if the user's demand instruction is not an explicit instruction, only a reply cannot be understood or a practicality-low bottom reply can be made; the task decomposition based on a general model cannot effectively align the function range that can be covered by the cockpit system, and it is easy to appear the case that the decomposed task cannot be served and satisfied in the cockpit (i.e., the task planning and system service provision do not match); the entertainment, travel and vehicle control modules in the cockpit cannot be organically linked and matched with the user's demand, and the intelligent body task decomposition reasoning (i.e., the user's input demand is decomposed into a specific task executed by the system, such as the user saying "plan a trip", which is decomposed into sub-tasks of weather, hotel, ticket, trip guide) cannot be dynamically adjusted and adapted.

[0004] Therefore, the prior art still needs to be improved and developed. SUMMARY

[0005] The main purpose of the present application is to provide a voice interaction method and system based on an intelligent vehicle-mounted entertainment screen, a terminal and a computer readable storage medium, which aims to solve the problem in the prior art that the function modules of the current intelligent cockpit system are independent of each other, the user needs to switch between different interfaces or perform multiple operations to complete a single demand, resulting in complicated operation and low efficiency, the user's deep intention cannot be recognized, and the voice interaction requirements of the user cannot be met.

[0006] To achieve the above purpose, the present application provides a voice interaction method based on an intelligent vehicle-mounted entertainment screen, which comprises the following steps:

[0007] Based on the agent technology, the travel service module, the cabin entertainment module and the vehicle control module are abstracted as the interactive agent MCP protocol module;

[0008] Based on voice recognition, the voice instruction of the user is collected, and a large model is used for intent analysis on the voice instruction to identify the deep intention of the user.

[0009] The deep intention is input into the agent MCP protocol module, and the agent MCP protocol module performs task decomposition on the deep intention to obtain one or more sub-tasks.

[0010] The agent MCP protocol module performs task execution according to one or more sub-tasks to obtain a task execution result.

[0011] Optionally, the voice interaction method based on the intelligent vehicle entertainment screen, wherein the agent MCP protocol module performs task execution according to one or more sub-tasks to obtain a task execution result, and then comprises:

[0012] The task execution result is fed back to the user through natural language generation technology, and the user is supported to correct the instruction through voice, touch or interface.

[0013] Optionally, the voice interaction method based on the intelligent vehicle entertainment screen, wherein the sub-tasks include travel service tasks, cabin entertainment tasks and vehicle control tasks.

[0014] Optionally, the voice interaction method based on the intelligent vehicle entertainment screen, wherein the task execution result includes travel service task execution result, cabin entertainment task execution result and vehicle control task execution result.

[0015] Optionally, the voice interaction method based on the intelligent vehicle entertainment screen, wherein the agent MCP protocol module performs task execution according to one or more sub-tasks to obtain a task execution result, and specifically comprises:

[0016] When the sub-task is a travel service task, the travel service module in the agent MCP protocol module calls the navigation function and obtains the route information to obtain the travel service task execution result.

[0017] When the sub-task is a cabin entertainment task, the cabin entertainment module in the agent MCP protocol module recommends music or video according to the user's emotion or scene to obtain the cabin entertainment task execution result.

[0018] When the sub-task is a vehicle control task, the vehicle control module in the agent MCP protocol module adjusts the air conditioner temperature, seat heating or window state to obtain the vehicle control task execution result.

[0019] Optionally, the voice interaction method based on the intelligent vehicle-mounted entertainment screen further comprises:

[0020] If the deep intention of the user cannot be determined, a task planning strategy is generated by using a large model inference based on user historical behavior data, real-time scene information and context information.

[0021] Optionally, the voice interaction method based on the intelligent vehicle-mounted entertainment screen further comprises:

[0022] When the sub-task execution fails, the intelligent agent MCP protocol module prompts the user to re-input instructions or provides alternative alternatives.

[0023] In addition, the present application also provides a voice interaction system based on an intelligent vehicle-mounted entertainment screen, wherein the voice interaction system based on the intelligent vehicle-mounted entertainment screen comprises:

[0024] A voice interaction module is configured to collect voice instructions of a user.

[0025] A large model is configured to analyze the voice instructions and identify the deep intention of the user.

[0026] An intelligent agent MCP protocol module comprises a travel service module, a cabin entertainment module and a vehicle control module, and is configured to decompose the deep intention into one or more sub-tasks, perform the sub-tasks and obtain a task execution result.

[0027] In addition, the present application also provides a terminal, wherein the terminal comprises a memory, a processor and a voice interaction program based on an intelligent vehicle-mounted entertainment screen stored in the memory and executable on the processor, and the voice interaction program based on the intelligent vehicle-mounted entertainment screen implements the steps of the voice interaction method based on the intelligent vehicle-mounted entertainment screen when executed by the processor.

[0028] In addition, the present application also provides a computer readable storage medium, wherein the computer readable storage medium stores a voice interaction program based on an intelligent vehicle-mounted entertainment screen, and the voice interaction program based on the intelligent vehicle-mounted entertainment screen implements the steps of the voice interaction method based on the intelligent vehicle-mounted entertainment screen when executed by a processor.

[0029] In the present application, based on the agent technology, the travel service module, the cabin entertainment module and the vehicle control module are abstracted as interactive agent MCP protocol modules; the voice recognition is used to collect the voice instruction of the user, the large model is used for intent analysis on the voice instruction, the deep intention of the user is recognized; the deep intention is input to the agent MCP protocol module, the agent MCP protocol module performs task decomposition on the deep intention, and one or more sub-tasks are obtained; the agent MCP protocol module performs task execution according to one or more sub-tasks, and obtains a task execution result. The present application uses the voice interaction technology, combines the large model and the agent technology, gives the ability to understand the user's intention, has the reasoning ability, the planning ability and the ability to execute the user's task, can recognize the deep intention of the user, improves the voice interaction experience of the user, and reduces the operation complexity. BRIEF DESCRIPTION OF DRAWINGS

[0030] Figure 1 is a flowchart of a preferred embodiment of the voice interaction method based on the intelligent vehicle-mounted entertainment screen of the present application;

[0031] Figure 2 is a schematic diagram of the whole process of the voice interaction of the intelligent vehicle-mounted entertainment screen based on the voice interaction, the agent technology and the large model in the preferred embodiment of the voice interaction method based on the intelligent vehicle-mounted entertainment screen of the present application;

[0032] Figure 3 is a structure diagram of a preferred embodiment of the voice interaction system based on the intelligent vehicle-mounted entertainment screen of the present application. DETAILED DESCRIPTION

[0033] In order to make the purpose, technical scheme and advantages of the present application more clear and definite, the present application is further described in detail below with reference to the drawings and examples. It should be understood that the specific embodiments described herein are only used to explain the present application, and are not used to limit the present application.

[0034] The voice interaction method based on the intelligent vehicle-mounted entertainment screen according to the preferred embodiment of the present application, as shown in Figure 1 and Figure 2 , the voice interaction method based on the intelligent vehicle-mounted entertainment screen comprises the following steps:

[0035] Step S10, based on the agent technology, the travel service module, the cabin entertainment module and the vehicle control module are abstracted as interactive agent MCP protocol modules.

[0036] Specifically, agent technology is a key branch of artificial intelligence, focusing on designing intelligent entities (called agents) that can autonomously perceive their environment, make decisions, and execute actions. Its core goal is to equip these entities with anthropomorphic capabilities, such as reasoning, learning, collaboration, and adapting to dynamic environments. The travel service module: invokes navigation functions and obtains route information; the in-cabin entertainment module: recommends music or videos based on user emotions or scenarios; and the vehicle control module: adjusts air conditioning temperature, seat heating, or window status. This invention introduces agent technology, abstracting the travel service module, in-cabin entertainment module, and vehicle control module into interactive agent MCP protocol modules, each with an independent interface (e.g., travel service MCP, in-cabin entertainment MCP, and vehicle control MCP). The agent MCP protocol (Model Context Protocol) is an open source protocol launched by Anthropic, designed to provide a standardized, secure, and bidirectional integration framework for large language models (LLMs) with external data sources and tools. Modules can dynamically collaborate through intent commands generated by the large model.

[0037] Step S20: Collect the user's voice commands based on voice recognition, use the big model to perform intent analysis on the voice commands, and identify the user's deep intention.

[0038] Specifically, the voice interaction module collects the user's voice commands (such as "play music and dim the lights"), and uses a large model (such as a large language model (LLM), which acquires the ability to understand and generate natural language through massive data training; or GPT) to perform intent analysis on the voice commands and identify the user's deep intentions, that is, it not only recognizes single linear commands, but also can handle complex needs, such as "navigate to the nearest gas station and turn on the air conditioner."

[0039] Step S30: input the deep intention into the agent MCP protocol module, and the agent MCP protocol module decomposes the deep intention into tasks to obtain one or more subtasks.

[0040] Specifically, the deep intention is input into the intelligent agent MCP protocol module, and the intelligent agent MCP protocol module decomposes the user's deep intention into specific subtasks. The number of subtasks is the same as the actual number of user needs. The subtasks include travel service tasks, cabin entertainment tasks and vehicle control tasks.

[0041] Step S40: The agent MCP protocol module executes the task according to one or more subtasks to obtain a task execution result.

[0042] Specifically, the task execution results include the travel service task execution results, the cabin entertainment task execution results and the vehicle control task execution results; when the subtask is a travel service task, the travel service module in the intelligent agent MCP protocol module calls the navigation function and obtains the route information to obtain the travel service task execution results; when the subtask is a cabin entertainment task, the cabin entertainment module in the intelligent agent MCP protocol module recommends music or video according to the scene to obtain the cabin entertainment task execution results; when the subtask is a vehicle control task, the vehicle control module in the intelligent agent MCP protocol module adjusts the air-conditioning temperature, seat heating or window status to obtain the vehicle control task execution results.

[0043] For example, the compound instruction "navigate to the nearest gas station and turn on the air conditioner" will be parsed into two independent tasks: calling the travel service module to plan the route, and triggering the vehicle control module to adjust the air conditioner temperature.

[0044] If the user's underlying intent is unclear, large-scale model reasoning can be used to generate task planning strategies based on historical user behavior data (e.g., user preferences, frequented locations), real-time scenario information (e.g., time, weather, driving patterns), and contextual information. For example, the system can automatically switch to "rest mode"—dimming the lights, playing soothing music, and suggesting a stop for a rest. For example, if the user says, "I want to take a break," the system can automatically switch to "rest mode," dimming the lights, turning off the air conditioning, playing soothing music, and suggesting a stop for a rest. For complex requests (e.g., "Help me find a restaurant with parking and reserve a table"), the system can break down the task into multiple subtasks, which are then distributed to the mobility services, in-cabin entertainment, and vehicle control modules for collaborative completion. The mobility services agent searches for the target location, the in-cabin entertainment agent recommends music based on the user's mood, and the vehicle control agent adjusts the seat angle and air conditioning temperature. Finally, the system provides feedback to the user through multimodal interaction (voice + touch) on the execution results (e.g., "We have found three restaurants that meet the requirements. Do you want to select 'XX restaurant' and make a reservation?"). If a module cannot complete the task (for example, there is no available parking lot), the system will trigger a backup plan (such as recommending nearby parking lots and adjusting the route).

[0045] like Figure 2 As shown, the system service is responsible for integrating the core functions within the cockpit system and providing users with a complete service experience, which is divided into travel services, cockpit entertainment services and vehicle control services.

[0046] In combination with voice, touch and other interaction modes, the natural language generation technology is used to feed back the task execution result to the user, and the user can correct the instruction (for example, "find a coffee shop with parking") or confirm the task execution state (for example, "the navigation route has been updated") through voice, touch or interface (head-up display, HUD), that is, the communication with the user is realized through various interaction modes such as voice, touch and the like. When the sub-task execution fails, the intelligent agent MCP protocol module prompts the user to re-input the instruction or provides an alternative solution.

[0047] The key innovation of the present application is:

[0048] (1) The intention analysis and task planning driven by a large model can process complex instructions and dynamically adapt to scenarios.

[0049] (2) Cross-module collaboration of the intelligent agent MCP protocol realizes efficient collaboration between modules through standardized interfaces.

[0050] (3) Multi-modal interaction closed loop, combined with voice, touch and visual feedback to improve the fluency of interaction, significantly reduce the complexity of user operation, improve the intelligent level and scenario adaptability of the cockpit system, and provide a new solution for the field of intelligent cockpit of automobile.

[0051] The present application aims to solve the problems of fragmented functions, complex operation and insufficient intelligence of existing intelligent cockpit systems by integrating voice interaction, intention understanding and multi-module collaboration capabilities. The core is to use a large model to deeply analyze user instructions and realize dynamic collaboration across modules through the intelligent agent MCP protocol. The present application can integrate multi-module functions and provide an intelligent in-vehicle entertainment screen system with autonomous reasoning and planning capabilities to improve user experience and reduce operation complexity.

[0052] Further, as shown in Figure 3 Based on the above voice interaction method based on an intelligent in-vehicle entertainment screen, the present application also correspondingly provides a voice interaction system based on an intelligent in-vehicle entertainment screen, wherein the voice interaction system based on an intelligent in-vehicle entertainment screen comprises:

[0053] A voice interaction module for collecting voice instructions of a user;

[0054] A large model for analyzing the intention of the voice instructions and identifying the deep intention of the user;

[0055] An intelligent agent MCP protocol module including a travel service module, a cockpit entertainment module and a vehicle control module for task decomposition of the deep intention, obtaining one or more sub-tasks, and performing tasks according to one or more sub-tasks to obtain a task execution result.

[0056] Further, based on the voice interaction method and system based on the intelligent vehicle entertainment screen, the application also provides a terminal, which comprises a processor, a memory and a display.

[0057] The memory can be an internal storage unit of the terminal, such as a hard disk or a memory of the terminal, in some embodiments. The memory can also be an external storage device of the terminal, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card and the like, in other embodiments. Further, the memory can comprise both the internal storage unit and the external storage device of the terminal. The memory is used to store application software and various data installed on the terminal, such as program codes of the terminal and the like. The memory can also be used to temporarily store data that has been output or is to be output. In an embodiment, the memory stores a voice interaction program based on the intelligent vehicle entertainment screen, which can be executed by the processor, so as to implement the voice interaction method based on the intelligent vehicle entertainment screen in the application.

[0058] The processor can be a central processing unit (CPU), a microprocessor or other data processing chip, which is used to run program codes or process data stored in the memory, such as executing the voice interaction method based on the intelligent vehicle entertainment screen and the like, in some embodiments.

[0059] The display can be an LED display, a liquid crystal display, a touch liquid crystal display, an OLED (Organic Light-Emitting Diode) touch and the like, in some embodiments. The display is used to display information of the terminal and to display visualized user interfaces. The processor, the memory and the display of the terminal communicate with each other through a system bus.

[0060] In an embodiment, the following steps are implemented when the processor executes the voice interaction program based on the intelligent vehicle entertainment screen in the memory:

[0061] Based on the intelligent agent technology, the trip service module, the cabin entertainment module and the vehicle control module are abstracted as interactive intelligent agent MCP protocol modules;

[0062] The voice instruction of the user is collected based on voice recognition, and the voice instruction is analyzed for intent by using a large model to identify the deep intent of the user;

[0063] input the deep intention into the agent MCP protocol module, the agent MCP protocol module task decomposition is obtained one or more sub-tasks;

[0064] The agent MCP protocol module performs task execution according to one or more sub-tasks to obtain a task execution result.

[0065] Among them, the agent MCP protocol module performs task execution according to one or more sub-tasks to obtain a task execution result, and then includes:

[0066] The task execution result is fed back to the user through natural language generation technology, and the user is supported to correct the instruction through voice, touch or interface.

[0067] Among them, the sub-tasks include travel service tasks, cabin entertainment tasks and vehicle control tasks.

[0068] Among them, the task execution result includes travel service task execution result, cabin entertainment task execution result and vehicle control task execution result.

[0069] Among them, the agent MCP protocol module performs task execution according to one or more sub-tasks to obtain a task execution result, specifically including:

[0070] When the sub-task is a travel service task, the travel service module in the agent MCP protocol module calls the navigation function and obtains the route information to obtain the travel service task execution result;

[0071] When the sub-task is a cabin entertainment task, the cabin entertainment module in the agent MCP protocol module recommends music or video according to user emotion or scene to obtain the cabin entertainment task execution result;

[0072] When the sub-task is a vehicle control task, the vehicle control module in the agent MCP protocol module adjusts the air conditioner temperature, seat heating or window state to obtain the vehicle control task execution result.

[0073] Among them, the voice interaction method based on intelligent vehicle entertainment screen further includes:

[0074] If the user's deep intention cannot be determined, based on user historical behavior data, real-time scene information and context information, a large model is used to infer to generate a task planning strategy.

[0075] Among them, the voice interaction method based on intelligent vehicle entertainment screen further includes:

[0076] When the sub-task execution fails, the agent MCP protocol module prompts the user to re-input the instruction or provides alternative options.

[0077] The application further provides a computer readable storage medium, wherein the computer readable storage medium stores a voice interaction program based on an intelligent vehicle entertainment screen, and the voice interaction program based on the intelligent vehicle entertainment screen, when executed by a processor, implements the steps of the voice interaction method based on the intelligent vehicle entertainment screen.

[0078] To sum up, the application provides a voice interaction method, system, terminal and computer readable storage medium based on an intelligent vehicle entertainment screen, the method comprising: based on an agent technology, abstracting a travel service module, a cabin entertainment module and a vehicle control module into an interactive agent MCP protocol module; based on voice recognition, collecting a voice instruction of a user, using a large model to analyze the voice instruction, and identifying a deep intention of the user; inputting the deep intention into the agent MCP protocol module, the agent MCP protocol module performing task decomposition on the deep intention to obtain one or more subtasks; and the agent MCP protocol module performing a task according to the one or more subtasks to obtain a task execution result. The application uses voice interaction technology, combines a large model and an agent technology, gives the ability to understand the intention of the user, and has the ability of reasoning, planning and executing the task of the user, can identify the deep intention of the user, improve the voice interaction experience of the user, and reduce the operation complexity.

[0079] It should be noted that in this document, the terms "comprising", "containing", or any other variant thereof are intended to cover non-exclusive inclusion, so that processes, methods, articles or terminals including a series of elements not only include those elements, but also include other elements not explicitly listed, or include elements inherent to such processes, methods, articles or terminals. Without more limitations, the element defined by the statement "comprising a" does not exclude the presence of another identical element in the process, method, article or terminal including the element.

[0080] Of course, those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by a computer program instructing related hardware (such as a processor, a controller, etc.), and the program can be stored in a computer readable storage medium, and the program can include the processes of the above-mentioned method embodiments when executed. The computer readable storage medium can be a memory, a disk, an optical disk, etc.

[0081] It should be understood that the application is not limited to the above examples, and those skilled in the art can make improvements or changes according to the above description, and all these improvements and changes should be within the protection scope of the appended claims of the application.

Claims

1. A voice interaction method based on an intelligent vehicle entertainment screen, characterized in that: The voice interaction method based on the smart car entertainment screen includes: Based on agent technology, the travel service module, cockpit entertainment module and vehicle control module are abstracted into an interactive agent MCP protocol module; Based on speech recognition, the user's voice commands are collected, and the intention of the voice commands is analyzed using a large model to identify the user's deep intention; Inputting the deep intention into the agent MCP protocol module, and the agent MCP protocol module decomposing the deep intention into tasks to obtain one or more subtasks; The agent MCP protocol module performs a task according to one or more subtasks to obtain a task execution result.

2. The voice interaction method based on the intelligent vehicle entertainment screen according to claim 1 is characterized in that: The agent MCP protocol module performs a task according to one or more subtasks to obtain a task execution result, and then further includes: The task execution results are fed back to the user through natural language generation technology, and the user is supported to correct instructions through voice, touch or interface.

3. The voice interaction method based on the intelligent vehicle entertainment screen according to claim 1 is characterized in that: The subtasks include travel service tasks, cabin entertainment tasks and vehicle control tasks.

4. The voice interaction method based on the intelligent vehicle entertainment screen according to claim 3 is characterized in that: The task execution results include travel service task execution results, cabin entertainment task execution results and vehicle control task execution results.

5. The voice interaction method based on the intelligent vehicle entertainment screen according to claim 4 is characterized in that: The agent MCP protocol module performs a task according to one or more subtasks and obtains a task execution result, specifically including: When the subtask is a travel service task, the travel service module in the agent MCP protocol module calls the navigation function and obtains the route information to obtain the travel service task execution result; When the subtask is a cockpit entertainment task, the cockpit entertainment module in the agent MCP protocol module recommends music or videos based on the user's mood or scene, and obtains the cockpit entertainment task execution result; When the subtask is a vehicle control task, the vehicle control module in the agent MCP protocol module adjusts the air conditioning temperature, seat heating or window status to obtain the vehicle control task execution result.

6. The voice interaction method based on the intelligent vehicle entertainment screen according to claim 1 is characterized in that: The voice interaction method based on the smart car entertainment screen also includes: If the user's deep intention is unclear, a task planning strategy is generated using large model reasoning based on the user's historical behavior data, real-time scenario information, and contextual information.

7. The voice interaction method based on the intelligent vehicle entertainment screen according to claim 5 is characterized in that: The voice interaction method based on the smart car entertainment screen also includes: When the subtask fails to execute, the agent MCP protocol module prompts the user to re-enter the instruction or provide an alternative solution.

8. A voice interaction system based on an intelligent vehicle entertainment screen, characterized in that: The voice interaction system based on the smart car entertainment screen includes: Voice interaction module, used to collect user voice commands; A large model is used to analyze the intent of the voice command and identify the user's deep intention; The intelligent agent MCP protocol module includes a travel service module, a cockpit entertainment module and a vehicle control module, which is used to decompose the deep intention into tasks, obtain one or more subtasks, and execute the task according to one or more subtasks to obtain the task execution result.

9. A terminal, characterized in that: The terminal includes: a memory, a processor, and a voice interaction program based on the smart car entertainment screen stored in the memory and runnable on the processor. When the voice interaction program based on the smart car entertainment screen is executed by the processor, the steps of the voice interaction method based on the smart car entertainment screen as described in any one of claims 1-7 are implemented.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a voice interaction program based on the smart car entertainment screen. When the voice interaction program based on the smart car entertainment screen is executed by the processor, the steps of the voice interaction method based on the smart car entertainment screen as described in any one of claims 1-7 are implemented.

Images