Multifunctional vehicle intelligent voice system, control method, vehicle and storage medium

Abstract

The invention relates to the technical field of vehicles, in particular to a multifunctional vehicle intelligent voice system, a control method, a vehicle and a storage medium, and the system comprises a voice recognition module which is used for converting a voice instruction of a current task into a text instruction; the short-term memory module is used for storing execution data of the historical tasks, and the execution data comprises at least one function module and an execution sequence of the function modules; the central large model module is used for determining execution data for executing the current task according to the text instruction and the historical task execution data, and updating the execution data of the historical task according to the execution data of the current task; and the adapter module is used for controlling at least one functional module according to the execution data of the current task. Therefore, the problems that in the prior art, when a vehicle intelligent voice system faces a complex task, multi-step or concurrent instruction processing is difficult to complete, interfaces between modules are not unified, and expansibility is poor are solved.

Description

Technical Field

[0001] The present application relates to the field of vehicle technology, and in particular to a multifunctional vehicle intelligent voice system, a control method, a vehicle, and a storage medium. Background Art

[0002] Vehicle intelligent voice systems mainly rely on a single module or a relatively simple voice recognition and control system. Such systems usually include modules such as voice recognition, natural language understanding, and command execution. After voice recognition, commands are generated and executed by the vehicle's control system. However, the vehicle intelligent voice systems in related technologies have limited capabilities when facing complex tasks and have difficulty completing multi-step or concurrent command processing. For example, multiple commands issued by users (such as "navigate to a certain place, play music and turn on the air conditioner at the same time") can often only be executed in a single step, lacking task coordination capabilities, and the interfaces between different modules are not unified. Different tasks need to rely on their own protocols or parameter formats, which limits the system's scalability and module collaboration. Summary of the Invention

[0003] The present application provides a multifunctional vehicle intelligent voice system, control method, vehicle and storage medium to solve the problems in related technologies such as the difficulty of vehicle intelligent voice systems in completing multi-step or concurrent command processing when faced with complex tasks, the lack of unified interfaces between modules, and poor scalability.

[0004] The first embodiment of the present application provides a multifunctional vehicle intelligent voice system, including: a voice recognition module, used to convert the voice instructions of the current task into text instructions; a short-term memory module, used to store the execution data of historical tasks, wherein the execution data includes at least one functional module and the execution order of the functional modules; a central large model module, used to determine the execution data of the current task based on the text instructions and the historical task execution data, and update the execution data of the historical tasks based on the execution data of the current task; an adapter module, used to control at least one functional module according to the execution data of the current task.

[0005] Optionally, the functions performed by the central large model module include task allocation and process planning function, instruction allocation and coordination function, and result integration and response decision function, wherein the task allocation and process planning function uses language to receive and analyze the text instructions of the speech recognition module, and determines at least one functional module to execute the text instructions based on the text instructions and the functional definitions of each functional module; the instruction allocation and coordination function is used to control the control sequence and update transmission of execution data of at least one functional module; the result integration and response decision function is used to determine whether other functional modules need to be called based on the execution data of the current task, and feedback the final response to the user.

[0006] Optionally, the adapter module is further configured to convert the text instruction into a format adapted by at least one functional module, and transmit the format to the at least one functional module.

[0007] Optionally, the at least one functional module includes a vehicle control functional module, a navigation functional module, a multimedia functional module, and a chat functional module.

[0008] The second aspect of the present application provides a multifunctional vehicle intelligent voice control method, which is implemented based on the above-mentioned multifunctional vehicle intelligent voice system, including: obtaining the user's voice instructions for the current task; inputting the voice instructions of the current task into the voice recognition module, and the voice recognition module converts the voice instructions of the current task into text instructions; inputting the text instructions into the central large model module, wherein the central large model module determines the execution data of the current task based on the text instructions and the historical task execution data stored in the short-term memory module, and updates the execution data of the historical tasks based on the execution data of the current task; inputting the execution data of the current task into the adapter module, wherein the adapter module controls at least one functional module based on the execution data of the current task.

[0009] Optionally, before inputting text instructions into the central large model module, it also includes: obtaining the user's voice instruction data set; marking the context information of multiple instructions in the voice instruction data set, and training the central large model module based on the context information; and using the trained central large model module to respond to the user's voice instructions.

[0010] Optionally, the context information of multiple instructions in the voice instruction dataset is labeled, including: labeling the intention of each instruction in the voice instruction dataset, and performing task decomposition on each instruction based on the intention; and correlating the decomposed tasks to obtain the context information of each instruction.

[0011] The third embodiment of the present application provides a vehicle, including: the multifunctional vehicle intelligent voice system of the above embodiment.

[0012] The fourth aspect of the present application provides a computer-readable storage medium on which a computer program or instruction is stored. When the computer program or instruction is executed, it is used to implement the multi-functional vehicle intelligent voice control method as described in the above embodiment.

[0013] The fifth embodiment of the present application provides a computer program product, including: a computer program or instructions, which, when executed, implements the multi-functional vehicle intelligent voice control method as described in the above embodiment.

[0014] Therefore, this application has at least the following beneficial effects:

[0015] In the embodiment of the present application, the user's voice command is converted into a text command through the voice recognition module. The central large model module receives the text command output of the voice recognition module, combines the execution data of the historical tasks stored in the short-term memory module and the functional description of each functional module to analyze the task requirements, decides which functional module to call, and sends the task command to the adapter module, which converts it into a format that meets the interface requirements of each functional module and transmits it to the corresponding functional module. Thus, through the memory function of the short-term memory module, the most recent user command history and task status are retained in the command processing, providing a guarantee for the continuous understanding of the task, realizing the efficient processing of the vehicle voice system in complex task scenarios, and solving the problems in the related art that the vehicle intelligent voice system is difficult to complete multi-step or concurrent command processing when facing complex tasks, and the interfaces between modules are not unified, and there is poor scalability.

[0016] Additional aspects and advantages of the present application will be given in part in the description below, and in part will become apparent from the description below, or will be learned through practice of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The above and / or additional aspects and advantages of the present application will become apparent and easily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which:

[0018] Figure 1 Schematic diagram of a multifunctional vehicle intelligent voice system according to an embodiment of the present application;

[0019] Figure 2 is a flow chart of a multi-functional vehicle intelligent voice control method according to an embodiment of the present application;

[0020] Figure 3 This is a workflow diagram of a multifunctional vehicle intelligent voice control system according to an embodiment of the present application. DETAILED DESCRIPTION

[0021] The following describes in detail embodiments of the present application. Examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present application, and should not be construed as limiting the present application.

[0022] The following describes the multifunctional vehicle intelligent voice system, method, vehicle and storage medium of the embodiment of the present application with reference to the accompanying drawings. In response to the problems mentioned in the above background technology, the present application provides a multifunctional vehicle intelligent voice system, in which the user issues instructions through voice, and the voice signal is converted into text instructions through the voice recognition module. The central large model module receives the text instruction output of the voice recognition module, combines the execution data of the historical tasks stored in the short-term memory module and the functional description of each functional module to analyze the task requirements, decide which functional module to call, and send the task instruction to the adapter module, which converts it into a format that meets the interface requirements of each functional module and transmits it to the corresponding functional module. Thus, through the memory function of the short-term memory module, the most recent user instruction history and task status are retained in the instruction processing, providing a guarantee for the continuous understanding of the task, realizing the efficient processing of the vehicle voice system in complex task scenarios, and solving the problems in the related art that the vehicle intelligent voice system is difficult to complete multi-step or concurrent instruction processing when facing complex tasks, and the interfaces between modules are not unified, and there is poor scalability.

[0023] Specifically, Figure 1 It is a block diagram of a multifunctional vehicle intelligent voice system according to an embodiment of the present application.

[0024] like Figure 1 As shown, the multifunctional vehicle intelligent voice system 10 includes: a voice recognition module 101, a short-term memory module 102, a central large model module 103 and an adapter module 104.

[0025] The speech recognition module 101 is used to convert the voice instructions of the current task into text instructions; the short-term memory module 102 is used to store the execution data of historical tasks, where the execution data includes at least one functional module and the execution order of the functional modules; the central large model module 103 is used to determine the execution data for the current task based on the text instructions and historical task execution data, and update the execution data of historical tasks based on the execution data of the current task; the adapter module 104 is used to control at least one functional module based on the execution data of the current task. The functional modules may include vehicle control function modules, navigation function modules, multimedia function modules, and chat function modules.

[0026] It is understandable that if Figure 1As shown, the multifunctional vehicle intelligent voice system in the embodiment of the present application combines the decision-making ability of the large model and the collaborative mode of the multifunctional modules, which can realize the decomposition of complex tasks, the processing of contextual continuous instructions, and the efficient intercommunication of functional modules. Specifically, the speech recognition module 101 receives the user's voice instructions and converts them into text format for subsequent processing. The short-term memory module 102 is used to store the history of the last 10 user instructions and the relevant task status. The short-term memory module 102 provides contextual information support for the central large model in the processing of complex instructions or continuous instructions. Whenever the user issues a new instruction, the short-term memory module 102 can help the subsequent central large model module 103 to refer to the previous instructions and execution status to ensure the information consistency during the instruction processing. The central large model module 103 is the core decision-making unit of the system. The central large model module 103 decides which one or several functional modules to use to complete the task based on the user instruction text transmitted by the speech recognition module 101 and the context information in the short-term memory module 102. For complex instructions, the central large model module 103 will decompose the task into multiple steps and coordinate multiple functional modules to work together. The central large model module 103 is responsible for coordinating the execution process of each multifunctional module to ensure the accuracy and consistency of the execution results. The adapter module 104 is responsible for adapting the data between the central large model module 103 and each functional module. Since the parameter formats and interface protocols of different functional modules may differ, the adapter module 104 is used to smooth out these differences and ensure that data can be seamlessly transmitted between the central large model and each functional module, thereby achieving seamless intercommunication between the functional modules, solving the problem of multi-module interface and parameter differences, and improving the scalability and compatibility of the system.

[0027] In one embodiment of the present application, the functions performed by the central large model module 103 include task allocation and process planning function, instruction allocation and collaboration function, and result integration and response decision function, wherein the task allocation and process planning function uses language to receive and analyze the text instructions of the speech recognition module 101, and determines at least one functional module to execute the text instructions based on the text instructions and the functional definitions of each functional module; the instruction allocation and collaboration function is used to control the control sequence and update transmission of execution data of at least one functional module; the result integration and response decision function is used to determine whether other functional modules need to be called based on the execution data of the current task, and feedback the final response to the user.

[0028] Specifically, the task allocation and process planning functions: receive and analyze the text instructions output by the voice recognition module 101, combine the functional definitions of each functional module, and select the most suitable functional module to process the task. If the instruction involves multiple steps or multiple functions (such as the linkage of navigation and multimedia), the central model will disassemble the task and generate a task process.

[0029] Instruction Distribution and Collaboration: When a task requires collaboration among multiple functional modules, the central large model module 103 coordinates the operation sequence and parameter transfer of each functional module. For information that requires further interaction, the model can continuously interact with the user based on contextual understanding and semantic analysis.

[0030] Result integration and response decision-making function: For the results returned by the multi-functional module, the central large model module 103 determines whether it is necessary to call other functional modules based on the preset process or real-time analysis, and finally decides whether to end the process and feeds back the final response to the user.

[0031] It can be seen that the central large model module 103 in the embodiment of the present application makes decisions and assigns tasks to each functional module based on user instructions and vehicle multi-functional requirements, ensuring that complex tasks can be executed step by step and responded in linkage, and can also realize synchronous or concurrent task processing. Compared with the existing single-module system, it has stronger complex task response capabilities.

[0032] In one embodiment of the present application, the adapter module 104 is further configured to convert the text instruction into a format adapted by at least one functional module, and transmit the format to the at least one functional module.

[0033] It is understood that the adapter module 104 converts the instructions of the central model into the specific format required by each functional module and transmits them to the target functional module. It is also responsible for calibrating and adjusting the instruction parameters to ensure that each functional module can receive and correctly execute the task instructions. Therefore, the adapter module achieves compatibility of protocols and data formats, resolves parameter field and protocol differences, and realizes smooth data exchange.

[0034] It should be noted that the embodiment of the present application summarizes that each functional module is independent and has clear functions, which is convenient for expanding new functional modules and dynamically combining tasks according to user needs. Each functional module provides the implementation of the corresponding function for the central large model, completes specific tasks after receiving parameters, and feeds back the execution results. For example: the vehicle control function module realizes the in-car control functions such as windows, air conditioning, and seats; the multimedia function module realizes multimedia-related functions such as music playback and video control; the navigation function module handles navigation route planning and real-time updates, etc.; the chat provides social interaction functions such as chatting, question and answer when there are no specific functional requirements.

[0035] According to the multifunctional vehicle intelligent voice system proposed in the embodiment of the present application, the user issues a command through voice, and the voice signal is converted into a text command through the voice recognition module. The central large model module receives the text command output of the voice recognition module, combines the execution data of the historical tasks stored in the short-term memory module and the functional description of each functional module to analyze the task requirements, decides which functional module to call, and sends the task command to the adapter module, which converts it into a format that meets the interface requirements of each functional module and transmits it to the corresponding functional module. Thus, through the memory function of the short-term memory module, the most recent user command history and task status are retained in the command processing, providing a guarantee for the continuous understanding of the task, realizing the efficient processing of the vehicle voice system in complex task scenarios, and solving the problems in the related art that the vehicle intelligent voice system is difficult to complete multi-step or concurrent command processing when facing complex tasks, and the interfaces between modules are not unified, and there is poor scalability.

[0036] Next, the multifunctional vehicle intelligent voice control method proposed in accordance with an embodiment of the present application will be described with reference to the accompanying drawings.

[0037] Figure 2 A flowchart of a multi-functional vehicle intelligent voice control method provided in an embodiment of the present application.

[0038] like Figure 2 As shown, the multifunctional vehicle intelligent voice control method includes the following steps:

[0039] In step S101 , the user's voice instruction for the current task is obtained.

[0040] It is understood that the embodiments of the present application can capture the voice commands issued by the user through the vehicle microphone or other voice collection device. The voice commands are usually in the form of natural language and include operation requests for vehicle functions (such as "turn on the air conditioner and navigate to the nearest gas station").

[0041] In step S102 , the voice instruction of the current task is input into the voice recognition module, and the voice recognition module converts the voice instruction of the current task into a text instruction.

[0042] In step S103, the text instruction is input into the central large model module, wherein the central large model module determines the execution data of the current task based on the text instruction and the historical task execution data stored in the short-term memory module, and updates the execution data of the historical task based on the execution data of the current task.

[0043] In one embodiment of the present application, before inputting text instructions into the central large model module, it also includes: obtaining the user's voice instruction data set; marking the context information of multiple instructions in the voice instruction data set, and training the central large model module based on the context information; and using the trained central large model module to respond to the user's voice instructions.

[0044] It is understandable that the central large model module in the embodiment of the present application can select a large language model (such as qwen, chatglm, etc.) that has been pre-trained with large-scale data as a base model for fine-tuning. In order to make the central large model module adapt to the complex scenarios in the vehicle intelligent voice system, the embodiment of the present application performs instruction fine-tuning training on the central large model module. The specific training process includes: obtaining the user's voice command data set, wherein the voice command data set is constructed to cover various driving scenarios, in-car control, navigation guidance, media management and other related voice command data sets. The data should include single-step instructions (such as "open the window"), multi-step complex instructions (such as "navigate to a place and play music"), and continuous dialogue scenarios (such as "turn off the air conditioner" and "open the window" continuous operations).

[0045] Furthermore, the context information of multiple instructions in the voice instruction dataset is annotated, including: annotating the intention of each instruction in the voice instruction dataset, and decomposing each instruction into tasks based on the intention; and correlating the decomposed tasks to obtain the context information of each instruction.

[0046] Specifically, the embodiment of the present application labels the intent of each instruction in the data set and designs multi-task decomposition examples so that the large model can learn how to decompose tasks into individual steps and assign them to appropriate functional modules when receiving complex instructions. For example, the user's "navigate to XXX and play music" instruction is decomposed into two tasks, navigation and multimedia, and marked with the appropriate execution order. The embodiment of the present application incorporates continuous instructions and multi-round dialogue data into training, so that the central large model module can understand the user's changing intentions in multiple rounds of instructions based on the contextual information provided by the short-term memory module. By introducing context variables, the context relevance and memory ability of the central large model module are enhanced.

[0047] In step S104 , the execution data of the current task is input to the adapter module, wherein the adapter module controls at least one functional module according to the execution data of the current task.

[0048] It should be noted that the aforementioned explanation of the embodiment of the multifunctional vehicle intelligent voice system is also applicable to the multifunctional vehicle intelligent voice control method of this embodiment, and they can be referenced to each other and will not be repeated here.

[0049] The following combination Figure 3The workflow of the multifunctional vehicle intelligent voice control method according to the embodiment of the present application is described in detail as follows:

[0050] Voice input and recognition: Users issue commands through voice, and the voice signals are converted into text through the voice recognition module.

[0051] Instruction parsing and task allocation: The central large module receives the text output of the speech recognition module, combines the context information stored in the short-term memory module and the functional description of the functional module to analyze the task requirements, and decides which functional module or modules to call.

[0052] Adapter conversion and instruction transmission: The central large model module sends the task instructions to the adapter module, which converts them into a format that meets the interface requirements of the functional module and transmits them to the corresponding functional module.

[0053] Task execution and result feedback: Each functional module executes the task according to the received instructions and feeds back the results to the central model.

[0054] Results Integration and Task Termination: The central model determines whether further calls to other functional modules are needed based on the execution feedback. If not, the instruction flow is terminated and the instruction and execution results are updated to the short-term memory module for reference by subsequent instructions.

[0055] According to the multifunctional vehicle intelligent voice control method proposed in the embodiment of the present application, the user issues a command through voice, and the voice signal is converted into a text command through the voice recognition module. The central large model module receives the text command output of the voice recognition module, combines the execution data of the historical tasks stored in the short-term memory module and the functional description of each functional module to analyze the task requirements, decides which functional module to call, and sends the task command to the adapter module, which converts it into a format that meets the interface requirements of each functional module and transmits it to the corresponding functional module. Thus, through the memory function of the short-term memory module, the most recent user command history and task status are retained in the command processing, providing a guarantee for the continuous understanding of the task, realizing the efficient processing of the vehicle voice system in complex task scenarios, and solving the problems in the related art that the vehicle intelligent voice system is difficult to complete multi-step or concurrent command processing when facing complex tasks, and the interfaces between modules are not unified, and there is poor scalability.

[0056] In addition, an embodiment of the present application also provides a vehicle, including: the multifunctional vehicle intelligent voice system 10 of the above embodiment.

[0057] An embodiment of the present application also provides a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, the multi-functional vehicle intelligent voice control method as described above is implemented.

[0058] An embodiment of the present application provides a computer program product, including: a computer program or instructions, which, when executed, implements the multi-functional vehicle intelligent voice control method as described in the above embodiment.

[0059] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or N embodiments or examples in a suitable manner. In addition, those skilled in the art can combine and combine different embodiments or examples described in this specification and features of different embodiments or examples without contradiction.

[0060] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be understood to indicate or imply relative importance or implicitly specify the number of technical features indicated. Thus, a feature specified as "first" or "second" may explicitly or implicitly include at least one such feature. In the description of this application, "N" means at least two, for example, two, three, etc., unless otherwise specifically defined.

[0061] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, fragment or portion of code comprising one or more executable instructions for implementing the steps of a custom logical function or process, and the scope of the preferred embodiments of the present application includes alternative implementations in which functions may be performed out of the order shown or discussed, including performing functions in a substantially simultaneous manner or in reverse order depending on the functions involved, which should be understood by those skilled in the art to which the embodiments of the present application belong.

[0062] It should be understood that various parts of the present application can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiment, the N steps or methods can be implemented using software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented using hardware, as in another embodiment, any one of the following technologies known in the art or a combination thereof can be used: a discrete logic circuit having a logic gate circuit for implementing a logic function on a data signal, an application-specific integrated circuit having a suitable combination of logic gate circuits, a programmable gate array, a field programmable gate array, etc.

[0063] Those skilled in the art will understand that all or part of the steps in the method of the above embodiment can be completed by instructing related hardware through a program, and the program can be stored in a computer-readable storage medium. When the program is executed, it includes one or a combination of the steps of the method embodiment.

[0064] Although the embodiments of the present application have been shown and described above, it can be understood that the above embodiments are exemplary and cannot be understood as limitations on the present application. Ordinary technicians in this field can change, modify, replace and modify the above embodiments within the scope of the present application.

Claims

1. A multifunctional vehicle intelligent voice system, characterized in that: include: Speech recognition module, used to convert the voice instructions of the current task into text instructions; A short-term memory module, configured to store execution data of historical tasks, wherein the execution data includes at least one functional module and the execution order of the functional modules; A central large model module, configured to determine the execution data for executing the current task based on the text instruction and the historical task execution data, and update the execution data of the historical task based on the execution data of the current task; The adapter module is used to control at least one functional module according to the execution data of the current task.

2. The multifunctional vehicle intelligent voice system according to claim 1, characterized in that: The functions performed by the central large model module include task allocation and process planning, instruction allocation and coordination, result integration and response decision-making. wherein the task allocation and process planning function receives and analyzes the text instructions of the speech recognition module in language, and determines at least one functional module to execute the text instructions based on the text instructions and the functional definitions of the various functional modules; The instruction allocation and coordination function is used to control the control sequence of the at least one functional module and the update and transmission of execution data; The result integration and response decision function is used to determine whether other functional modules need to be called based on the execution data of the current task, and to feed back the final response to the user.

3. The multifunctional vehicle intelligent voice system according to claim 1, characterized in that: The adapter module is further configured to: The text instruction is converted into a format adapted by the at least one functional module and transmitted to the at least one functional module.

4. The multifunctional vehicle intelligent voice system according to any one of claims 1 to 3, characterized in that: The at least one functional module includes a vehicle control functional module, a navigation functional module, a multimedia functional module, and a chat functional module.

5. A multifunctional vehicle intelligent voice control method, characterized in that: The method is implemented based on the multifunctional vehicle intelligent voice system, wherein the method includes the following steps: Get the user's voice command for the current task; Inputting the voice instruction of the current task into a voice recognition module, and the voice recognition module converting the voice instruction of the current task into a text instruction; Inputting the text instruction into the central large model module, wherein the central large model module determines the execution data of the current task based on the text instruction and the historical task execution data stored in the short-term memory module, and updates the execution data of the historical task based on the execution data of the current task; The execution data of the current task is input into an adapter module, wherein the adapter module controls at least one functional module according to the execution data of the current task.

6. The multifunctional vehicle intelligent voice control method according to claim 5, characterized in that: Before inputting the text instruction into the central large model module, the method further includes: Obtain the user's voice command dataset; Labeling context information of multiple instructions in the voice instruction dataset, and training the central large model module based on the context information; The trained central model module is used to respond to the user's voice commands.

7. The multifunctional vehicle intelligent voice control method according to claim 6, characterized in that: The context information of multiple commands in the voice command dataset is annotated, including: Marking the intent of each instruction in the voice instruction dataset, and performing task decomposition on each instruction based on the intent; The decomposed tasks are correlated with each other to obtain context information of each instruction.

8. A vehicle, characterized in that: include: The multifunctional vehicle intelligent voice system according to any one of claims 1 to 4.

9. A computer-readable storage medium having a computer program or instruction stored thereon, characterized in that: When the computer program or instruction is executed, the multifunctional vehicle intelligent voice control method according to any one of claims 5 to 7 is implemented.

10. A computer program product comprising: A computer program or instruction, characterized in that when the computer program or instruction is executed, it implements the multi-functional vehicle intelligent voice control method described in any one of claims 5-7.

Images