Task processing method, device, electronic equipment, medium and product for vehicle-mounted system

Abstract

The present application provides a task processing method, device, electronic device, medium and product for an in-vehicle system, which belongs to the field of vehicle technology. The method obtains the performance data of the in-vehicle system in executing a first user demand task and the user's execution status feedback information on the system executing the first user demand task, and then optimizes the intent recognition model used by the in-vehicle system to recognize the intent of vehicle environmental perception information and the function call model used to call the system service when the in-vehicle system executes the user demand task based on the performance data execution status feedback information, and then uses the optimized intent recognition model to recognize the intent of the vehicle environmental perception information to obtain a new second user intent, and calls the system service through the optimized function call model to execute the second user demand task corresponding to the second user intent. The present application can improve the efficiency of the in-vehicle system in responding to user needs in complex and changing scenarios.

Description

Technical Field

[0001] The present application relates to the field of vehicle technology, and in particular to a task processing method, device, electronic equipment, medium and product for an on-vehicle system. Background Art

[0002] With the continuous development and progress of artificial intelligence technology, the role of in-vehicle artificial intelligence systems (referred to as "in-vehicle systems") in intelligent driving and human-computer interaction of vehicles is becoming increasingly important. In related technologies, in-vehicle systems usually rely on predefined rules and limited dialogue models to provide intelligent services to users. However, with the complexity and changeability of vehicle application scenarios, it is difficult for in-vehicle systems to accurately identify user intentions and then perform related operations. As a result, the efficiency of in-vehicle systems in responding to users' intelligent service needs for using vehicles in complex and changeable vehicle application scenarios is relatively low. Summary of the invention

[0003] The main purpose of the embodiments of the present application is to propose a task processing method, device, electronic device, medium and product for an in-vehicle system, aiming to improve the efficiency of the in-vehicle system in responding to the user's intelligent service needs for using the vehicle in complex and changeable vehicle application scenarios.

[0004] To achieve the above-mentioned purpose, a first aspect of an embodiment of the present application provides a task processing method of an in-vehicle system, the method comprising:

[0005] Acquire performance data of the vehicle-mounted system executing a first user requirement task, and acquire execution status feedback information of the first user requirement task; wherein the first user requirement task is a task corresponding to a first user intention obtained by performing intention recognition on vehicle environment perception information through an intention recognition model;

[0006] Based on the performance data and the execution status feedback information, the intention recognition model and the function call model are optimized to obtain an optimized intention recognition model and an optimized function call model; wherein the function call model is used to call the system service when the vehicle-mounted system executes the first user demand task;

[0007] Performing intent recognition on the vehicle environment perception information through the optimized intent recognition model to obtain a second user intent;

[0008] The optimized function call model is used to call the system service to execute the second user demand task corresponding to the second user intention.

[0009] In some embodiments, the optimizing the intent recognition model and the function call model based on the performance data and the execution status feedback information to obtain an optimized intent recognition model and an optimized function call model includes:

[0010] Performing reward calculation based on the performance data and the execution status feedback information to obtain a reward function;

[0011] The intention recognition model and the function call model are optimized based on the reward function to obtain an optimized intention recognition model and an optimized function call model.

[0012] In some embodiments, the optimizing the intent recognition model and the function call model based on the reward function includes:

[0013] Based on the reward function, the intention recognition strategy and the function call strategy of the vehicle system are updated to obtain an updated intention recognition strategy and an updated function call strategy; wherein the intention recognition strategy is an application strategy for performing intention recognition on vehicle environment perception information through an intention recognition model, and the function call strategy is an application strategy for calling a system service to perform a task through a function call model;

[0014] Optimizing the model parameters of the intent recognition model based on the updated intent recognition strategy to obtain an optimized intent recognition model;

[0015] The model parameters of the function call model are optimized based on the updated function call strategy to obtain an optimized function call model.

[0016] In some embodiments, the updating of the intention recognition strategy and the function call strategy of the vehicle-mounted system based on the reward function includes:

[0017] Performing reinforcement learning processing on a preset interaction data set based on the reward function to update an intention recognition strategy and a function call strategy for the vehicle-mounted system;

[0018] Among them, the interaction data set includes the first state data of the vehicle-mounted system, the action data of the vehicle-mounted system performing intent recognition and function call execution corresponding to the first state data, the second state data of the vehicle-mounted system, and the action data of the vehicle-mounted system performing intent recognition and function call execution corresponding to the second state data, and the first state data and the second state data are the state data of the vehicle-mounted system at different times.

[0019] In some embodiments, the intention recognition model includes a multimodal intention recognition large model, and the step of performing intention recognition on the vehicle environment perception information through the optimized intention recognition model to obtain the second user intention includes:

[0020] Acquiring vehicle environment perception information; wherein the vehicle environment perception information at least includes vehicle environment sound information and vehicle environment image information;

[0021] The vehicle environment sound information and the vehicle environment image information are subjected to multimodal intention recognition by using the optimized multimodal intention recognition large model to obtain the second user intention.

[0022] In some embodiments, the method further comprises:

[0023] Obtaining the confidence level of the user intention recognition result corresponding to the vehicle environment sound information;

[0024] The multimodal intention recognition is performed on the vehicle environment sound information and the vehicle environment image information by using the optimized multimodal intention recognition large model, including:

[0025] When the confidence level is less than or equal to a preset confidence level threshold, multimodal intention recognition is performed on the vehicle environment sound information and the vehicle environment image information through the multimodal intention recognition large model.

[0026] In some embodiments, the vehicle environment sound information includes voice information of a vehicle user, and the vehicle environment image information includes character image information of the vehicle user;

[0027] The method further comprises:

[0028] Acquire a first acquisition time of the voice information, and acquire a second acquisition time of the character image information;

[0029] The multimodal intention recognition is performed on the vehicle environment sound information and the vehicle environment image information by using the optimized multimodal intention recognition large model, including:

[0030] When the first acquisition time and the second acquisition time are the same, multimodal intent recognition is performed on the voice information and the character image information through the optimized multimodal intent recognition large model.

[0031] To achieve the above-mentioned purpose, a second aspect of an embodiment of the present application provides a task processing device of an in-vehicle system, the device comprising:

[0032] an acquisition module, used to acquire performance data of the vehicle-mounted system in executing a first user requirement task, and to acquire execution status feedback information of the first user requirement task; wherein the first user requirement task is a task corresponding to a first user intention obtained by performing intention recognition on vehicle environment perception information through an intention recognition model;

[0033] A model optimization module, configured to optimize the intention recognition model and the function call model based on the performance data and the execution status feedback information to obtain an optimized intention recognition model and an optimized function call model; wherein the function call model is used to call a system service when the vehicle-mounted system executes the first user demand task;

[0034] The task processing module is used to perform intent recognition on the vehicle environment perception information through the optimized intention recognition model to obtain the second user intention, and to call the system service through the optimized function call model to execute the second user demand task corresponding to the second user intention.

[0035] To achieve the above objectives, a third aspect of an embodiment of the present application proposes an electronic device, which includes a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the method described in the first aspect is implemented.

[0036] To achieve the above-mentioned purpose, the fourth aspect of an embodiment of the present application proposes a vehicle, which is equipped with an electronic device, the electronic device includes a memory and a processor, the memory stores a computer program, and the processor implements the method described in the first aspect when executing the computer program.

[0037] To achieve the above objectives, the fifth aspect of an embodiment of the present application proposes a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method described in the first aspect is implemented.

[0038] To achieve the above-mentioned purpose, the sixth aspect of the embodiments of the present application proposes a computer program product, wherein the computer program product comprises a computer program, and when the computer program is executed by a processor, the method provided in the first aspect above is implemented.

[0039] The task processing method, device, electronic device, vehicle, computer-readable storage medium and computer program product of the vehicle-mounted system proposed in the embodiments of the present application are achieved by obtaining performance data of the vehicle-mounted system executing a first user requirement task, and obtaining execution status feedback information of the first user requirement task; wherein the first user requirement task is a task corresponding to a first user intention obtained by performing intent recognition on vehicle environmental perception information through an intention recognition model; based on the performance data and the execution status feedback information, the intention recognition model and the function call model are optimized to obtain an optimized intention recognition model and an optimized function call model; wherein the function call model is used to call a system service when the vehicle-mounted system executes the first user requirement task; the second user intention is obtained by performing intent recognition on vehicle environmental perception information through the optimized intention recognition model; and the system service is called through the optimized function call model to execute a second user requirement task corresponding to the second user intention.

[0040] Compared with the way in which traditional in-vehicle systems rely on predefined rules and limited dialogue models to provide services to users, the embodiments of the present application obtain the performance data of the in-vehicle system in executing the first user requirement task and the user's execution status feedback information on the system executing the first user requirement task, and then optimize the intent recognition model used by the in-vehicle system to identify the intent of vehicle environment perception information and the function call model used to call system services when the in-vehicle system executes the user requirement task based on the performance data execution status feedback information, and then use the optimized intent recognition model to identify the intent of the vehicle environment perception information to obtain a new second user intention, and call the system service through the optimized function call model to execute the second user requirement task corresponding to the second user intention.

[0041] In this way, the embodiments of the present application monitor user feedback and system performance data, thereby dynamically optimizing the intent recognition model and function call model used by the vehicle-mounted system to provide intelligent services to users, thereby enabling the vehicle-mounted system to automatically adjust and optimize in different scenario tasks, ensuring the efficient operation of the vehicle-mounted system in different environments and tasks, thereby improving the efficiency of the vehicle-mounted system in responding to users' needs for intelligent services in complex and changeable vehicle application scenarios.

[0042] In addition, the embodiments of the present application can also improve the robustness and accuracy of the in-vehicle system in responding to user needs by dynamically adjusting the intent recognition model and function call model through real-time monitoring of user feedback and system performance data, thereby significantly improving the accuracy of intent recognition, the linkage efficiency of function calls, and the accuracy of skill completion of the in-vehicle system. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1A flowchart of the steps of the task processing method of the vehicle-mounted system provided in some embodiments of the present application;

[0044] Figure 2 for Figure 1 Schematic diagram of detailed process flow of step S102;

[0045] Figure 3 for Figure 2 Schematic diagram of detailed process flow of step S202;

[0046] Figure 4 A schematic diagram of an application flow of adaptive adjustment of an in-vehicle system involved in some embodiments of the task processing method of the in-vehicle system provided in an embodiment of the present application;

[0047] Figure 5 for Figure 1 Schematic diagram of detailed process flow of step S103;

[0048] Figure 6 A schematic diagram of a complete application flow involved in the task processing method of the vehicle-mounted system provided in an embodiment of the present application;

[0049] Figure 7 A schematic diagram of the structure of a task processing device of an on-board system provided in an embodiment of the present application;

[0050] Figure 8 It is a schematic diagram of the hardware structure of the electronic device provided in the embodiment of the present application. DETAILED DESCRIPTION

[0051] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0052] It should be noted that, although the functional modules are divided in the device schematic diagram and the logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than the module division in the device or the order in the flowchart. The terms "first", "second", etc. in the specification, claims and the above drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.

[0053] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein are only for the purpose of describing the embodiments of this application and are not intended to limit this application.

[0054] First, the professional and technical terms involved in the embodiments of the present application are briefly explained.

[0055] In-vehicle artificial intelligence system.

[0056] In-vehicle artificial intelligence system, referred to as "in-vehicle system". In recent years, natural language processing technology based on large-scale pre-trained models has made significant progress, enabling artificial intelligence (AI) systems to better understand and respond to users' natural language input. This trend has promoted the development of in-vehicle AI agents from rule-driven to intelligent and multi-modal fusion. The comprehensive in-vehicle system based on in-vehicle AI agents and vehicle configuration software and hardware to respond to the intelligent service needs of vehicle users is also constantly developing and improving in response to market demand and technological development.

[0057] Secondly, the overall concept of the task processing method of the vehicle-mounted system provided in the embodiment of the present application is explained.

[0058] With the continuous development and progress of artificial intelligence technology, the role of in-vehicle systems in vehicle intelligent driving and human-computer interaction is becoming increasingly important. For example, the application of in-vehicle systems in vehicle navigation, voice assistants, in-vehicle entertainment and security monitoring is becoming more and more extensive and is attracting more and more attention from vehicle users.

[0059] In related technologies, in-vehicle systems usually rely on predefined rules and limited dialogue models to provide intelligent services to users. However, with the complexity and changeability of vehicle application scenarios, it has become difficult for in-vehicle systems to accurately identify user intentions and then perform related operations. As a result, the efficiency of in-vehicle systems in responding to users' intelligent service needs for using vehicles in complex and changeable vehicle application scenarios is relatively low. Therefore, how to enable in-vehicle systems to accurately identify user intentions in a complex and changeable in-vehicle environment and perform related operations through efficient function calls is a technical problem that needs to be solved urgently in the field of vehicle and artificial intelligence technology.

[0060] Based on this, the embodiments of the present application provide a task processing method, device, electronic device, vehicle, medium and product for an in-vehicle system, aiming to improve the efficiency of the in-vehicle system in responding to the user's intelligent service needs for using the vehicle in complex and changeable vehicle application scenarios.

[0061] The embodiment of the present application obtains performance data of the vehicle-mounted system executing a first user requirement task, and obtains execution status feedback information of the first user requirement task; wherein the first user requirement task is a task corresponding to the first user intention obtained by performing intent recognition on vehicle environment perception information through an intention recognition model; based on the performance data and the execution status feedback information, the intention recognition model and the function call model are optimized to obtain an optimized intention recognition model and an optimized function call model; wherein the function call model is used to call a system service when the vehicle-mounted system executes the first user requirement task; the second user intention is obtained by performing intent recognition on vehicle environment perception information through the optimized intention recognition model; and the system service is called through the optimized function call model to execute a second user requirement task corresponding to the second user intention.

[0062] Compared with the way in which traditional in-vehicle systems rely on predefined rules and limited dialogue models to provide services to users, the embodiments of the present application obtain the performance data of the in-vehicle system in executing the first user requirement task and the user's execution status feedback information on the system executing the first user requirement task, and then optimize the intent recognition model used by the in-vehicle system to identify the intent of vehicle environment perception information and the function call model used to call system services when the in-vehicle system executes the user requirement task based on the performance data execution status feedback information, and then use the optimized intent recognition model to identify the intent of the vehicle environment perception information to obtain a new second user intention, and call the system service through the optimized function call model to execute the second user requirement task corresponding to the second user intention.

[0063] In this way, the embodiments of the present application monitor user feedback and system performance data, thereby dynamically optimizing the intent recognition model and function call model used by the vehicle-mounted system to provide intelligent services to users, thereby enabling the vehicle-mounted system to automatically adjust and optimize in different scenario tasks, ensuring the efficient operation of the vehicle-mounted system in different environments and tasks, thereby improving the efficiency of the vehicle-mounted system in responding to users' needs for intelligent services in complex and changeable vehicle application scenarios.

[0064] In addition, the embodiments of the present application can also improve the robustness and accuracy of the in-vehicle system in responding to user needs by dynamically adjusting the intent recognition model and function call model through real-time monitoring of user feedback and system performance data, thereby significantly improving the accuracy of intent recognition, the linkage efficiency of function calls, and the accuracy of skill completion of the in-vehicle system.

[0065] Next, the task processing method, device, electronic device, medium and product of the vehicle-mounted system provided in the embodiment of the present application are specifically described through the following embodiments, and the task processing method of the vehicle-mounted system provided in the embodiment of the present application is first described in detail.

[0066] It should be noted that the task processing method of the vehicle-mounted system provided in the embodiment of the present application relates to the field of vehicle technology. The task processing method of the vehicle-mounted system provided in the embodiment of the present application can be applied to the terminal, can also be applied to the server side, and can also be software running in the terminal or the server side. In some embodiments, the terminal can be a terminal configured on the vehicle, or can be an electronic device such as a smart phone, tablet computer, laptop computer, desktop computer, etc. associated with the vehicle. The terminal and the vehicle are associated with each other, which means that the terminal can communicate and interact with the vehicle based on the network. The server side can be a backend server terminal device of the vehicle, which can be configured as an independent physical server, or a server cluster or distributed system composed of multiple physical servers, and can also be configured as a cloud server that provides cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (Content Delivery Network, CDN) and basic cloud computing services such as big data and artificial intelligence platforms. The software can be an application, a computer program, and a storage medium carrying the computer program, etc. to implement the task processing method of the vehicle-mounted system. It should be understood that based on different design requirements of actual applications, in different feasible embodiments, the terminal, server side, and software that apply the task processing method of the vehicle-mounted system provided in the embodiments of the present application may of course also be in other forms not listed here. The task processing method of the vehicle-mounted system provided in the embodiments of the present application does not specifically limit this.

[0067] In addition, the present application can also be used in many general or special computer system environments or configurations. For example: vehicle-mounted systems, personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronic devices, personal computers (PCs), minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, etc. The present application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present application can also be practiced in distributed computing environments, in which tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media including storage devices.

[0068] For ease of understanding and explanation, the following text takes the vehicle system applying the task processing method of the vehicle system provided by the embodiment of the present application as an example to explain in detail each specific embodiment of the present application. The implementation of the task processing method of the vehicle system provided by the embodiment of the present application in any of the above-mentioned forms of the subject application can refer to the process of the vehicle system applying the task processing method of the vehicle system described later.

[0069] It should be noted that in each specific implementation of the present application, when it comes to the need to perform relevant processing based on data related to user identity or characteristics such as user information, user behavior data, user historical data, and user location information, the user's permission or consent will be obtained first, and the collection, use, and processing of these data will comply with relevant laws, regulations, and standards. In addition, when the embodiment of the present application needs to obtain the user's sensitive personal information, the user's separate permission or consent will be obtained through a pop-up window or by jumping to a confirmation page. After clearly obtaining the user's separate permission or consent, the necessary user-related data for the normal operation of the embodiment of the present application will be obtained.

[0070] Please refer to Figure 1 , Figure 1 The following is a flowchart of the steps of the task processing method of the vehicle-mounted system provided in the embodiment of the present application in some embodiments. It should be understood that although Figure 1 The subsequent flowcharts of other steps show the execution order of some method steps, but based on the different design requirements of actual applications, the task processing method of the vehicle-mounted system provided in the embodiment of the present application can certainly adopt an execution order different from the method steps shown in the figure. That is, Figure 1 The order of the steps of the method shown does not constitute a limitation on the execution logic order of the task processing method of the vehicle-mounted system provided in the embodiment of the present application. Figure 1 Reasonable changes in the order of the steps of the method shown should be included in the protection scope of the task processing method of the vehicle-mounted system provided in the embodiment of the present application.

[0071] like Figure 1 As shown, in some embodiments, the embodied intelligent system applies the task processing method of the vehicle-mounted system provided in the embodiment of the present application, which may include steps S101 to S104.

[0072] Step S101: Obtain performance data of the vehicle-mounted system in executing a first user requirement task, and obtain execution status feedback information of the first user requirement task; wherein the first user requirement task is a task corresponding to the first user intention obtained by performing intent recognition on vehicle environment perception information through an intention recognition model.

[0073] When the in-vehicle system responds to the user's demand for intelligent services in the vehicle, it uses the intent recognition model (such as the large-scale pre-trained language model GPT-4L, llama3 model, and LaMA3 model, etc.) to identify the vehicle environment perception information to obtain the first user intention, and then calls the system service through the function call model (such as the Function Call model, etc.) to execute the first user demand task corresponding to the first intention. In this way, during the process of the in-vehicle system executing the first user demand task, it monitors its own performance data (response time and resource usage, etc.) of executing the first user demand task in real time, and monitors the vehicle user's execution status feedback information on its own execution of the first user demand task in real time. Therefore, when responding to user needs, the in-vehicle system can obtain the performance data of executing the first user demand task and the execution status feedback information of the first user demand task.

[0074] It should be noted that the vehicle environment perception information may include voice information output by the vehicle user to the vehicle system.

[0075] In some embodiments, the vehicle system can configure the voice processing module to perform voice analysis and recognition on the voice information of the vehicle user, thereby converting the voice information into text and extracting emotional features. In addition, the vehicle system can also configure the intention large model module to call the intention recognition model to perform text understanding and intention recognition on the text and emotional features output by the voice processing module, as well as the context information and historical conversation records associated with the voice information of the vehicle user, thereby obtaining the first user intention.

[0076] It should be noted that the system service can be a function or application programming interface (API) that needs to be called when the vehicle system performs a task.

[0077] In some embodiments, the vehicle system can use the function call model by configuring the function call module to call the corresponding function or API with the first user intent output by the intent model module.

[0078] In some embodiments, the vehicle-mounted system can monitor system performance in real time by configuring a monitoring module, so as to monitor and obtain its own performance data during the execution of the first user demand task. In addition, the vehicle-mounted system monitors system performance in real time through the monitoring module, and can also provide optimization suggestions for subsequent real-time adjustment and optimization of the intent recognition model and the function call model.

[0079] In some embodiments, the vehicle system may actively interact with the vehicle user during the execution of the first user demand task to obtain the vehicle user's feedback on the vehicle user's execution of the first user demand task by the vehicle system, thereby using the user feedback as the execution status feedback information of the first user demand task. For example, after executing the first user demand task, the vehicle system may actively ask the vehicle user for the evaluation of the intelligent service provided to the vehicle user by the current execution of the first user demand task, and then use the evaluation information (such as satisfaction, dissatisfaction, very good, not good or very bad) or adjustment suggestions (such as the volume of music playback in the corresponding area should be reduced when there are passengers sleeping) fed back by the vehicle user as the execution status feedback information of the first user demand task. For another example, when the first user demand task contains multiple subtasks, the vehicle system may actively ask the vehicle user for the evaluation information or adjustment suggestions for the subtask currently being executed after each subtask is executed, and use the received user feedback as the execution status feedback information of the first user demand task. For another example, the vehicle system may also use the evaluation information or adjustment suggestions for the first user demand task directly and actively issued by the vehicle user as the execution status feedback information of the first user demand task during the execution of the first user demand task.

[0080] Step S102: Based on the performance data and the execution status feedback information, the intention recognition model and the function call model are optimized to obtain an optimized intention recognition model and an optimized function call model; wherein the function call model is used to call system services when the vehicle system executes the first user requirement task.

[0081] After receiving the performance data and execution status feedback information, the vehicle system further optimizes the intention recognition model previously used to identify the vehicle environment perception information to obtain the first user's intention based on the performance data and the execution status feedback information, thereby obtaining an optimized intention recognition model, and optimizes the function call model used by the previous user to call the system service to perform the first user's required task, thereby obtaining an optimized intention recognition model.

[0082] In some embodiments, the vehicle-mounted system can dynamically adjust the intent recognition model and the function call model by configuring the optimization module. That is, the vehicle-mounted system uses the performance data and execution status feedback information monitored and acquired by the monitoring module in real time as the input of the optimization module, so that the optimization module dynamically adjusts the intent recognition and function call strategies according to the input data and outputs them, so that the vehicle-mounted system can dynamically optimize the intent recognition model and the function call model according to the adjustment strategy output by the optimization module.

[0083] Step S103: performing intent recognition on the vehicle environment perception information through the optimized intent recognition model to obtain a second user intent.

[0084] In some embodiments, when responding to a vehicle user's demand for new intelligent services of the vehicle, the in-vehicle system performs intent recognition on the vehicle environment perception information through an optimized intent recognition model to obtain a second user intention.

[0085] It should be noted that the vehicle environment perception information used by the vehicle-mounted system for intent recognition using the optimized intent recognition model may be new vehicle environment perception information that is different from the vehicle environment perception information used to recognize intent using the intent recognition model before optimization after dynamic changes. Of course, in some embodiments, the vehicle environment perception information used by the vehicle-mounted system for intent recognition using the optimized intent recognition model may also be the same as the vehicle environment perception information used to recognize intent previously.

[0086] Step S104: calling the system service through the optimized function call model to execute the second user demand task corresponding to the second user intention.

[0087] After the vehicle-mounted system performs intent recognition on the vehicle environment perception information to obtain the second user intention, the vehicle-mounted system calls the system service through an optimized function call model to execute the second user requirement task indicated by the vehicle user corresponding to the second user intention.

[0088] Afterwards, the vehicle system can continue to monitor the performance data of its own system and the execution status feedback information of the vehicle user for the second user demand task in real time during the execution of the second user demand task, and continue to optimize the intent recognition model and function call model based on the performance data and execution status feedback information, so as to further use the optimized intent recognition model and function call model to respond to the new intelligent service needs of the vehicle user. In this way, the vehicle system can optimize the intent recognition model and function call model through cyclic iteration in the process of responding to the intelligent service needs of the vehicle user, realize real-time adaptive dynamic adjustment and optimization, and thus intelligently respond to user needs to perform related task processing.

[0089] In the embodiment of the present application, during the process of executing the first user demand task through the vehicle-mounted system, the performance data (response time and resource usage, etc.) of executing the first user demand task is monitored in real time, and the execution status feedback information of the vehicle user for executing the first user demand task is monitored in real time. Afterwards, the vehicle-mounted system optimizes the intention recognition model previously used to identify the vehicle environment perception information to obtain the first user intention based on the performance data and the execution status feedback information, thereby obtaining an optimized intention recognition model, and optimizes the function call model for the previous user to call the system service to execute the first user demand task, thereby obtaining an optimized intention recognition model. Afterwards, when the vehicle-mounted system responds to the vehicle user's demand for a new intelligent service of the vehicle, the vehicle-mounted system uses the optimized intention recognition model to identify the vehicle environment perception information to obtain the second user intention, and for the second user demand task indicated by the vehicle user corresponding to the second user intention, the vehicle-mounted system uses the optimized function call model to call the system service to execute the second user demand task.

[0090] In this way, compared with the way traditional in-vehicle systems rely on predefined rules and limited dialogue models to provide services to users, the embodiments of the present application monitor user feedback and system performance data to dynamically optimize the intention recognition model and function call model used by the in-vehicle system to provide intelligent services to users, thereby enabling the in-vehicle system to automatically adjust and optimize in different scenario tasks, ensuring the efficient operation of the in-vehicle system in different environments and tasks, thereby improving the efficiency of the in-vehicle system in responding to users' intelligent service needs for using the vehicle in complex and changeable vehicle application scenarios.

[0091] In addition, the embodiments of the present application can also improve the robustness and accuracy of the in-vehicle system in responding to user needs by dynamically adjusting the intent recognition model and function call model through real-time monitoring of user feedback and system performance data, thereby significantly improving the accuracy of intent recognition, the linkage efficiency of function calls, and the accuracy of skill completion of the in-vehicle system.

[0092] In some embodiments, after each interaction with a vehicle user, the in-vehicle system can calculate a reward based on the execution status feedback information of a user requirement task (such as a first user requirement task or a second user requirement task) fed back by the user, and the performance data of the in-vehicle system in executing the user requirement task, and then use the calculated reward function to guide subsequent adjustments and optimizations of the intent recognition and function call models.

[0093] Please refer to Figure 2 , Figure 2 for Figure 1 Schematic diagram of the detailed step flow of step S102.

[0094] like Figure 2 As shown, in some embodiments, the above-mentioned step S102: based on the performance data and the execution status feedback information, optimizing the intent recognition model and the function call model to obtain an optimized intent recognition model and an optimized function call model may include steps S201 and S202 as shown below.

[0095] Step S201: Calculate a reward based on the performance data and the execution status feedback information to obtain a reward function.

[0096] After obtaining the performance data and the execution status feedback information, the vehicle system uses the reward calculation formula shown below to calculate the reward based on the performance data and the execution status feedback information to obtain the reward function R(t).

[0097] ;

[0098] It should be noted that in the above reward calculation formula, α, β, and γ are weight coefficients, each reflecting the importance of success rate, accuracy, and resource usage. Success Rate(t) represents the success rate at time t, and Accuracy(t) represents the accuracy at time t. Success Rate(t) and Accuracy(t) can be obtained based on the understanding and analysis of the execution status feedback information. Resource Usage(t) represents the resource usage at time t.

[0099] Step S202: Optimizing the intention recognition model and the function call model based on the reward function to obtain an optimized intention recognition model and an optimized function call model.

[0100] After calculating the reward function R(t), the vehicle system optimizes the intention recognition model and the function call model according to the guidance of the reward function R(t), thereby obtaining an optimized intention recognition model and an optimized function call model.

[0101] Please refer to Figure 3 , Figure 3 for Figure 2 Schematic diagram of the detailed step flow of step S202.

[0102] like Figure 3 As shown, in some embodiments, the above-mentioned step S202 may include steps S301 to S303 as shown below.

[0103] Step S301: Based on the reward function, the intention recognition strategy and function call strategy of the vehicle system are updated to obtain an updated intention recognition strategy and an updated function call strategy; wherein the intention recognition strategy is an application strategy for performing intent recognition on vehicle environment perception information through an intention recognition model, and the function call strategy is an application strategy for calling system services to perform tasks through a function call model.

[0104] In the process of optimizing the intent recognition model and the function call model based on the reward function R(t), the vehicle-mounted system first performs a strategy update based on the reward function R(t). That is, the vehicle-mounted system first updates the intent recognition strategy and the function call strategy of the vehicle-mounted system based on the reward function R(t) to obtain an updated intent recognition strategy and an updated function call strategy. Among them, the intent recognition strategy of the vehicle-mounted system is an application strategy for the vehicle-mounted system to recognize the intent of the vehicle environment perception information through the intent recognition model, and the function call strategy of the vehicle-mounted system is an application strategy for the vehicle-mounted system to call the system service to perform tasks through the function call model.

[0105] In some embodiments, the above step S301: updating the intention recognition strategy and function call strategy of the vehicle-mounted system based on the reward function may include the following steps:

[0106] Reinforcement learning is performed on a preset interaction data set based on the reward function to update the intent recognition strategy and function call strategy of the vehicle-mounted system.

[0107] It should be noted that the interaction data set includes the first state data of the vehicle-mounted system, the action data of the vehicle-mounted system performing intent recognition and function call execution corresponding to the first state data, the second state data of the vehicle-mounted system, and the action data of the vehicle-mounted system performing intent recognition and function call execution corresponding to the second state data, wherein the first state data and the second state data are the state data of the vehicle-mounted system at different times.

[0108] When the vehicle system updates the intention recognition strategy and function call strategy of the vehicle system based on the reward function R(t), the Q-learning strategy update formula shown below can be used to use the current reward function R(t) to perform reinforcement learning on the actual interaction data between the vehicle system and the vehicle user in the interaction data set, thereby adjusting the intention recognition strategy and function call strategy of the vehicle system according to the difference between the current intention recognition strategy and function call strategy of the vehicle system and the actual interaction data, thereby updating the intention recognition strategy and function call strategy of the vehicle system.

[0109] ;

[0110] It should be noted that in the above Q-learning strategy update formula, Q(s,a) is the Q value of the first state data s and the action data a corresponding to the first state data s. η is the learning rate, which controls the step size of each update. r is the current reward function R(t). γ is the discount factor, which weighs the importance of current rewards and future rewards. s′ is the second state data at the next moment, and a′ is the action data corresponding to the second state data at the next moment.

[0111] Step S302: Optimizing the model parameters of the intent recognition model based on the updated intent recognition strategy to obtain an optimized intent recognition model.

[0112] Step S303: Optimizing the model parameters of the function call model based on the updated function call strategy to obtain an optimized function call model.

[0113] After obtaining the updated intent recognition strategy, the vehicle system further adopts the model parameter update formula shown below to optimize the model parameters of the intent recognition model based on the updated intent recognition strategy, thereby obtaining an optimized intent recognition model, and optimizes the model parameters of the function call model based on the updated intent recognition strategy, thereby obtaining an optimized function call model. In this way, by adjusting the model parameters of the intent recognition model and the function call model, the optimized intent recognition model and the optimized function call module can be better adapted to new data and environments.

[0114] ;

[0115] It should be noted that in the above model parameter update formula, θ new is the updated model parameter, θ old is the model parameter before updating. λ is the learning rate. L is the loss function, which measures the difference between the model prediction and the actual result. The interactive dataset.

[0116] In some embodiments, the interaction dataset It can be a few-sample dataset. When the vehicle system updates the model parameters to optimize the intent recognition model and the function call model, it can use the few-sample learning technology to analyze the interaction dataset. The model parameters are updated by combining with reinforcement learning, and the updated model parameters are used to optimize and adjust the intent recognition model and the function call model.

[0117] In some embodiments, the vehicle system can optimize the intent recognition model and the function call model by configuring a reinforcement learning module. The vehicle system uses the interaction data set and environmental feedback (including performance data, execution status feedback information, and vehicle environment perception information) between the vehicle user as the input of the reinforcement learning module, and then uses the reinforcement learning module to adjust and optimize the difference between the current strategy (intent recognition strategy and function call strategy) and the actual interaction data through reinforcement learning through reinforcement learning algorithm, and finally obtains the optimized intent recognition model and function call model.

[0118] For example, please refer to Figure 4 , Figure 4 A schematic diagram of the application flow of the task processing method of the vehicle system provided in the embodiment of the present application in some embodiments involving the adaptive adjustment of the vehicle system. Figure 4 As shown, the vehicle-mounted system can optimize the intent recognition model and the function call model according to the algorithm flow shown below to obtain the optimized intent recognition model and the optimized function call model. That is, the vehicle-mounted system first performs the initialization step to initialize the strategy π and the model parameter θ, and sets the parameters α, β, and γ of the reward function. Afterwards, when the vehicle-mounted system interacts with the vehicle user, in response to the request issued by the vehicle user, the vehicle-mounted system performs intent recognition and function calls, thereby executing the user requirement task (the first user requirement task corresponding to the first user intention or the second user requirement task corresponding to the second user intention). In addition, the vehicle-mounted system records the interaction data with the vehicle user to form an interaction data set (including s, a, s′) and records the current reward r (in some embodiments, the vehicle-mounted system can form an interaction data set: s, a, r, s′) together with the interaction data. Afterwards, the vehicle-mounted system performs the feedback and reward calculation steps: the vehicle-mounted system calculates the reward based on the execution status feedback information (such as satisfaction, etc.) and performance data (such as response time and resource usage, etc.) fed back by the vehicle user: After that, the vehicle system updates the strategy based on the Q-learning algorithm: Finally, the vehicle system updates the model parameters of the intent recognition module and the function call model based on the model parameter update formula: , in order to optimize the intent recognition module and function call model. In addition, the vehicle system continues to receive new requests from vehicle users through iterative operations to enter the next interaction cycle, thereby repeating the interaction and strategy update process, so that the strategy and model parameters are continuously adjusted and optimized to improve the overall performance of the vehicle system.

[0119] It should be noted that the application strategy (later intention recognition strategy and function call strategy) output by the vehicle system through reinforcement learning mainly refers to selecting the most appropriate function call or response action under different user intentions and system states. Strategy updates or optimizations are continuously adjusted through reinforcement learning algorithms, thereby improving the accuracy and efficiency of the vehicle system in responding to user needs.

[0120] In this embodiment, the vehicle-mounted system uses reinforcement learning technology to respond to user needs and provide intelligent services to vehicle users. Through continuous user interaction data and feedback data, the application strategy and model parameters are adaptively adjusted, thereby achieving adaptive adjustment and optimization. In this way, the adaptive ability and task solving accuracy of the vehicle-mounted system in different task scenarios can be improved, thereby further improving the efficiency of the vehicle-mounted system in responding to user needs.

[0121] In addition, by combining the on-board system with reinforcement learning and few-sample learning techniques to update model parameters, the on-board system can also have the ability to self-optimize and quickly adapt to new needs.

[0122] In some embodiments, the vehicle-mounted system can perform multimodal intent recognition on vehicle environment perception information to combine voice, text and image data, and through multimodal information fusion, accurately identify the first user intention and / or the second user intention.

[0123] In some embodiments, the intent recognition model includes a multimodal intent recognition large model, such as a llama3 model and a qwen model.

[0124] Please refer to Figure 5 , Figure 5 for Figure 1 Schematic diagram of the detailed process flow of step S103.

[0125] like Figure 5 As shown, in some embodiments, the above-mentioned step S103: obtaining the second user intention by performing intent recognition on the vehicle environment perception information through the optimized intention recognition model may include steps S501 and S502 as shown below.

[0126] Step S501: Acquire vehicle environment perception information; wherein the vehicle environment perception information at least includes vehicle environment sound information and vehicle environment image information.

[0127] The in-vehicle system continuously obtains vehicle environment perception information while maintaining interaction with the vehicle user.

[0128] It should be noted that the vehicle environment perception information includes vehicle environment sound information and vehicle environment image information. Vehicle environment sound information includes voice information output by the vehicle user to the vehicle system, environmental noise information, etc. The vehicle system can collect sound around the vehicle environment and the internal environment by controlling the vehicle sound collection device (such as a microphone), thereby obtaining various environmental sound information including the voice information output by the vehicle user. In addition, the vehicle system can also obtain vehicle environment image information by controlling the vehicle image collection device (such as a camera) to collect images around the vehicle environment and the internal environment.

[0129] Step S502: Perform multimodal intention recognition on the vehicle environment sound information and the vehicle environment image information through the optimized multimodal intention recognition large model to obtain a second user intention.

[0130] After obtaining the optimized intention recognition big model (multimodal intention recognition big model), in order to respond to the vehicle users' new intelligent service demands for using the vehicle, the vehicle system performs multimodal intention recognition on the vehicle environment sound information and vehicle environment image information through the optimized multimodal intention recognition big model, and continuously fuses the recognized information of multiple modalities through the multimodal intention recognition big model to obtain the second user intention.

[0131] It should be noted that the vehicle-mounted system may obtain the first user intention by performing intent recognition on the vehicle environment perception information through an intention recognition model, or may obtain the first user intention by performing multimodal intent recognition processing on the vehicle environment perception information including multiple types of information such as sound, text and / or images through a large multimodal intention recognition model.

[0132] In some embodiments, the vehicle-mounted system can configure a multimodal intent recognition module, use a large multimodal intent recognition model (llama3 model and qwen model, etc.), take images, sounds, and text in the vehicle environment perception information as input, and comprehensively analyze and recognize the vehicle user's voice and personal image (recognize gestures and expressions) to assist in multimodal intent recognition and output the recognized user intent (such as a first user intention or a second user intention that includes at least image features).

[0133] In some embodiments, the task processing method of the vehicle-mounted system provided in the embodiment of the present application may further include the following steps:

[0134] Obtain the confidence level of the user intention recognition result corresponding to the vehicle environment sound information.

[0135] During each process of intent recognition for vehicle environment perception information, the vehicle-mounted system can first perform intent recognition on the vehicle environment sound information (especially the voice information or text of the vehicle user) through the text intention recognition module, and calculate the confidence level for the recognized user intention, thereby obtaining the confidence level of the user intention recognition result corresponding to the vehicle environment sound information.

[0136] In some embodiments, the vehicle-mounted system can calculate the confidence of the result obtained by performing intent recognition on the vehicle environment sound by pre-building a special confidence assessment model. For example, the vehicle-mounted system constructs the vehicle environment sound information of the known user's accurate intention and the result obtained by the vehicle-mounted system performing intent recognition on the vehicle environment sound information as a model training sample for model training, so that the model can accurately calculate the proportional relationship between different results in the recognition results and the known user's accurate intention, and then use the calculated ratio as the confidence of the result obtained by the vehicle-mounted system performing intent recognition on the vehicle environment sound information.

[0137] In the case where the in-vehicle system obtains the confidence of the user intention recognition result, the above step S502: performing multimodal intention recognition on the vehicle environment sound information and the vehicle environment image information through the optimized multimodal intention recognition large model may include the following steps:

[0138] When the confidence level is less than or equal to a preset confidence level threshold, multimodal intention recognition is performed on the vehicle environment sound information and the vehicle environment image information through the multimodal intention recognition large model.

[0139] After the in-vehicle system obtains the confidence level of the user intention recognition result, it compares the confidence level with a preset confidence threshold (for example, 80%). Then, if the comparison finds that the confidence level is less than or equal to the confidence threshold, the in-vehicle system determines that it is necessary to further perform multimodal intention recognition processing on the vehicle environment perception information, thereby triggering a multimodal intention recognition large model (such as the LLaMA3 multimodal large model), and then performing multimodal intention recognition on the vehicle environment sound information and vehicle environment image information, so as to output a more accurate user intention after comprehensive analysis based on the voice emotion features and image data.

[0140] In some embodiments, if the in-vehicle system compares the confidence level of the user intention recognition result with a preset confidence threshold and finds that the confidence level is greater than the confidence threshold, then in this case, the in-vehicle system determines that the current preliminary intention recognition of the vehicle environmental sound information has obtained a sufficiently accurate intention recognition result, and thus directly uses the intention recognition result as the user intention.

[0141] In some embodiments, the vehicle environment sound information includes voice information of the vehicle user, and the vehicle environment image information includes character image information of the vehicle user. In this case, the task processing method of the vehicle-mounted system provided in the embodiment of the present application may also include the following steps:

[0142] A first acquisition time of the voice information is acquired, and a second acquisition time of the character image information is acquired.

[0143] When the vehicle-mounted system controls the vehicle sound collection device to collect vehicle environment sound information, it also records the time information when the vehicle sound collection device collects each sound information in the vehicle environment. In this way, when the vehicle-mounted system obtains the voice information output by the vehicle user, it also obtains the first collection time of the voice information. Similarly, when the vehicle-mounted system controls the vehicle image collection device to collect vehicle environment image information, it also records the time information when the vehicle image collection device collects each image information in the vehicle environment. Therefore, when the vehicle-mounted system obtains the person image information of the vehicle user, it can also obtain the second collection time of the person image information.

[0144] Based on this, the above step S502: performing multimodal intention recognition on the vehicle environment sound information and the vehicle environment image information through the optimized multimodal intention recognition large model may also include the following steps:

[0145] When the first acquisition time and the second acquisition time are the same, multimodal intent recognition is performed on the voice information and the character image information through the optimized multimodal intent recognition large model.

[0146] After obtaining the first collection time and the second collection time, the in-vehicle system compares the first collection time with the second collection time. If the comparison finds that the first collection time is the same as the second collection time, it confirms that the vehicle user is expressing the need for the in-vehicle system to provide intelligent services through voice and body movements at the same time. In this way, the in-vehicle system triggers the multimodal intention recognition large model to perform multimodal intention recognition on the vehicle environment sound information and vehicle environment image information, and outputs user intentions with higher accuracy after comprehensive analysis based on voice emotional features and image data.

[0147] For example, assuming that a vehicle user says "navigate to the nearest gas station" while driving the vehicle and waves his hand at the same time, the in-vehicle system will use a large multimodal intent recognition model to comprehensively analyze the user's voice and gestures, thereby accurately recognizing that the user's intention is to find a gas station.

[0148] In this embodiment, the vehicle-mounted system combines the vehicle user's voice information, image data and even text information (if any), and uses a large multimodal intent recognition model (such as LLaMA3) to perform comprehensive analysis and intent recognition, thereby improving the accuracy and robustness of identifying user intentions, thereby being able to more comprehensively and accurately understand the vehicle user's intentions, allowing the vehicle-mounted system to accurately respond to the vehicle user's needs for intelligent service in complex and changing scenarios.

[0149] Next, a complete embodiment of the task processing method of the vehicle-mounted system provided by the embodiment of the present application is proposed.

[0150] Please refer to Figure 6 , Figure 6 A schematic diagram of a complete application flow involved in the task processing method of the vehicle-mounted system provided in an embodiment of the present application.

[0151] like Figure 6 As shown, in some embodiments, the vehicle-mounted system receives voice information output by the strategic user through voice during the process of interaction with the user. After receiving the voice information, the vehicle-mounted system first converts the voice into text, then queries the context information and related historical conversation records for text processing, and then performs preliminary intent recognition on the text to obtain the text intent recognition result output by the intent large model. Then, the vehicle-mounted system compares the confidence of the text intent recognition result with the preset confidence threshold, so that when the confidence of the text intent recognition result is low, the multimodal intent recognition large model is triggered to assist in the recognition of the vehicle environment perception information (including voice information and other vehicle environment image information) (i.e., multimodal intent recognition). Among them, in other cases where comprehensive analysis of multimodal data is required (such as the above-mentioned first acquisition time and the second acquisition time are the same, etc.), the vehicle-mounted system also triggers the multimodal intent recognition large model to assist in the recognition of the vehicle environment perception information.

[0152] Afterwards, when the confidence of the result of intent recognition is higher than the confidence threshold, the vehicle system uses the function call model (function call) to call the corresponding function or API according to the recognized user intent (first user intent or second user intent) to execute the task requested by the user (first user requirement task or second user requirement task). In addition, after executing the task, the vehicle system collects user feedback (execution status feedback information). In this way, the vehicle system can monitor the performance data and user feedback in real time through the monitoring module, use the optimization module to output the optimization strategy based on the reinforcement learning algorithm, and conduct few-sample learning on real-time new data (interaction data set) to continuously optimize the intent recognition strategy and function call strategy by combining reinforcement learning and few-sample learning technology, and further adjust the model parameters of the intent recognition model and the model parameters of the function call model in real time according to the optimized strategy, so as to obtain the optimized whole intent recognition model and function call model after data fine-tuning. In this way, it can ensure the best performance of the vehicle system in responding to user needs in different tasks or environmental scenarios.

[0153] See also Figure 7 The embodiment of the present application also provides a task processing device for an in-vehicle system, which can implement the task processing method for the in-vehicle system. The device includes an acquisition module 701, a model optimization module 702, and a task processing module 703.

[0154] The acquisition module 701 is used to acquire performance data of the vehicle-mounted system in executing a first user demand task, and to acquire execution status feedback information of the first user demand task; wherein the first user demand task is a task corresponding to a first user intention obtained by performing intention recognition on vehicle environment perception information through an intention recognition model;

[0155] A model optimization module 702 is used to optimize the intention recognition model and the function call model based on the performance data and the execution status feedback information to obtain an optimized intention recognition model and an optimized function call model; wherein the function call model is used for the vehicle system to call the system service when executing the first user demand task;

[0156] The task processing module 703 is used to perform intent recognition on the vehicle environment perception information through the optimized intention recognition model to obtain the second user intention, and to call the system service through the optimized function call model to execute the second user demand task corresponding to the second user intention.

[0157] In some embodiments, the model optimization module 702 is also used to calculate a reward based on the performance data and the execution status feedback information to obtain a reward function; and to optimize the intent recognition model and the function call model based on the reward function to obtain an optimized intent recognition model and an optimized function call model.

[0158] In some embodiments, the model optimization module 702 is also used to update the intention recognition strategy and function call strategy of the vehicle-mounted system based on the reward function to obtain an updated intention recognition strategy and an updated function call strategy; wherein the intention recognition strategy is an application strategy for performing intent recognition on vehicle environment perception information through an intention recognition model, and the function call strategy is an application strategy for calling system services to perform tasks through a function call model; based on the updated intention recognition strategy, the model parameters of the intention recognition model are optimized to obtain an optimized intention recognition model; and, based on the updated function call strategy, the model parameters of the function call model are optimized to obtain an optimized function call model.

[0159] In some embodiments, the model optimization module 702 is also used to perform reinforcement learning processing on a preset interaction data set based on the reward function to update the intention recognition strategy and function call strategy of the vehicle-mounted system; wherein the interaction data set includes first state data of the vehicle-mounted system, action data of the vehicle-mounted system performing intent recognition and function call execution corresponding to the first state data, second state data of the vehicle-mounted system, and action data of the vehicle-mounted system performing intent recognition and function call execution corresponding to the second state data, wherein the first state data and the second state data are state data of the vehicle-mounted system at different times.

[0160] In some embodiments, the intention recognition model includes a multimodal intention recognition large model, and the task processing module 703 is also used to obtain vehicle environment perception information; wherein the vehicle environment perception information includes at least vehicle environment sound information and vehicle environment image information; the vehicle environment sound information and the vehicle environment image information are multimodally recognized by the optimized multimodal intention recognition large model to obtain the second user intention.

[0161] In some embodiments, the task processing module 703 is also used to obtain the confidence of the user intention recognition result corresponding to the vehicle environment sound information; and, when the confidence is less than or equal to a preset confidence threshold, multimodal intention recognition is performed on the vehicle environment sound information and the vehicle environment image information through the multimodal intention recognition large model.

[0162] In some embodiments, the task processing module 703 is also used to obtain a first acquisition time of the voice information and a second acquisition time of the character image information; and, when the first acquisition time and the second acquisition time are the same, multimodal intent recognition is performed on the voice information and the character image information through an optimized multimodal intent recognition large model.

[0163] The specific implementation of the task processing device of the vehicle-mounted system provided in the embodiment of the present application is basically the same as the specific implementation of the task processing method of the vehicle-mounted system described above, and will not be repeated here.

[0164] The embodiment of the present application also provides an electronic device, the electronic device includes a memory and a processor, the memory stores a computer program, and the processor implements the task processing method of the vehicle-mounted system when executing the computer program. The electronic device can be any intelligent terminal including a tablet computer, a vehicle-mounted computer, etc.

[0165] See also Figure 8 , Figure 8 The hardware structure of an electronic device of another embodiment is illustrated, and the electronic device includes:

[0166] The processor 801 may be implemented by a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (Application Specific Integrated Circuit, ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of the present application;

[0167] The memory 802 can be implemented in the form of a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 802 can store an operating system and other application programs. When the technical solution provided in the embodiment of this specification is implemented by software or firmware, the relevant program code is stored in the memory 802, and the processor 801 calls and executes the task processing method of the vehicle-mounted system in the embodiment of this application;

[0168] Input / output interface 803, used to implement information input and output;

[0169] The communication interface 804 is used to realize the communication interaction between the device and other devices. The communication can be realized through wired means (such as USB, network cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.);

[0170] A bus 805 that transmits information between various components of the device (e.g., the processor 801, the memory 802, the input / output interface 803, and the communication interface 804);

[0171] The processor 801 , the memory 802 , the input / output interface 803 and the communication interface 804 are connected to each other in communication within the device via a bus 805 .

[0172] An embodiment of the present application also provides a vehicle, which is equipped with an electronic device, the electronic device includes a memory and a processor, the memory stores a computer program, and the processor implements the task processing method of the above-mentioned vehicle-mounted system when executing the computer program.

[0173] An embodiment of the present application also provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, it implements the task processing method of the above-mentioned vehicle-mounted system.

[0174] The memory, as a non-transient computer-readable storage medium, can be used to store non-transient software programs and non-transient computer executable programs. In addition, the memory may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory includes a memory remotely arranged relative to the processor, and these remote memories can be connected to the processor via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0175] The embodiment of the present application also provides a computer program product, including a computer program. The steps implemented when the computer program is executed by a processor are basically the same as the specific embodiment of the task processing method of the above-mentioned vehicle-mounted system, and will not be repeated here.

[0176] The embodiments described in the embodiments of the present application are intended to more clearly illustrate the technical solutions of the embodiments of the present application and do not constitute a limitation on the technical solutions provided in the embodiments of the present application. Those skilled in the art will appreciate that with the evolution of technology and the emergence of new application scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

[0177] Those skilled in the art will appreciate that the technical solutions shown in the figures do not constitute a limitation on the embodiments of the present application, and may include more or fewer steps than shown in the figures, or a combination of certain steps, or different steps.

[0178] The device embodiments described above are merely illustrative, and the units described as separate components may or may not be physically separated, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0179] Those skilled in the art will appreciate that all or some of the steps in the methods disclosed above, and the functional modules / units in the systems and devices may be implemented as software, firmware, hardware, or a suitable combination thereof.

[0180] The terms "first", "second", "third", "fourth", etc. (if any) in the specification of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0181] It should be understood that in the present application, "at least one (item)" means one or more, and "plurality" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the objects associated before and after are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0182] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the above units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0183] The units described above as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0184] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0185] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including multiple instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, referred to as ROM), random access memory (Random Access Memory, referred to as RAM), disk or optical disk and other media that can store programs.

[0186] The preferred embodiments of the present invention are described above with reference to the accompanying drawings, but the scope of the rights of the present invention is not limited thereto. Any modification, equivalent substitution and improvement made by a person skilled in the art without departing from the scope and essence of the present invention should be within the scope of the rights of the present invention.

Claims

1. A task processing method for a vehicle-mounted system, characterized in that: The method comprises: Acquire performance data of the vehicle-mounted system executing a first user requirement task, and acquire execution status feedback information of the first user requirement task; wherein the first user requirement task is a task corresponding to a first user intention obtained by performing intention recognition on vehicle environment perception information through an intention recognition model; Based on the performance data and the execution status feedback information, the intention recognition model and the function call model are optimized to obtain an optimized intention recognition model and an optimized function call model; wherein the function call model is used to call the system service when the vehicle-mounted system executes the first user demand task; Performing intent recognition on the vehicle environment perception information through the optimized intent recognition model to obtain a second user intent; Calling a system service through the optimized function call model to execute a second user demand task corresponding to the second user intention; The performance data includes the response time and resource usage of the vehicle-mounted system in executing the first user-required task, and obtaining the performance data of the vehicle-mounted system in executing the first user-required task includes: In the process of the vehicle-mounted system executing the first user requirement task, the monitoring module of the vehicle-mounted system monitors and obtains the response time and the resource usage in real time; The optimizing the intention recognition model and the function call model based on the performance data and the execution status feedback information includes: The response time, the resource usage, and the execution status feedback information monitored and acquired by the monitoring module in real time are used as inputs of the optimization module of the vehicle-mounted system, and the optimization module dynamically adjusts the intent recognition strategy and the function call strategy according to the input data and outputs the adjustment strategy; The intent recognition model and the function call model are dynamically optimized according to the adjustment strategy output by the optimization module.

2. The method according to claim 1, characterized in that The optimizing the intention recognition model and the function call model based on the performance data and the execution status feedback information to obtain an optimized intention recognition model and an optimized function call model includes: Performing reward calculation based on the performance data and the execution status feedback information to obtain a reward function; The intention recognition model and the function call model are optimized based on the reward function to obtain an optimized intention recognition model and an optimized function call model.

3. The method according to claim 2, characterized in that The optimizing the intention recognition model and the function call model based on the reward function includes: Based on the reward function, the intention recognition strategy and the function call strategy of the vehicle system are updated to obtain an updated intention recognition strategy and an updated function call strategy; wherein the intention recognition strategy is an application strategy for performing intention recognition on vehicle environment perception information through an intention recognition model, and the function call strategy is an application strategy for calling a system service to perform a task through a function call model; Optimizing the model parameters of the intent recognition model based on the updated intent recognition strategy to obtain an optimized intent recognition model; The model parameters of the function call model are optimized based on the updated function call strategy to obtain an optimized function call model.

4. The method according to claim 3, characterized in that The updating of the intention recognition strategy and the function calling strategy of the vehicle-mounted system based on the reward function includes: Performing reinforcement learning processing on a preset interaction data set based on the reward function to update an intention recognition strategy and a function call strategy for the vehicle-mounted system; Among them, the interaction data set includes the first state data of the vehicle-mounted system, the action data of the vehicle-mounted system performing intent recognition and function call execution corresponding to the first state data, the second state data of the vehicle-mounted system, and the action data of the vehicle-mounted system performing intent recognition and function call execution corresponding to the second state data, and the first state data and the second state data are the state data of the vehicle-mounted system at different times.

5. The method according to claim 1, characterized in that The intention recognition model includes a multimodal intention recognition large model, and the step of performing intention recognition on the vehicle environment perception information through the optimized intention recognition model to obtain the second user intention includes: Acquiring vehicle environment perception information; wherein the vehicle environment perception information at least includes vehicle environment sound information and vehicle environment image information; The vehicle environment sound information and the vehicle environment image information are subjected to multimodal intention recognition by using the optimized multimodal intention recognition large model to obtain the second user intention.

6. The method according to claim 5, characterized in that The method further comprises: Obtaining the confidence level of the user intention recognition result corresponding to the vehicle environment sound information; The multimodal intention recognition is performed on the vehicle environment sound information and the vehicle environment image information by using the optimized multimodal intention recognition large model, including: When the confidence level is less than or equal to a preset confidence level threshold, multimodal intention recognition is performed on the vehicle environment sound information and the vehicle environment image information through the multimodal intention recognition large model.

7. The method according to claim 5, characterized in that The vehicle environment sound information includes voice information of the vehicle user, and the vehicle environment image information includes character image information of the vehicle user; The method further comprises: Acquire a first acquisition time of the voice information, and acquire a second acquisition time of the character image information; The multimodal intention recognition is performed on the vehicle environment sound information and the vehicle environment image information by using the optimized multimodal intention recognition large model, including: When the first acquisition time and the second acquisition time are the same, multimodal intent recognition is performed on the voice information and the character image information through the optimized multimodal intent recognition large model.

8. A task processing device for a vehicle-mounted system, characterized in that: The device comprises: an acquisition module, used to acquire performance data of the vehicle-mounted system in executing a first user requirement task, and to acquire execution status feedback information of the first user requirement task; wherein the first user requirement task is a task corresponding to a first user intention obtained by performing intention recognition on vehicle environment perception information through an intention recognition model; A model optimization module, configured to optimize the intention recognition model and the function call model based on the performance data and the execution status feedback information, to obtain an optimized intention recognition model and an optimized function call model; wherein the function call model is used to call a system service when the vehicle-mounted system executes the first user demand task; A task processing module, configured to perform intent recognition on the vehicle environment perception information through the optimized intent recognition model to obtain a second user intent, and to call a system service through the optimized function call model to execute a second user demand task corresponding to the second user intent; The performance data includes the response time and resource usage of the vehicle-mounted system when executing the first user-required task, and the acquisition module is further used to monitor and acquire the response time and resource usage in real time during the vehicle-mounted system executing the first user-required task through the monitoring module of the vehicle-mounted system; The model optimization module is also used to use the response time, resource usage, and execution status feedback information monitored and acquired in real time by the monitoring module as input to the optimization module of the vehicle system, and the optimization module dynamically adjusts the intention recognition strategy and function call strategy according to the input data and outputs the adjustment strategy; and dynamically optimizes the intention recognition model and the function call model according to the adjustment strategy output by the optimization module.

9. An electronic device, characterized in that: The electronic device includes a memory and a processor, the memory stores a computer program, and the processor implements the task processing method of the vehicle-mounted system according to any one of claims 1 to 7 when executing the computer program.

10. A vehicle, characterized in that: The vehicle includes an electronic device, the electronic device includes a memory and a processor, the memory stores a computer program, and the processor implements the task processing method of the vehicle-mounted system according to any one of claims 1 to 7 when executing the computer program.

11. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the task processing method of the vehicle-mounted system according to any one of claims 1 to 7 is implemented.

12. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the task processing method of the in-vehicle system according to any one of claims 1 to 7 are implemented.

Images