A method, apparatus and vehicle for skill scheduling

Abstract

The present disclosure provides a skill scheduling method, device and vehicle, the method comprising: acquiring a multi-modal interaction instruction and writing the multi-modal interaction instruction into a shared state area; reading the multi-modal interaction instruction from the shared state area by a cockpit agent, performing intent analysis on the multi-modal interaction instruction, and determining an intent analysis result corresponding to the multi-modal interaction instruction; determining a target cockpit skill corresponding to the intent analysis result from a cockpit skill pool according to the intent analysis result; evaluating a risk level of the target cockpit skill to obtain a risk level evaluation result; and executing the target cockpit skill according to a target processing path corresponding to the risk level evaluation result to obtain a skill execution result and output the skill execution result.

Description

Technical Field

[0001] This disclosure relates to the field of computer technology, and more particularly to a method, apparatus, and vehicle for skill scheduling. Background Technology

[0002] With the development of intelligent cockpit technology, in-vehicle systems are gradually evolving into intelligent cockpit entities with autonomous perception, decision-making, and interaction capabilities, becoming the core control hub of the next generation of intelligent cockpits.

[0003] In related technologies, cockpit intelligent agents are mostly based on a unidirectional acyclic graph architecture to construct skill scheduling processes. Each skill node is executed sequentially according to a preset order, and task flow is realized through interface calls. However, in this architecture, the states of each skill node are maintained independently, resulting in inconsistent global states and affecting the continuity of long-term interactions. In addition, security checks are often deployed as add-on modules, which are easily bypassed and pose security risks. Therefore, a new approach is urgently needed to improve the above problems. Summary of the Invention

[0004] One objective of this disclosure is to provide a method for skill scheduling to address the issues of inconsistent global states and safe skill scheduling; another objective is to provide a device for skill scheduling; and a third objective is to provide a vehicle.

[0005] To achieve the above objectives, the technical solution adopted in this disclosure is as follows: This disclosure proposes a skill scheduling method, which includes, Acquire multimodal interaction commands and write them into the shared state area; The cockpit agent reads multimodal interaction commands from the shared state area, performs intent parsing on the multimodal interaction commands, and determines the intent parsing result corresponding to the multimodal interaction commands. Based on the intent parsing results, determine the target cockpit skill corresponding to the intent parsing results from the cockpit skill pool; The risk level of the target cockpit skills is assessed, and the risk level assessment results are obtained; Execute the target cockpit skills according to the target processing path corresponding to the risk level assessment results, obtain the skill execution results, and output them.

[0006] Based on the aforementioned technical means, multimodal interaction commands are first acquired and written into a shared state area to achieve unified management of the global state, avoiding inconsistencies caused by decentralized state management. Next, the cockpit agent reads the multimodal interaction commands and performs intent parsing to accurately identify the user's interaction intent. Then, the target cockpit skill is matched based on the intent parsing results, and its risk level is assessed. Finally, based on the assessment results, the corresponding target processing path is selected to execute the skill, ensuring that high-risk skills operate in a safe environment and low-risk skills are executed efficiently, thereby achieving end-to-end security control. In this way, by using the shared state area as a global data foundation, the problems of data silos and state asynchrony between nodes in a unidirectional acyclic graph architecture are effectively solved. Simultaneously, the risk-based routing mechanism enhances system security and stability.

[0007] In some embodiments, the target cockpit skill is executed according to the target processing path corresponding to the risk level assessment result, and the skill execution result is obtained, including: If the risk level assessment result is Level 1, the target cockpit skill is executed through the first processing path corresponding to Level 1 to obtain the skill execution result; the first processing path includes at least file permission verification, vehicle environment isolation, tamper interception, execution of the target cockpit skill in the sandbox, and desensitization processing of the skill execution result; If the risk level assessment result is Level 2, the target cockpit skill is executed through the second processing path corresponding to Level 2 to obtain the skill execution result; Write the skill execution result to the shared state area.

[0008] Based on the aforementioned technical means, for Level 1 (i.e., high-risk) skills, multiple protective measures, including file permission verification, vehicle-mounted environment isolation, tamper interception, and sandbox execution, ensure their operation in a secure environment and prevent damage to the vehicle system. For Level 2 (i.e., low-risk) skills, execution is performed directly to improve response efficiency. After execution, the skill execution result is written back to the shared state area to ensure state consistency. This achieves differentiated security control for skills of different risk levels, ensuring the security of high-risk operations while also considering the real-time nature of low-risk operations, effectively enhancing the overall system's security capabilities.

[0009] In some embodiments, before determining the target cockpit skill matching the intent resolution result from the cockpit skill pool based on the intent resolution result, the method further includes: Perform scene recognition on multimodal interaction commands and determine the scene recognition result corresponding to the multimodal interaction commands; Based on the scene recognition results, execute the initialization operations corresponding to the scene recognition results. The scene recognition results include at least children's scenes, movie-watching scenes, parking scenes, vehicle parking scenes, and traffic warning scenes.

[0010] Based on the aforementioned technical means, by performing scene recognition on multimodal interaction commands, the current cockpit scene can be dynamically determined, such as a children's scene or a movie-watching scene, and corresponding initialization operations can be executed accordingly, such as enabling content filtering, adjusting permission settings, or configuring specific functional modules. This allows for pre-configuration of system resources and functional modules, reducing the response latency of user commands and improving the interaction efficiency and stability of the cockpit's intelligent agent.

[0011] In some embodiments, the method further includes: When the cockpit agent identifies an abnormal skill execution result, it performs data comparison and constraint verification based on the operation data of the target node in the shared state area to determine the abnormal problem point. The target node is the node related to the skill execution result. Correct any abnormal issues, obtain the corrected skill execution results, and output them.

[0012] Based on the aforementioned technical means, after the cockpit agent identifies anomalies in skill execution results, it performs data comparison and constraint verification based on the operational data of target nodes within the shared state area to determine the anomaly points. These anomalies are then corrected, resulting in the corrected skill execution results, which are then output. This allows for precise location and efficient repair of skill execution anomalies, thereby enhancing the system's self-healing capabilities and stability, and ultimately improving the overall user experience and safety of the intelligent cockpit.

[0013] In some embodiments, the method further includes, If the stored data in the shared state region is greater than or equal to a preset threshold, the stored data in the shared state region is state-trimmed according to preset trimming parameters to obtain trimmed data. State pruning includes at least session context cleanup, temporary variable deletion, historical log archiving, and target data retention. The target data includes at least user preference data and vehicle status data.

[0014] Based on the aforementioned technical means, state pruning is triggered when the accumulated data in the shared state area reaches a preset threshold. This effectively prevents memory explosion caused by long session interactions, ensuring the stability and response speed of the system under long-term operation, and meeting the high stability requirements of automotive scenarios.

[0015] In some embodiments, the method further includes: Preset trimming parameters are generated based on the user's session preference data.

[0016] Based on the aforementioned technical methods, personalized preset trimming parameters are dynamically generated by analyzing user session preference data. This makes the trimming process more aligned with actual user habits, avoiding excessive cleanup that could negatively impact user experience, while ensuring efficient utilization of system resources. This design enhances the intelligence level of state management, achieving a balance between personalization and system stability.

[0017] In some embodiments, prior to intent parsing of multimodal interaction instructions, the following is included: The cockpit intelligent agent performs security verification on multimodal interaction commands to determine the first security verification result; the security verification includes at least voice desensitization verification, target keyword interception, and target information recognition. If the first security check result indicates that the check has passed, update the security check identifier corresponding to the multimodal interaction command in the shared state area; If the first security verification result indicates that the verification fails, the response result is output according to the first protection template, and the first record data is formed based on the multimodal interaction instructions.

[0018] Based on the aforementioned technical means, multimodal interaction commands undergo voice anonymization, keyword interception, and sensitive information identification through security verification, effectively preventing malicious input. Upon successful verification, the security identifier is updated to ensure the security of subsequent processes; if verification fails, the system is immediately intercepted and a protective template response is output, while simultaneously recording the data. This design constructs the first line of defense for end-to-end security protection, significantly enhancing the system's resistance to attacks.

[0019] In some embodiments, after obtaining the skill execution result, the method further includes, A second safety verification is performed on the skill execution results to determine the second safety verification result; the second safety verification includes at least content screening and driving safety verification. If the second security check result indicates that the check has passed, the skill execution result will be output. If the second security verification result indicates that the verification fails, the response result is output according to the second protection template, and the second record data is formed based on the skill execution result.

[0020] Based on the aforementioned technical means, a secondary security verification step is added after skill execution to screen the execution result for content compliance and driving safety, preventing the output of illegal or dangerous content. If the verification passes, the content is output normally; if it fails, a protective template response is triggered and recorded. This design achieves closed-loop security control over skill execution results, further ensuring the security and compliance of the user interaction process.

[0021] In some embodiments, the cockpit skill pool includes at least cockpit hardware control skills, navigation map update skills, multimedia playback skills, vehicle status diagnosis skills, cockpit configuration skills, interactive question and answer skills, children's scenario skills, and sleep mode skills.

[0022] Based on the aforementioned technical means, by constructing a modular cockpit skill pool, the cockpit intelligent agent can accurately respond to diverse user needs, thereby improving the scalability, maintainability, and user experience of the intelligent cockpit system.

[0023] In some embodiments, the method further includes: If no feedback instruction regarding the skill execution result is received within the first duration, a sleep mode is triggered, and a cleanup operation is performed through the reducer.

[0024] Based on the aforementioned technical means, if no user feedback command is received within a set time, a sleep mode is triggered, and a deep cleanup operation is performed through a reducer to shut down unnecessary functions, reduce power consumption, and release memory resources. This enables intelligent power management and dynamic memory resource reclamation of the cockpit system, preventing system lag and memory bloat caused by prolonged inactivity, and ensuring high stability and low power consumption of the vehicle system during long-term operation.

[0025] This disclosure proposes a skill scheduling apparatus, the apparatus comprising, The acquisition module is used to acquire multimodal interaction commands and write them into the shared state area. The processing module is used to read multimodal interaction commands from the shared state area through the cockpit agent, perform intent parsing on the multimodal interaction commands, and determine the intent parsing result corresponding to the multimodal interaction commands; based on the intent parsing result, determine the target cockpit skill corresponding to the intent parsing result from the cockpit skill pool; assess the risk level of the target cockpit skill and obtain the risk level assessment result; and execute the target cockpit skill according to the target processing path corresponding to the risk level assessment result, obtain the skill execution result, and output it.

[0026] This disclosure provides a vehicle, which includes a vehicle controller, a memory, and a computer program stored in the memory and capable of running on the vehicle controller. When the computer program is executed by the vehicle controller, it implements the steps of the aforementioned method.

[0027] This disclosure provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the aforementioned method.

[0028] This disclosure provides a computer program product, including a computer program or instructions, which, when executed by a processor, implement the steps of the aforementioned method.

[0029] The skill scheduling method provided in this disclosure first acquires multimodal interaction instructions and writes them into a shared state region to achieve unified management of the global state and avoid inconsistencies caused by decentralized state management. Next, the cockpit agent reads the multimodal interaction instructions and performs intent parsing to accurately identify the user's interaction intent. Then, based on the intent parsing results, it matches target cockpit skills and assesses their risk levels. Finally, based on the assessment results, it selects the corresponding target processing path to execute the skill, ensuring that high-risk skills operate in a safe environment and low-risk skills are executed efficiently, thereby achieving end-to-end security control. In this way, by using the shared state region as a global data foundation, the problem of data silos and state asynchrony between nodes in a unidirectional acyclic graph architecture is effectively solved. Simultaneously, the risk-based routing mechanism improves system security and stability. Attached Figure Description

[0030] Figure 1 This is a flowchart illustrating a skill scheduling method provided in an embodiment of this disclosure. Figure 1 ; Figure 2 This is a flowchart illustrating a skill scheduling method provided in an embodiment of this disclosure. Figure 2 ; Figure 3 This is a schematic diagram of the structure of a skill scheduling device provided in an embodiment of this disclosure; Figure 4 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this disclosure.

[0031] It should be noted that the terms "first" and "second" mentioned above are only used to distinguish between different options and do not represent the degree of superiority or inferiority of the options or their priority in the implementation process. Detailed Implementation

[0032] To make the objectives, technical solutions, and advantages of this disclosure clearer, the disclosure will be further described in detail below with reference to the accompanying drawings. The described embodiments should not be regarded as limitations on this disclosure. All other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this disclosure.

[0033] To make the objectives, technical solutions, and advantages of the embodiments of this disclosure clearer, the specific technical solutions of the application will be further described in detail below with reference to the accompanying drawings of the embodiments of this disclosure. The following embodiments are used to illustrate this disclosure, but are not intended to limit the scope of this disclosure.

[0034] In the following description, references are made to “some embodiments,” which describe a subset of all possible embodiments. However, it is understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0035] In the following description, the terms "first," "second," and "third" are used only to distinguish different objects and do not represent a specific order of objects or have any chronological limitation. It is understood that "first," "second," and "third" may be interchanged in a specific order or sequence where permitted, so that the embodiments of this disclosure described herein can be implemented in an order other than that illustrated or described herein.

[0036] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this disclosure belongs. The terminology used herein is for the purpose of describing embodiments of this disclosure only and is not intended to be limiting of this disclosure.

[0037] This disclosure provides a method, apparatus, and vehicle for skill scheduling. In practical applications, the skill scheduling method can be implemented by an on-board terminal. The skill scheduling method provided in this disclosure will now be described using a vehicle as the executing entity.

[0038] Figure 1 This is a flowchart illustrating a skill scheduling method provided in an embodiment of the present disclosure, as shown below. Figure 1 As shown, the skill scheduling method can be implemented through steps 101-105.

[0039] Step 101: Obtain the multimodal interaction instructions and write them into the shared state area.

[0040] In some embodiments, multimodal interaction commands from the user are obtained through a multimodal interaction entry layer. Exemplarily, these commands may originate from voice wake-up, central control touchscreen operation, steering wheel physical buttons, or automatic triggering by in-vehicle sensors. For example, the user might say: "Turn on the air conditioning and set it to 24 degrees," or click the "Navigate Home" button on the central control screen, or press the voice button on the steering wheel to issue the command.

[0041] In some embodiments, after obtaining the multimodal interaction command, the multimodal interaction command can be preprocessed to obtain a processed multimodal interaction command. For example, the preprocessing includes: speech-to-text processing, coordinate mapping processing, and encoding parsing processing.

[0042] The processed multimodal interaction commands can be stored in a structured data format, such as JSON. This structured data is then written to a shared state region, which can also be called a shared state unit of the state graph nodes.

[0043] This shared state area serves as the data foundation for the cockpit's intelligent agent, storing not only currently processed multimodal interaction commands but also historical session contexts, security verification identifiers, skill call parameters, system operating status, and other data to ensure data consistency across all nodes. For example, when a user issues two multimodal interaction commands consecutively—playing music and adjusting volume—the second command can read the audio playback status after the first command was executed, achieving seamless transition.

[0044] In some embodiments, the shared state region can adopt a graph structure, where each node in the graph represents a state type, and all nodes share the same memory space, fundamentally solving the data silo problem caused by each node maintaining its own data independently.

[0045] In some embodiments, a skill scheduling system is provided. The skill scheduling system includes a multimodal interaction entry layer, a cockpit agent, a skill execution layer (also known as a skill execution and safety protection layer), and an output feedback layer (also known as an output feedback and closed-loop layer).

[0046] The multimodal interaction entry layer includes a multi-source command acquisition module, a command preprocessing module, and a pre-security screening module. The multi-source command acquisition module acquires multimodal interaction commands. The command preprocessing module preprocesses the multimodal interaction commands to obtain the processed commands. The pre-security screening module performs security checks on the processed multimodal interaction commands, identifying malicious command formats and illegal characters, providing pre-filtering for subsequent deep security verification.

[0047] The cockpit intelligent agent includes a shared state area, an intent parsing and scene recognition unit, a state pruning unit, and a global scheduling and routing unit.

[0048] The shared state area serves as the data foundation, storing the state of all nodes. For example... Figure 2 As shown, the shared state area includes at least: user interaction commands, session context memory, security injection verification identifier, Skills call parameters, sandbox operation audit logs, content compliance markers, real-time status of vehicle hardware, cockpit configuration / map / firmware status, child scene switch configuration, active sleep timeout / wake-up status, sensitive information protection markers, and state accumulation / memory usage markers.

[0049] The intent parsing and scene recognition module is used to perform intent parsing and scene recognition on multimodal interaction commands, and obtain intent parsing results and scene recognition results.

[0050] The state trimming unit is used to trim the stored data in the shared state area to obtain the trimmed data.

[0051] The global scheduling and routing unit is used to generate skill call parameters based on the intent parsing results and scene recognition results, and to route the scheduling request to the corresponding cockpit skill in order to output the final skill execution result.

[0052] The skill execution and safety protection layer includes the cockpit skill pool, skill risk classification and routing unit, and sandbox isolation protection sub-graph.

[0053] The cockpit skill pool adopts a modular approach, including multiple cockpit skills. For example... Figure 2 As shown, the cockpit skill pool includes cockpit hardware control skills, navigation map update skills, multimedia playback skills, vehicle status diagnosis skills, cockpit configuration skills, interactive question and answer skills, children's scenario skills, and sleep mode skills.

[0054] Among them, the cockpit hardware control skills are used to control various physical hardware devices in the smart cockpit, such as adjusting the air conditioning temperature, controlling the window lifting and lowering, switching the lighting mode, and adjusting the seat position.

[0055] The navigation map update feature allows users to acquire, download, install, and manage map data for in-vehicle navigation systems. It automatically detects whether map versions are outdated and prompts users or automatically downloads update packages when updates are available, ensuring navigation accuracy and route planning precision.

[0056] Multimedia playback technology is used to play audio, video and other content in the cockpit, including music playback, video viewing, radio listening and audiobook playback.

[0057] Vehicle status diagnostics are used to monitor and analyze vehicle operating status in real time, including engine status, battery level, tire pressure, braking system, and fault code reading.

[0058] The cockpit configuration editing skills are used to manage cockpit personalization settings, such as interface theme switching, volume adjustment, voice assistant wake word settings, and shortcut function binding.

[0059] Interactive question-answering skills refer to intelligent dialogue based on natural language processing and knowledge bases, used to answer users' questions.

[0060] Child-friendly scenario skills refer to safety and content filtering modules designed for child passengers, including screen content tiered filtering, application permission restrictions, call blocking, and emergency contact settings. Child-friendly scenario skills automatically activate when the child-friendly scenario is enabled, preventing children from accidentally performing dangerous actions or accessing inappropriate content.

[0061] Sleep mode skills refer to energy-saving and safety protection functions that are activated when the user has no interaction for a long time, including turning off unnecessary electronic devices, reducing screen brightness, locking some functions, and continuously listening for wake-up commands.

[0062] The skills mentioned above in the cockpit skill pool enable the cockpit agent to respond accurately to diverse user needs, thereby improving the scalability, maintainability, and user experience of the intelligent cockpit system.

[0063] The skill risk classification routing unit is used to assess the risk level of the target cockpit skills, obtain the risk level assessment result, and route the target processing path corresponding to the risk level assessment result.

[0064] The sandbox isolation protection sub-graph is used to execute Level 1 cockpit skills to ensure that the execution of Level 1 cockpit skills does not damage the vehicle's systems.

[0065] The output feedback and closed-loop layer includes a multimodal output control unit, an active sleep management unit, and a full-process log archiving unit.

[0066] The multimodal output control unit is used to provide feedback on operation information to the user through multimodal methods such as central control screen display, voice broadcast, and hardware status synchronization based on the skill execution results, ensuring that the user obtains the skill execution results in a timely manner.

[0067] The active sleep control unit monitors the user's interaction status in real time. When no interaction is detected, it triggers sleep mode, shuts down unnecessary cockpit functions, reduces system power consumption, and continuously listens for wake-up commands while in sleep mode.

[0068] The full-process log archiving unit is used to uniformly archive all interaction processes, skill execution records, security verification results, and abnormal behaviors, and synchronize them to the vehicle backend management terminal to achieve traceability and auditability of all operations.

[0069] During the power-on initialization phase, the communication middleware is activated to establish a real-time communication link between the multimodal interaction entry layer, cockpit agent, skill execution layer, and output feedback layer in the aforementioned skill invocation system. This ensures that the end-to-end response time for multimodal interaction commands, from acquisition to writing, is controlled within milliseconds. Simultaneously, the pre-security screening module performs preliminary detection on the multimodal interaction commands, filtering out obviously illegal characters or malicious formats to prevent invalid or harmful data from entering the shared state area.

[0070] Step 102: Read multimodal interaction commands from the shared state area through the cockpit intelligent agent, perform intent parsing on the multimodal interaction commands, and determine the intent parsing result corresponding to the multimodal interaction commands.

[0071] In some embodiments, the cockpit agent is the central decision-making unit of the cockpit system, used to receive, parse, schedule and respond to all multimodal interaction commands.

[0072] In some embodiments, the process of performing intent parsing on multimodal interaction instructions and determining the intent parsing result corresponding to the multimodal interaction instructions may include: performing intent parsing on multimodal interaction instructions based on global state information such as historical session context, user preferences, and current scenario, and obtaining intent parsing results.

[0073] For example, intent parsing of multimodal interaction commands can be performed using Natural Language Understanding (NLU) and / or pre-trained language models or rule engines to obtain intent parsing results. For instance, when a user says, "Find me the nearest gas station," the cockpit agent parses the intent as "search for nearby gas stations" and extracts the entity "gas station" and the implicit condition "nearest." The intent parsing results can include fields such as intent category, entity label, confidence score, and scene context. These results can then be recorded in a structured format in a shared state area.

[0074] Step 103: Based on the intent parsing results, determine the target cockpit skill corresponding to the intent parsing results from the cockpit skill pool.

[0075] As described above, the cockpit skill pool includes multiple cockpit skills. These cockpit skills interface with the cockpit agent through standardized interfaces, enabling functional decoupling and flexible expansion.

[0076] Once the cockpit agent receives the intent parsing results, it performs a precise match within the cockpit skill pool based on the intent category and entity label in the results. For example, if the intent parsing result is to search for nearby gas stations, the navigation map update skill will be matched. Furthermore, the cockpit skill pool can also configure new cockpit skills and update existing ones. For instance, it can configure weather forecast skills or external environment monitoring skills, which users can use without restarting the system.

[0077] In some embodiments, the process of determining the target cockpit skill corresponding to the intent parsing result from the cockpit skill pool may further include: determining the priority and conflict handling of multiple target cockpit skills. When multiple cockpit skills satisfy the same intent (e.g., turning on the air conditioner can be performed by either a hardware control skill or a comfort adjustment skill), the optimal cockpit skill can be determined based on the session context; when there are dependencies between cockpit skills (e.g., navigating home requires calling the positioning skill first and then the navigation skill), the execution sequence between cockpit skills can be automatically generated.

[0078] Step 104: Assess the risk level of the target cockpit skills and obtain the risk level assessment results.

[0079] In some embodiments, the risk level assessment result refers to the determination of the safety level of the target cockpit skills.

[0080] When the risk level assessment result is Level 1, it means that the execution of the target cockpit skill affects the vehicle system or user privacy, such as modifying navigation map files, changing cockpit configuration parameters, writing system firmware, accessing sensitive data, or conducting network communication. When the risk level assessment result is Level 2, it means that the execution of the target cockpit skill does not involve destructive operations, such as controlling the air conditioning temperature, playing music, displaying information, or answering simple questions.

[0081] In some embodiments, the process of assessing the risk level of a target cockpit skill and obtaining a risk level assessment result may include: assessing the risk level of the target cockpit skill using a preset rule base or machine learning model and obtaining a risk level assessment result.

[0082] In some embodiments, the risk level assessment results may also be correlated with the scene identification results. For example, in a children's scenario, even if a cockpit skill itself is low-risk, the risk level assessment result may be raised due to inappropriate content.

[0083] Step 105: Execute the target cockpit skill according to the target processing path corresponding to the risk level assessment result, obtain the skill execution result and output it.

[0084] In some embodiments, the target cockpit skill is routed to different processing paths based on the risk level assessment results. For example, if the target cockpit skill is a Level 1 skill, the request to invoke the target cockpit skill is routed to the first processing path. For instance, when updating the navigation map, operations such as file download, verification, and writing are completed within the first processing path. After execution, only the anonymized result (e.g., successful map update) is written back to the shared state area. This hierarchical processing path achieves strict isolation of high-risk operations and efficient execution of low-risk operations, balancing security and performance, and ensuring the long-term stable operation of the system.

[0085] In this embodiment, multimodal interaction instructions are first acquired and written into a shared state region to achieve unified management of the global state and avoid inconsistencies caused by decentralized state management. Next, the cockpit agent reads the multimodal interaction instructions and performs intent parsing to accurately identify the user's interaction intent. Then, target cockpit skills are matched based on the intent parsing results, and their risk levels are assessed. Finally, the corresponding target processing path is selected to execute the skill based on the assessment results, ensuring that high-risk skills operate in a safe environment and low-risk skills are executed efficiently, thereby achieving end-to-end security control. In this way, by using the shared state region as a global data foundation, the problems of data silos and state asynchrony between nodes in a unidirectional acyclic graph architecture are effectively solved. At the same time, the risk-based routing mechanism improves system security and stability.

[0086] In some embodiments, the process of executing the target cockpit skill according to the target processing path corresponding to the risk level assessment result in step 105, and obtaining the skill execution result, may include steps 1051-1053, wherein... Step 1051: If the risk level assessment result is Level 1, execute the target cockpit skill through the first processing path corresponding to Level 1 to obtain the skill execution result.

[0087] Here, the first processing path includes at least file permission verification, vehicle environment isolation, tamper interception, execution of target cockpit skills within the sandbox, and desensitization of skill execution results.

[0088] For example, the first level can also be called the high-risk level; the second level can also be called the low-risk level.

[0089] The first processing path is an execution procedure designed for high-risk cockpit skills, designed to ensure that the execution of high-risk cockpit skills does not damage the vehicle's systems.

[0090] File permission verification refers to checking whether the target cockpit skill has permission to access the target file or directory based on preset whitelist rules, and only allowing read and write operations within the authorized scope.

[0091] Vehicle-to-everything (V2X) environment isolation refers to limiting the execution scope of high-risk cockpit skills by building an independent operating environment, preventing these skills from affecting the main operating system or other critical services. This isolation can be achieved based on containerization or virtual machine technology, ensuring that even if the target cockpit skill has vulnerabilities or malicious behavior, it will not spread to the vehicle system.

[0092] The tamper interception mechanism refers to real-time monitoring of the target cockpit skills during execution, detecting any unauthorized attempts to modify system files, configuration parameters, or memory data. Upon detecting abnormal writes or unauthorized operations, the execution of the target cockpit skills will be terminated, and an audit log will be logged.

[0093] The execution of target cockpit skills within a sandbox refers to running target cockpit skills in a secure and isolated sandbox environment. All operations are constrained by the boundaries and rules of the sandbox, thereby minimizing threats to the vehicle system. Sandboxes typically have functions such as resource quota restrictions, network access control, and process isolation to ensure that target cockpit skills can only operate within preset limits.

[0094] The desensitization of the skill execution results is a process of protecting the privacy and filtering the security of the skill execution results. It removes or encrypts sensitive information in the skill execution results, such as user location, personal identification, vehicle VIN code, etc., and retains only the necessary business data for subsequent use.

[0095] In some embodiments, when the cockpit agent recognizes that the user's multimodal interaction command requires the invocation of high-risk cockpit skills (such as modifying vehicle configuration or updating map firmware), the first processing path is triggered.

[0096] Step 1052: If the risk level assessment result is Level 2, execute the target cockpit skill through the second processing path corresponding to Level 2 to obtain the skill execution result.

[0097] In some embodiments, the second processing path refers to the execution flow adopted for low-risk cockpit skills. Low-risk cockpit skills mostly do not involve core system functions or sensitive data operations. For example, low-risk cockpit skills could be playing music, adjusting the air conditioning temperature, or checking the vehicle's remaining battery power.

[0098] It should be noted that the second processing path can also retain basic security checks, such as input parameter validity checks and replay attack detection.

[0099] Step 1053: Write the skill execution result into the shared state area.

[0100] In some embodiments, the process of writing skill execution results into the shared state area may include: synchronously writing skill execution results (such as navigation routes, entertainment content, vehicle status updates, etc.) into the shared state area so that other cockpit skills or nodes can access and utilize this information subsequently. For example, when a user completes navigation settings, the navigation results are written into the shared state area, and then the voice assistant can directly read the navigation results when broadcasting the planned new route without recalculating or querying.

[0101] In this embodiment, for skills at the first level (i.e., high-risk level), multiple protective measures, including file permission verification, vehicle environment isolation, tamper interception, and sandbox execution, are implemented to ensure their operation in a secure environment and prevent damage to the vehicle system. For skills at the second level (i.e., low-risk level), execution is performed directly to improve response efficiency. After execution, the skill execution result is written back to the shared state area to ensure state consistency. This achieves differentiated security control for skills at different risk levels, ensuring the security of high-risk operations while also considering the real-time nature of low-risk operations, effectively enhancing the overall system's security capabilities.

[0102] In some embodiments, before determining the target cockpit skill matching the intent parsing result from the cockpit skill pool based on the intent parsing result in step 103, the method further includes steps 1031-1032, wherein... Step 1031: Perform scene recognition on the multimodal interaction commands and determine the scene recognition result corresponding to the multimodal interaction commands.

[0103] In some embodiments, scene recognition refers to the process of determining the environment or intent of a multimodal interaction command, used to identify the cockpit usage scenario corresponding to the user's current operation. The scene recognition results can guide subsequent system behavior, such as adjusting the interface layout, enabling specific functional modules, or activating security protection mechanisms.

[0104] For example, scene recognition results include children's scenes, movie-watching scenes, parking scenes, stationary scenes, and traffic warning scenes. Children's scenes refer to interactive situations where there are child passengers in the vehicle or when children's scenes are activated, requiring the use of functions such as content filtering and permission restrictions; movie-watching scenes refer to users watching videos or movies while the vehicle is parked, requiring optimization of screen brightness, volume, and playback control logic; parking scenes refer to situations where the vehicle is in the process of automatic parking or assisted parking, requiring priority to ensure the safety of parking functions and provide real-time guidance; stationary scenes refer to situations where the vehicle is stationary and the engine is off, but the cabin still maintains some functions, such as the entertainment system or air conditioning; traffic warning scenes refer to situations where potential dangers are detected while the vehicle is in motion (such as sudden braking by the vehicle in front or lane departure), requiring the triggering of alarms or active intervention.

[0105] Step 1032: Based on the scene recognition results, perform the initialization operations corresponding to the scene recognition results. The scene recognition results include at least children's scenes, movie-watching scenes, parking scenes, vehicle parking scenes, and traffic warning scenes.

[0106] In some embodiments, the process of performing initialization operations corresponding to the scene recognition results may include: performing operations to preload and / or activate the corresponding functional modules, resource files, security policies, and interface configurations based on the scene recognition results.

[0107] For example, when the scene recognition result is a children's scene, the initialization operations include loading a children's exclusive content library, enabling the content filtering engine, setting a permission whitelist, reducing screen brightness, and calling the children's voice assistant; when the scene recognition result is a movie-watching scene, the initialization operations include switching to the cinema mode interface, turning off navigation prompts, adjusting speaker output equalization, and starting the video playback service; when the scene recognition result is a parking scene, the initialization operations include activating the automatic parking algorithm, connecting the reversing camera system, displaying parking guide lines, and disabling unnecessary interactive functions; when the scene recognition result is a parking scene, the initialization operations include entering low-power mode, retaining power to the air conditioning and entertainment systems, and starting the timed shutdown timer; when the scene recognition result is a traffic warning scene, the initialization operations include enabling the ADAS system, increasing the radar / camera data refresh rate, preloading emergency braking logic, and displaying warning icons.

[0108] In practice, once the cockpit AI detects a parking scenario, it immediately activates parking assistance features, simultaneously turning on the rearview camera, millimeter-wave radar, and ultrasonic sensors, and displays a parking trajectory prediction animation on the central control screen. This initialization process can be completed within seconds, ensuring that the cockpit AI responds quickly and operates smoothly when the user executes parking commands.

[0109] In this embodiment, by performing scene recognition on multimodal interaction commands, the current cockpit scene can be dynamically determined, such as a children's scene or a movie-watching scene, and corresponding initialization operations can be performed accordingly, such as enabling content filtering, adjusting permission settings, or configuring specific functional modules. This allows for pre-configuration of system resources and functional modules, reducing the response latency of user commands and improving the interaction efficiency and stability of the cockpit intelligent agent.

[0110] In some embodiments, the method further includes: Step 201: If the skill execution result is found to be abnormal through the cockpit intelligent agent, data comparison and constraint verification are performed based on the operation data of the target node in the shared state area to determine the abnormal problem point.

[0111] Among them, the target node is the node related to the skill execution result.

[0112] In some embodiments, a target node refers to a node that is directly associated with the current skill execution result and participates in target cockpit skill scheduling or data processing. For example, if the target cockpit skill execution involves navigation map updates, then the map update node is the target node.

[0113] In some embodiments, the runtime data of the target node includes the node's input parameters, intermediate calculation results, execution logs, state change records, historical runtime data, preset baseline data, and expected output data.

[0114] Comparing skill execution results with target node operational data can involve comparing multiple aspects, including numerical accuracy, data structure, time series, and logical consistency. For example, in navigation map skill execution, the currently calculated route coordinates can be compared with the recommended route coordinates under similar historical road conditions. During the data comparison process, tolerance ranges and judgment thresholds can be set to distinguish between normal fluctuations and abnormal deviations.

[0115] Constraint verification of skill execution results and target node operation data can include: performing legality, compliance, and security constraint verification on skill execution results and target node operation data based on preset business rules, security policies, or technical specifications. Constraint verification can include data format verification (such as whether the JSON structure is correct), permission verification (such as whether the user has permission to modify configuration), content security verification (such as whether the content security verification contains prohibited words), and performance boundary verification (such as whether the response time exceeds the threshold), etc.

[0116] Data comparison and constraint verification are performed based on the operational data of target nodes within the shared state area. Specific problem locations or processes that do not meet expectations or violate rules are identified as anomalous problem points. For example, in navigation skills, an anomalous problem point could be an error in the calculation of destination coordinates.

[0117] This effectively avoids the waste of system resources and the decline in user experience caused by blind retries or extensive troubleshooting, thereby improving the efficiency of fault diagnosis and the system's self-healing ability, and thus enhancing the robustness and reliability of the cockpit intelligent agent.

[0118] Step 202: Correct the abnormal issues, obtain the corrected skill execution results, and output them.

[0119] In some embodiments, the process of correcting abnormal issues to obtain corrected skill execution results may include: performing data repair, parameter reset, process rollback, module restart, strategy adjustment, etc., on the abnormal issues to obtain corrected skill execution results.

[0120] For example, when the cockpit agent determines that the traffic congestion prediction data is abnormal, it calls the backup traffic data source to obtain the latest road conditions, then recalculates the optimal route and updates the navigation status information in the shared status area; finally, it redisplays the corrected navigation route on the central control screen.

[0121] In this embodiment, after the cockpit agent identifies an anomaly in the skill execution result, it performs data comparison and constraint verification based on the operational data of the target node within the shared state area to determine the anomaly point, corrects the anomaly point, and obtains and outputs the corrected skill execution result. This enables precise location and efficient repair of skill execution anomalies, thereby improving the system's self-healing capability and stability, and ultimately enhancing the overall user experience and safety of the intelligent cockpit.

[0122] In some embodiments, the method further includes: Step 311: If the stored data in the shared state region is greater than or equal to a preset threshold, perform state trimming on the stored data in the shared state region according to preset trimming parameters to obtain trimmed data.

[0123] State pruning includes at least session context cleanup, temporary variable deletion, historical log archiving, and target data retention. The target data includes at least user preference data and vehicle status data.

[0124] As can be seen from the foregoing, the stored data in the shared state area ensures the consistency and synchronization of the global state, avoiding the problem of data silos.

[0125] In some embodiments, a preset threshold is used to determine whether the amount of data stored in the shared state area exceeds a critical value. This threshold can be determined by a percentage of memory usage, the number of data entries, or the number of accumulated state entries. For example, state pruning is triggered when the system memory usage reaches 80% or the number of data entries in the shared state area exceeds 100,000. The preset threshold can be dynamically configured based on factors such as vehicle performance, cabin load, and user interaction frequency to balance system stability and resource utilization.

[0126] The preset pruning parameters are a set of specific control strategies for the state pruning process, including the scope of cleanup, retention priority, archiving period, and deletion granularity. For example, preset pruning parameters include rules such as cleaning up session contexts that have not been used for more than 24 hours, immediately deleting temporary variables, archiving historical logs monthly, and retaining user preference data and vehicle status data.

[0127] In some embodiments, the process of performing state trimming on the stored data in the shared state region according to preset trimming parameters to obtain trimmed data may include: performing session context cleanup, temporary variable deletion, historical log archiving, and target data retention on the stored data in the shared state region according to preset trimming parameters to obtain trimmed data.

[0128] Among them, conversation context cleanup refers to removing inactive or expired conversation records and related context information, such as the user's intent in the previous round of conversation, intermediate inference results, temporary parameters, etc.

[0129] Temporary variable deletion refers to clearing temporary data generated during cockpit skill execution, such as intermediate calculation results, cached objects, and debug logs.

[0130] Historical log archiving refers to compressing and storing completed interaction records, security verification logs, skill execution logs, etc., according to time or event type, moving them out of the shared state area, but still retaining them in the system backup or dedicated log library for subsequent auditing and troubleshooting.

[0131] Target data retention refers to ensuring that system data is not deleted during the state pruning process, including user preference data and vehicle status data. User preference data may include the user's frequently used settings, personalized recommendations, theme styles, voice assistant habits, etc.; vehicle status data may include real-time hardware information such as vehicle speed, fuel level, battery charge, air conditioning temperature, and light status.

[0132] In this embodiment, state pruning is triggered when the accumulated data in the shared state area reaches a preset threshold. This effectively prevents memory explosion caused by long session interactions, ensures the stability and response speed of the system under long-term operation, and meets the high stability requirements of automotive scenarios.

[0133] In some embodiments, the method further includes: Step 411: Generate preset trimming parameters based on the user's session preference data.

[0134] In some embodiments, user session preference data refers to data generated during the user's interaction with the cockpit agent.

[0135] User conversation preference data can be determined through data such as user's historical interaction records, command frequency, frequently used function modules, response time, and interaction time distribution. For example, this user uses the navigation function module between 7-9 am and plays music between 8-10 pm, preferring voice interaction to touch operation.

[0136] Once the user's session preference data is obtained, the preset trimming parameters can be dynamically adjusted based on the user's session preference data to match the user's actual usage habits.

[0137] In some embodiments, the process of generating preset trimming parameters based on user session preference data may include: performing real-time analysis of user session preference data using a machine learning model or rule engine to generate preset trimming parameters.

[0138] In this embodiment, personalized preset trimming parameters are dynamically generated by analyzing user session preference data. This makes the trimming process more aligned with actual user habits, avoiding excessive cleanup that could negatively impact user experience, while ensuring efficient use of system resources. This design enhances the intelligence level of state management, achieving a balance between personalization and system stability.

[0139] In some embodiments, prior to performing intent parsing on the multimodal interaction instructions in step 102, the method further includes: Step 1021: The cockpit intelligent agent performs security verification on the multimodal interaction commands to determine the first security verification result; the security verification includes at least voice desensitization verification, target keyword interception, and target information recognition.

[0140] In some embodiments, the process of performing security verification on multimodal interaction commands through a cockpit agent and determining a first security verification result may include: performing voice desensitization verification, target keyword interception, and target information recognition on the multimodal interaction commands through a cockpit agent to obtain a first security verification result.

[0141] Among them, voice desensitization verification refers to filtering privacy data on the text content after voice-to-text conversion, such as automatically identifying and blocking personal information such as ID card number, mobile phone number, and bank card number; target keyword interception is based on a preset security rule base to block keywords of target instructions (such as deleting system files or restarting the vehicle system) or illegal content (such as attacking others or spreading rumors); target information recognition is used to determine whether the multimodal interaction instructions input by the user involve sensitive information, such as copyright information or inappropriate information.

[0142] By introducing a security verification mechanism before processing multimodal interaction commands, potential malicious or high-risk commands can be effectively identified and blocked, ensuring the operational safety of the vehicle system and preventing system crashes or data leaks caused by the abuse of malicious or high-risk commands.

[0143] Step 1022: If the first security verification result indicates that the verification has passed, update the security verification identifier corresponding to the multimodal interaction command in the shared state area.

[0144] In some embodiments, the security verification identifier is a status marker stored in a shared state area, used to characterize whether the current multimodal interaction instruction has passed the security verification. The security verification identifier may exist in the form of a Boolean value (true / false), an enumeration type (PASS / FAIL), or a structured field (such as {status: "passed", timestamp: "2024-06-01T10:00:00Z"}).

[0145] In some scenarios, after a multimodal interaction command passes voice desensitization verification, target keyword interception, and target information recognition, and the verification is confirmed to be successful, the cockpit agent will write a security verification identifier to the shared state area, such as {"instruction_id": "uuid-123", "security_check_status": "passed", "timestamp": "2024-06-01T10:00:01Z"}. By dynamically updating the security verification identifier in the shared state area, all nodes can achieve unified awareness of the security status of multimodal interaction commands, avoiding duplicate verifications and logical conflicts, and improving the overall execution efficiency and stability of the system.

[0146] Step 1023: If the first security verification result indicates that the verification fails, output the reply result according to the first protection template, and form the first record data based on the multimodal interaction command.

[0147] In some scenarios, when a multimodal interaction command fails the verification process by passing voice desensitization verification, target keyword interception, and target information recognition, the first protection template is fed back to the user, and the first record data is formed based on the multimodal interaction command.

[0148] The first protection template is the prompt message returned to the user. For example, for multimodal interaction commands containing sensitive information, the first protection template is: Your message contains private information and has been automatically anonymized. For multimodal interaction commands that exceed permissions, the first protection template is: This operation exceeds the scope of permissions and cannot be executed.

[0149] The first set of recorded data consists of log entries generated by the vehicle system, which include at least the multimodal interaction commands (after being anonymized), the reason for verification failure, the trigger time, the device ID, and the user's identity.

[0150] In this embodiment, security verification is used to anonymize voice commands, intercept keywords, and identify sensitive information, effectively preventing malicious input. Upon successful verification, the security identifier is updated to ensure the security of subsequent processes. If verification fails, the system immediately intercepts the input and outputs a protective template response, while simultaneously recording the data. This design constructs the first line of defense for end-to-end security, significantly enhancing the system's resistance to attacks.

[0151] In some embodiments, after obtaining the skill execution result in step 105, the method further includes: Step 106: Perform a second security check on the skill execution result and determine the second security check result.

[0152] Here, secondary security verification includes at least content screening and driving safety verification.

[0153] In some embodiments, the process of performing a second security check on the skill execution result and determining the second security check result may include: performing content screening and driving safety verification on the skill execution result to determine the second security check result.

[0154] After the skill execution result is generated, a second security check is performed on the content and operational behavior within that result to ensure that it does not pose a potential threat to the vehicle system or the user. This second security check is a post-processing step in the security protection system, supplementing and strengthening the initial security screening and sandbox isolation protection.

[0155] Here, content screening refers to identifying whether the multimedia content, such as text, images, and audio, in the results of this skill's execution contains illegal information, sensitive words, inappropriate language, or content that violates rules regarding child-friendly content. For example, if the results of this skill's execution include illegal content such as violence or pornography, a blocking mechanism will be triggered.

[0156] Driving safety verification refers to whether the execution result of this skill involves operations that affect driving safety, such as triggering high-risk behaviors such as changing navigation routes, opening windows, or suddenly increasing volume while the vehicle is in motion.

[0157] In addition, if the scene recognition result is a children's scene, it is also necessary to adapt the skill execution result to children's content so as to output the skill execution result corresponding to the children's scene.

[0158] Step 107: If the second security check result indicates that the check has passed, output the skill execution result.

[0159] When the second safety verification result indicates that the verification has passed, it means that no violations were found in the skill execution result and the driving safety verification confirmed that there were no high-risk operations. At this time, the skill execution result can be fed back to the user.

[0160] Step 108: If the second security verification result indicates that the verification fails, output the response result according to the second protection template, and form the second record data based on the skill execution result.

[0161] If the second safety verification result indicates that the verification failed, it means that a violation was found in the skill execution result, or the driving safety verification confirmed that there was a high-risk operation.

[0162] In this case, a response result can be output according to the second protection template, and second record data can be generated based on the skill execution result. The second protection template is a preset security response script used to provide warning information to the user when the skill execution result verification fails.

[0163] For example, the content of the second protection template could be: "The request involves sensitive content, which has been blocked for you," or "The current operation may affect driving safety; please try again later." Additionally, the second protection template can be displayed in conjunction with a target icon or target animation to ensure that the user clearly understands the reason for the rejection.

[0164] Subsequently, the entire execution process of the target cockpit skills and the data of safety verification failures can be archived to obtain second record data. For example, the second record data may include multimodal interaction instructions, processing paths, verification failure reasons, timestamps, device IDs, and other data.

[0165] In this embodiment, a secondary security verification step is added after skill execution to screen the execution result for content compliance and driving safety, preventing the output of illegal or dangerous content. If the verification passes, the content is output normally; if it fails, a protective template response is triggered and recorded. This design achieves closed-loop security control over skill execution results, further ensuring the security and compliance of the user interaction process.

[0166] In some embodiments, the method further includes: Step 501: If no feedback instruction for the skill execution result is received within the first duration, a sleep mode is triggered, and a cleanup operation is performed through the reducer.

[0167] In some embodiments, the first duration is a preset time threshold. If no user interaction feedback (such as voice confirmation, touch operation, hardware button response, etc.) is detected within this time threshold, it is determined that the current interaction process has ended or the user has no further intention to operate. The first duration can be dynamically configured according to different cockpit scenarios. For example, it can be set to 30 to 60 seconds in a normal driving scenario, and shortened to less than 10 seconds in a child scenario or a high-safety-level scenario to improve system response sensitivity and safety.

[0168] In some embodiments, feedback instructions refer to the interactive behavior of a user after the skill execution result is output, used to indicate acceptance, follow-up questioning, correction, or rejection of the skill execution result. Forms of feedback instructions include, but are not limited to, voice commands, touchscreen clicks, steering wheel button inputs, and auxiliary feedback automatically triggered by the in-vehicle system (such as gesture recognition and eye tracking).

[0169] Sleep mode is a low-power operating state that is actively entered to reduce system power consumption and free up resources. In this mode, non-essential functions (such as the central control screen remaining on, background music playback, and some sensor monitoring) will be turned off or paused, while only wake-up mechanisms (such as voice wake-up and emergency call monitoring) and safety monitoring modules (such as collision warning and fatigue monitoring) will continue to run. Sleep mode not only effectively extends the vehicle's battery life but also avoids system lag and memory leaks caused by too many background processes.

[0170] In some scenarios, if no feedback instruction regarding the skill execution result is received within the first period, a sleep mode is triggered, and a deep cleanup operation is performed through the reducer to ensure that the system memory usage is stably controlled within a preset threshold without affecting the user experience, thus fundamentally solving the memory explosion problem.

[0171] In this embodiment, a first duration window is set after the skill execution result is output. If no user feedback instruction is received, a sleep mode is triggered, and a cleanup operation is performed by the reducer. This enables intelligent power consumption management and dynamic reclamation of memory resources in the cockpit system, avoiding cockpit system lag and memory bloat caused by prolonged inactivity, and ensuring high stability and low power consumption characteristics of the vehicle system during long-term operation.

[0172] In some embodiments, a skill scheduling method is provided, such as Figure 2 As shown, the method of skill scheduling includes the following.

[0173] S11, acquire the multimodal interaction command and write the multimodal interaction command into the shared state area.

[0174] The intelligent cockpit interaction interface can obtain multimodal interaction commands, which can include voice wake-up commands, central control touch operation, steering wheel hardware buttons, and automatic triggering by the vehicle system.

[0175] After obtaining the multimodal interaction instructions, the multimodal interaction instructions can be written into the shared state area. The shared state area can include: user interaction instructions, session context memory, security injection verification identifier, Skills call parameters, sandbox operation audit log, content compliance mark, real-time status of vehicle hardware, cockpit configuration / ground / firmware status, child scene switch configuration, active sleep timeout / wake-up status, sensitive information protection mark, and status accumulation / memory usage mark.

[0176] Multimodal interaction commands are input for preprocessing and security verification via nodes. Security verification includes voice desensitization, prompt injection interception, and sensitive information identification. If verification passes, step S12 is executed. If verification fails, a violation / sensitive request is injected, triggering security interception, a sensitive template response pop-up in the central control window, voice alarms, and log archiving. Finally, full-process security log archiving is performed to achieve synchronous and auditable in-vehicle backend.

[0177] S12, Intent parsing and scene recognition.

[0178] After performing intent parsing and scene recognition at the node, if the current scene is a children's scene, then children's scene instructions are matched, children's scene initialization is executed, and content filtering and permission restrictions are completed. If the current scene is a regular scene, then ordinary cockpit instructions are matched, and the instructions are distributed according to the regular salesperson's intent.

[0179] After determining the intent recognition result, the target cockpit skill corresponding to the intent parsing result is determined from the cockpit skill pool according to the intent recognition result. The cockpit skill pool includes: cockpit hardware control skills, navigation map update skills, multimedia playback skills, vehicle status diagnosis skills, cockpit configuration skills, interactive question and answer skills, children's scenario skills, and sleep mode skills.

[0180] S13, Skill Risk Classification Routing.

[0181] When the target cockpit skill is a high-risk skill (e.g., performing map / configuration / file modification operations), the skill is executed through the process in the sandbox isolation and protection subgraph. When the target cockpit skill is a low-risk skill (e.g., performing control / entertainment / question-answering operations), the skill is executed through node execution LLM inference and cockpit hardware linkage. The sandbox isolation and protection subgraph includes file permission whitelist verification, vehicle OS / ECU environment isolation, real-time monitoring and interception of tampering behavior, secure execution of operations within the sandbox, and anonymized write-back of execution results.

[0182] S14, Post-processing content filtering.

[0183] Post-processing content filtering is performed through nodes, specifically including sensitivity screening, child-friendly content adaptation, and driving safety verification.

[0184] After post-processing content filtering, if the content is confirmed to be illegal or sensitive, operations such as content blocking, calling protection templates, voice prompts, and audit logs are performed.

[0185] S15, cockpit multimodal output.

[0186] After post-processing content filtering and confirming compliance, cockpit multimodal output is executed. Cockpit multimodal output can include central control display, voice broadcast, and hardware execution.

[0187] S16, whether the cumulative state exceeds the threshold.

[0188] The system monitors in real time whether the accumulated state in the shared state area exceeds a threshold. If the threshold is exceeded, state reducer pruning is performed. State reducer pruning includes cleaning up expired session contexts, deleting temporary variables / intermediate results, archiving historical logs to local storage, and retaining user preference data / target state.

[0189] If the threshold is not exceeded, an interaction timeout determination is performed.

[0190] S17, Interaction timeout determination.

[0191] If the interaction does not time out, the interaction loop closes, awaiting the next instruction. If the interaction times out, a sleep mode is actively triggered to reduce power consumption, continuously monitor the system in standby mode, and force a deep cleanup by the Reducer. Upon receiving a wake-up instruction, the wake-up instruction can be updated in the shared state area.

[0192] The skill scheduling method provided in this disclosure will be described in detail below through an embodiment, taking into account a specific application scenario.

[0193] As intelligent cockpit technology evolves into a "third living space," in-vehicle systems are upgrading from a collection of single functions to an intelligent cockpit agent with autonomous perception, decision-making, and interaction capabilities. The intelligent cockpit agent has become the core brain of the next generation of intelligent cockpits.

[0194] Data released by the Gaogong Intelligent Vehicle Research Institute shows that by 2025, the pre-installation rate of intelligent cockpits in Chinese passenger vehicles reached 76.62%, and the delivery volume of cockpit domain controllers exceeded 9.27 million vehicles, of which models equipped with basic cockpit intelligent agent functions accounted for more than 42%. At the same time, users have put forward higher requirements for the naturalness of cockpit intelligent agent interaction, long-term operational stability and security. The defects of the existing cockpit intelligent agent underlying architecture based on unidirectional acyclic graphs are becoming increasingly prominent, and have become the core bottleneck restricting the large-scale mass production and deployment of such systems.

[0195] In terms of architecture design, the unidirectional acyclic graph (AGraph) agent architecture only supports unidirectional sequential flow between nodes and lacks complex scheduling capabilities such as iterative loops, dynamic conditional routing, and state jumps. As the number of cockpit skills increases from dozens to hundreds, and complex cockpit scenarios such as children's scenarios, active sleep, and multi-turn continuous interactions are added, the limitations of linear orchestration of the unidirectional acyclic graph are drastically amplified. This architecture can only achieve simple serial execution of instructions and cannot adapt to the multi-scenario, multi-branch, and cyclical interaction logic of the cockpit. At the same time, the unidirectional acyclic graph agent architecture generally lacks a standardized skill pool decoupling design, and the functional nodes are highly coupled. When adding or modifying a cockpit skill, the entire link node orchestration needs to be adjusted, resulting in an exponential increase in maintenance costs and severely restricting the rapid iteration and over-the-air upgrade of cockpit agent functions.

[0196] In terms of state management, cockpit agents based on a unidirectional acyclic graph (AGraph) agent architecture suffer from two major flaws that directly impact the long-term operational stability of the system: First, decentralized state management leads to inconsistencies in the global state. Unidirectional AGraph agent architectures generally lack a unified global state management paradigm. Existing cockpit agents typically have each skill node independently maintain its own operational state and interaction context, resulting in numerous data silos between nodes. State synchronization between different nodes requires complex interface calls, a mechanism that is highly susceptible to problems such as state desynchronization and data loss.

[0197] Industry statistics show that some users complain that cabin settings revert to default values ​​after the vehicle is turned off and restarted, and the failure rate of synchronizing personalized settings for the same account across different vehicles is as high as 17%, which seriously undermines the consistency of the user experience.

[0198] Furthermore, this decentralized state management approach prevents the cockpit agent from achieving true long-term memory capabilities. Users must repeatedly issue the same commands every time they get in the car, resulting in extremely high interaction friction costs. Second, the unidirectional acyclic graph agent architecture lacks a built-in state constraint mechanism, making it highly susceptible to state accumulation leading to memory explosions. This architecture itself lacks memory management mechanisms such as state pruning and reducer reduction. During long-duration sessions, session context, intermediate inference results, node execution logs, and temporary parameters accumulate without limit. In continuous interaction scenarios such as long-distance driving, the cockpit agent's memory usage can surge to 300% to 500%, directly causing vehicle system lag, interaction delays, and sluggish voice response. In severe cases, it can even cause process crashes and vehicle system restarts. Current solutions only passively release memory by forcibly restarting the system or clearing the session cache. These operations directly interrupt user interaction and lose critical context information, failing to meet the stringent requirements of high stability and uninterrupted operation in in-vehicle scenarios.

[0199] In terms of security, cockpit agents based on a unidirectional acyclic graph (AGraph) agent architecture have weak security protection capabilities. The native security mechanisms of the unidirectional AGraph agent architecture are relatively basic and cannot forcibly embed security nodes such as Prompt injection prevention, sensitive information filtering, file tampering protection, and sandbox isolation into the entire process. Existing cockpit agents often treat security verification as an external module, which can be bypassed by commands. Malicious users can tamper with core files such as cockpit configuration and map firmware through prompt injection and unauthorized calls, threatening vehicle driving safety. At the same time, high-risk file operations are not sandboxed, and abnormal operations can directly damage the underlying environment of the vehicle system.

[0200] In terms of scenario adaptation, the unidirectional acyclic graph agent architecture has a unidirectional serial characteristic, which makes it difficult to be compatible with some complex cockpit-specific scenarios. In addition, the use of a large number of hard-coded branches to achieve scenario switching not only increases system redundancy but also makes it difficult to achieve flexible scenario configuration and adaptive scheduling, resulting in poor overall adaptability and scalability.

[0201] Therefore, there is an urgent need for an intelligent cockpit skill scheduling method that takes a cockpit agent as its core, has globally unified state management, a built-in state reducer pruning mechanism, modular skill scheduling, full-link security protection, and multi-scenario adaptive capabilities. This method would completely solve the inherent defects of the unidirectional acyclic graph agent architecture and the memory explosion problem caused by state accumulation, while improving the system's scalability, maintainability, and security, and providing users with a stable, smooth, and secure intelligent cockpit interaction experience.

[0202] To overcome the aforementioned shortcomings of existing technologies, this paper provides a method for skill invocation, enabling unified parsing, intelligent routing, and end-to-end secure execution of multimodal interaction commands in intelligent cockpits. This method uses a shared state region as the global data foundation, effectively alleviating core pain points such as the linear orchestration limitations of unidirectional acyclic graph agent architectures, global state inconsistencies, long-session memory explosions, and weak security protection. While ensuring high system stability and security, it also possesses strong functional scalability and engineering implementation capabilities. This method uses the cockpit agent as a unified intelligent decision-making center. Through globally unified state management, modular skill pool scheduling, a built-in state reducer pruning mechanism, and end-to-end security protection, it achieves complex interactive scheduling for multiple scenarios, multiple branches, and cyclical interactions in intelligent cockpits. The overall architecture of this method consists of four parts: a multimodal interaction entry layer, a cockpit agent, a skill execution layer, and an output feedback layer.

[0203] The multimodal interaction entry layer is primarily responsible for the collection, cleaning, and standardized preprocessing of various cockpit interaction commands, providing input data in a unified format for all subsequent intelligent decision-making processes. It serves as the front-end entry point for the entire methodology. This layer includes the following functional modules: Multi-source command acquisition module: used to synchronously acquire multimodal interaction commands in the cockpit, including voice wake-up commands, central control touch operation commands, steering wheel hardware button commands, and vehicle system automatic trigger commands, covering all possible user interaction methods.

[0204] Command preprocessing module: Standardizes the collected multimodal interaction commands, including voice-to-text conversion and privacy information desensitization of voice commands, coordinate transformation and intent mapping of touch commands, and encoding and parsing of hardware button commands, converting all types of commands into a structured format that can be recognized by the cockpit intelligent agent.

[0205] Pre-security screening module: Performs preliminary security checks on the pre-processed raw instructions, identifies obvious malicious instruction formats and illegal characters, and provides pre-filtering for subsequent in-depth security verification.

[0206] As the "brain" of the entire system, the cockpit intelligent agent is mainly responsible for the unified management of the global state, the accurate parsing of user interaction intentions, the intelligent recognition of scenarios, and the generation of scheduling decisions. It is the core link in realizing the complex interaction scheduling of the intelligent cockpit. The cockpit intelligent agent includes the following functional units: The shared state area serves as a globally unique data foundation, uniformly managing all system states, including user interaction commands, session context memory, security injection verification identifiers, Skills call parameters, sandbox operation audit logs, content compliance markers, real-time status of in-vehicle hardware, cockpit configuration / map / firmware status, child scene switch configuration, active sleep timeout / wake-up status, sensitive information protection markers, and status accumulation / memory usage markers, achieving real-time data sharing and consistency among all nodes.

[0207] The intent parsing and scene recognition unit performs deep semantic parsing of the multimodal interaction commands input by the user based on global state data, identifies the user's core interaction intent, and automatically determines the specific scene in the cockpit based on the current system state, including children's scenes, sleeping scenes, and regular interaction scenes, providing clear decision-making basis for the subsequent skill routing module.

[0208] State Pruning Unit: This unit is the core memory management module in the system. It integrates a dual-trigger state cleanup mechanism, continuously monitors the total amount of global state data and the memory usage during system operation, automatically cleans up expired state data, deletes temporary variables, archives historical logs, and retains target state data. Through architectural design, it effectively alleviates the memory bloat problem in long session scenarios.

[0209] Global Scheduling and Routing Unit: Based on the intent parsing results and scene recognition results, it generates standardized skill call parameters, accurately routes scheduling requests to the corresponding cockpit skills, and is responsible for the node flow and exception handling of the entire process.

[0210] The skill execution layer is responsible for translating the scheduling decisions generated by the cockpit agent into specific skill execution actions, and ensuring the legality and security of all operations through a full-link security protection mechanism. This layer includes the following functions: Cockpit Skill Pool: Adopting a modular design, it contains independent skill units related to all cockpit interactions, including cockpit hardware control skills, navigation map update skills, multimedia playback skills, vehicle status diagnosis skills, cockpit configuration skills, interactive Q&A skills, children's scenario skills, and sleep mode skills. All skills can be independently developed, tested, deployed, and upgraded without modifying the main scheduling process. Skill Risk Classification and Routing Unit: Assess the risk level of all skills to be executed, classify skills into high-risk skills (involving map, configuration, and file modification) and low-risk skills (involving control, entertainment, and Q&A), and route them to different execution paths respectively; Sandbox Isolation Protection Sub-graph: Specifically designed for the secure execution of high-risk skills. Through a triple protection mechanism of file permission whitelist verification, vehicle operating system / electronic control unit environment isolation, and real-time monitoring and interception of tampering behavior, it ensures that high-risk operations will not damage the core vehicle system. After execution, only the de-identified results are written back to the global state.

[0211] The output feedback layer is responsible for transforming the skill execution results into multimodal feedback that the user can perceive, completing the closed loop of the entire interaction process, and simultaneously implementing system power consumption management and log auditing. This layer includes the following functions: Multimodal output control: Based on the skill execution results, the system provides feedback on operation information to the user through multimodal methods such as central control screen display, voice broadcast, and hardware status synchronization, ensuring that the user can obtain the execution status in a timely manner; The active sleep control unit monitors the user interaction status in real time. When it detects that there has been no interaction for a timeout, it automatically triggers the active sleep mode, turns off unnecessary cockpit functions, reduces system power consumption, and continues to listen for wake-up commands while in sleep mode.

[0212] Full-process log archiving unit: All interaction processes, skill execution records, security verification results, and abnormal behaviors are uniformly archived and synchronized to the vehicle back-end management terminal to achieve traceability and auditability of all operations.

[0213] Next, the specific implementation method of this solution will be described in detail. The implementation steps of this solution are as follows: Step 201: After the system is powered on, initialize the in-cabin voice acquisition device, touch screen, hardware buttons and other interactive devices, start the vehicle communication middleware, and establish a real-time communication link between the multimodal interaction entry layer, the cockpit intelligent agent, the skill execution layer and the output feedback layer; at the same time, load the cockpit skill pool, the security verification rule base, user personalization settings and preset trimming parameters.

[0214] Step 202: Synchronously collect various multimodal interaction commands in the cockpit through the multimodal interaction entry layer, complete the standardized preprocessing and preliminary safety screening of the commands, and write the processed structured commands into the shared state area.

[0215] Step 203: The cockpit agent reads multimodal interaction commands from the shared state area, performs deep pre-security verification, including voice desensitization secondary verification, Prompt injection interception, and sensitive information identification; if the verification fails, it triggers security interception and sensitive template reply, records security logs, and terminates the interaction; if the verification passes, it updates the security verification flag in the shared state area.

[0216] Step 204: The cockpit agent performs intent parsing and scene recognition on the verified commands to determine whether they match a children's scene; if they match, it executes the children's scene initialization process and enables content filtering and permission restrictions; if they are ordinary cockpit commands, it executes the regular business intent distribution.

[0217] Step 205: Based on the intent parsing results, match the corresponding target cockpit skill from the cockpit skill pool, generate standardized skill call parameters and write them into the shared state area; at the same time, conduct a risk level assessment of the target cockpit skill and determine the execution path.

[0218] Step 206: If the skill is determined to be high-risk, the request will be routed to the sandbox isolation protection subgraph, and file permission whitelist verification, vehicle OS / ECU environment isolation, and real-time monitoring and interception of tampering behavior will be performed in sequence. The skill operation will be executed safely in the sandbox environment. After the execution is completed, the de-identified result will be written back to the shared state area. If the skill is determined to be low-risk, the process will directly enter the large-scale language model reasoning and cockpit hardware linkage process.

[0219] Step 207: Perform post-processing content security filtering, screen for sensitive content, adapt to children's content, and verify driving safety in the skill execution results; if the content violates the rules, trigger content blocking and protection template response, record audit logs, and terminate this interaction; if the content is compliant, generate multimodal output instructions.

[0220] Step 208: Through the multimodal output control unit, the execution result is fed back to the user in the form of central control screen display, voice broadcast, and hardware execution synchronization, thus completing the basic closed loop of this interaction.

[0221] Step 209: Trigger state accumulation detection to determine whether the global state accumulation exceeds the preset threshold; if it exceeds, perform state pruning, clean up expired session contexts, delete temporary variables, archive historical logs, retain only user preference data and system state, and write the pruned state back to the shared state area; if it does not exceed, proceed directly to the next step.

[0222] Step 210: Determine if there is a timeout due to no interaction; if a timeout occurs, trigger active sleep mode, disable unnecessary cockpit functions, reduce system power consumption, and force deep cleaning and pruning, while continuously listening for wake-up commands; if no timeout occurs, the system returns to real-time listening state and waits for the next interaction command.

[0223] Step 211: Synchronize and archive the entire process data of this interaction to the security log, and upload it to the vehicle backend management terminal to complete the final closed loop of the entire interaction process.

[0224] This disclosure employs a shared state region as a globally unique data foundation, completely resolving the global state inconsistency problem caused by decentralized state management in unidirectional acyclic graph intelligent agent architectures. This data foundation enables real-time data sharing and strong synchronization among all nodes, effectively eliminating data silos and significantly improving system stability and reliability. A built-in dual-trigger state reducer pruning mechanism combines automatic lightweight cleanup after interaction with forced deep cleanup during active sleep, achieving dynamic control of the global state. Without affecting normal user interaction, it stably controls system memory usage within a preset threshold during long sessions, completely resolving the memory explosion problem caused by state accumulation at the architectural level, meeting the stringent requirements of high stability and uninterrupted operation in automotive scenarios. A modular cockpit skill pool is constructed, with all skill units independently decoupled. Adding or modifying cockpit skills only requires registering the corresponding module in the skill pool, without adjusting the main scheduling process, significantly reducing system maintenance costs. It supports rapid iteration and OTA upgrades of cockpit functions, adapting to the increasingly rich functional requirements of intelligent cockpits. A full-link security protection system has been established, which forcibly embeds pre-security verification, sandbox isolation protection, and post-content filtering into the main scheduling process. This enables security control over the entire skill execution process, effectively preventing security risks such as prompt injection, unauthorized calls, and file tampering, and ensuring the operational safety of the vehicle system.

[0225] This public release natively supports multi-scenario adaptive scheduling. Through the cyclic iteration of the state graph, dynamic conditional routing, and state jump capability, it can perfectly adapt to complex cockpit-specific scenarios such as permission control for children's scenarios and active sleep wake-up loop monitoring. This avoids the use of a large number of hard-coded branch judgments, thereby improving the scenario adaptability and scalability of the entire scheduling system.

[0226] This disclosure achieves full-process observability and auditability. All node execution processes, skill call records, and security verification results are fully archived, facilitating online problem investigation, behavior auditing, and security tracing, and meeting the log auditing requirements of vehicle systems.

[0227] This disclosure has a clear overall architecture, well-defined responsibilities at each level, and collaborative work, which ensures both the system's simplicity and efficiency, while also achieving intelligent, safe, and scalable cockpit intelligent agent skill scheduling.

[0228] Secondly, embodiments of this disclosure provide another skill scheduling apparatus, referring to... Figure 3 As shown, the skill scheduling device may include an acquisition module 301 and a processing module 302, wherein; The acquisition module 301 is used to acquire multimodal interaction instructions and write the multimodal interaction instructions into the shared state area; The processing module 302 is used to read multimodal interaction commands from the shared state area through the cockpit agent, perform intent parsing on the multimodal interaction commands, and determine the intent parsing result corresponding to the multimodal interaction commands; based on the intent parsing result, determine the target cockpit skill corresponding to the intent parsing result from the cockpit skill pool; assess the risk level of the target cockpit skill and obtain the risk level assessment result; and execute the target cockpit skill according to the target processing path corresponding to the risk level assessment result, obtain the skill execution result, and output it.

[0229] In some embodiments, the processing module 302 is further configured to execute the target cockpit skill through the first processing path corresponding to the first level when the risk level assessment result is the first level, and obtain the skill execution result; the first processing path includes at least file permission verification, vehicle environment isolation, tamper interception, execution of the target cockpit skill in the sandbox, and desensitization processing of the skill execution result; If the risk level assessment result is Level 2, the target cockpit skill is executed through the second processing path corresponding to Level 2 to obtain the skill execution result; Write the skill execution result to the shared state area.

[0230] In some embodiments, the processing module 302 is further configured to perform scene recognition on the multimodal interaction command and determine the scene recognition result corresponding to the multimodal interaction command; Based on the scene recognition results, execute the initialization operations corresponding to the scene recognition results. The scene recognition results include at least children's scenes, movie-watching scenes, parking scenes, vehicle parking scenes, and traffic warning scenes.

[0231] In some embodiments, the processing module 302 is further configured to, when the skill execution result is identified as abnormal by the cockpit agent, perform data comparison and constraint verification based on the running data of the target node in the shared state area to determine the abnormal problem point, wherein the target node is a node related to the skill execution result; Correct any abnormal issues, obtain the corrected skill execution results, and output them.

[0232] In some embodiments, the processing module 302 is further configured to perform state trimming on the stored data in the shared state region according to preset trimming parameters when the stored data in the shared state region is greater than or equal to a preset threshold, so as to obtain trimmed data. State pruning includes at least session context cleanup, temporary variable deletion, historical log archiving, and target data retention. The target data includes at least user preference data and vehicle status data.

[0233] In some embodiments, the processing module 302 is further configured to generate preset trimming parameters based on the user's session preference data.

[0234] In some embodiments, the processing module 302 is further configured to perform security verification on the multimodal interaction commands through the cockpit intelligent agent and determine a first security verification result; the security verification includes at least voice desensitization verification, target keyword interception and target information recognition; If the first security check result indicates that the check has passed, update the security check identifier corresponding to the multimodal interaction command in the shared state area; If the first security verification result indicates that the verification fails, the response result is output according to the first protection template, and the first record data is formed based on the multimodal interaction instructions.

[0235] In some embodiments, the processing module 302 is further configured to perform a second security check on the skill execution result and determine a second security check result; the second security check includes at least content screening and driving safety check. If the second security check result indicates that the check has passed, the skill execution result will be output. If the second security verification result indicates that the verification fails, the response result is output according to the second protection template, and the second record data is formed based on the skill execution result.

[0236] In some embodiments, the cockpit skill pool includes at least cockpit hardware control skills, navigation map update skills, multimedia playback skills, vehicle status diagnosis skills, cockpit configuration skills, interactive question and answer skills, children's scenario skills, and sleep mode skills.

[0237] In some embodiments, the processing module 302 is further configured to trigger a sleep mode and perform a cleanup operation via a reducer if no feedback instruction for the skill execution result is received within a first duration.

[0238] It should be noted that the skill scheduling device provided in this embodiment includes all the units included, which can be implemented by a processor in an electronic device; of course, it can also be implemented by specific logic circuits; in the implementation process, the processor can be a central processing unit (CPU), a microprocessor unit (MPU), a digital signal processor (DSP), or a field-programmable gate array (FPGA), etc.

[0239] The description of the above apparatus embodiments is similar to that of the above method embodiments, and has similar beneficial effects. For technical details not disclosed in the apparatus embodiments of this disclosure, please refer to the description of the method embodiments of this disclosure for understanding.

[0240] It should be noted that, in the embodiments of this disclosure, if the above-described vehicle driving control method is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiments of this disclosure, or the part that contributes to related technologies, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several voice commands to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the methods of the various embodiments of this disclosure. The aforementioned storage medium includes various media capable of storing program code, such as a USB flash drive, external hard drive, read-only memory (ROM), magnetic disk, or optical disk. Thus, the embodiments of this disclosure are not limited to any specific hardware and software combination.

[0241] Thirdly, embodiments of this disclosure provide another electronic device that can implement the skill scheduling method provided in the first aspect above.

[0242] In one example, reference Figure 4The electronic device 40, as shown, includes a vehicle controller 401, at least one communication bus 402, a user interface 403, at least one external communication interface 404, and a memory 405. The communication bus 402 is configured to enable communication between these components. The user interface 403 may include a display screen, a microphone, etc. The external communication interface 404 may include standard wired and wireless interfaces.

[0243] The memory 405 is configured to store voice commands and applications executable by the vehicle controller 401, and can also cache data to be processed or already processed (e.g., image data, audio data, voice communication data and video communication data) in the vehicle controller 401 and various modules in the electronic devices. It can be implemented by flash memory or random access memory (RAM).

[0244] In some embodiments, this disclosure provides a vehicle including a battery, a vehicle controller, and a memory, wherein the memory stores a computer program or voice commands, and when the computer program is executed by the vehicle controller, it implements the method provided in the first aspect.

[0245] Fourthly, embodiments of this disclosure provide a storage medium, namely a computer-readable storage medium, on which a computer program or voice instruction is stored, wherein when the computer program or voice instruction is executed by a processor, the steps of any of the skill scheduling methods provided in the first aspect of the above embodiments are implemented.

[0246] Fifthly, embodiments of this disclosure provide a computer program product, which includes a computer program or voice instructions. When the computer program or voice instructions are executed by a processor, they implement the steps in any of the skill scheduling methods provided in the first aspect of the above embodiments.

[0247] It should be noted that the descriptions of the above embodiments of storage media, devices, apparatuses, and program products are similar to the descriptions of the above method embodiments, and have similar beneficial effects. For technical details not disclosed in the embodiments of storage media, devices, apparatuses, and program products of this disclosure, please refer to the descriptions of the method embodiments of this disclosure for understanding.

[0248] It should be understood that the phrase "one embodiment" or "an embodiment" throughout the specification means that a specific feature, structure, or characteristic related to the embodiment is included in at least one embodiment of this disclosure. Therefore, "in one embodiment" or "in some embodiments" appearing throughout the specification do not necessarily refer to the same embodiment. Furthermore, these specific features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. It should be understood that in the various embodiments of this disclosure, the sequence numbers of the above-described processes do not imply a sequential order of execution; the execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this disclosure. The sequence numbers of the above-described embodiments are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0249] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0250] In the several embodiments provided in this disclosure, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined, integrated into another system, or some features may be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the various components shown or discussed may be through some interfaces. The indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0251] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units. They may be located in one place or distributed across multiple network units. Some or all of the units may be selected to achieve the purpose of this embodiment according to actual needs.

[0252] In addition, each functional unit in the various embodiments of this disclosure can be integrated into one processing unit, or each unit can be a separate unit, or two or more units can be integrated into one unit; the integrated unit can be implemented in hardware or in the form of hardware plus software functional units.

[0253] Those skilled in the art will understand that all or part of the steps of the above method embodiments can be implemented by hardware related to program voice commands. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it performs the steps of the above method embodiments. The aforementioned storage medium includes various media that can store program code, such as mobile storage devices, read-only memory (ROM), magnetic disks, or optical disks.

[0254] Alternatively, if the integrated units described above are implemented as software functional modules and sold or used as independent products, they can also be stored in a computer-readable storage medium. Based on this understanding, the technical solutions of the embodiments of this disclosure, or the parts that contribute to related technologies, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several voice commands to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the methods of the various embodiments of this disclosure. The aforementioned storage medium includes various media capable of storing program code, such as mobile storage devices, ROMs, magnetic disks, or optical disks.

[0255] The above are merely embodiments of this disclosure, but the scope of protection of this disclosure is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this disclosure should be included within the scope of protection of this disclosure.

[0256] It should be understood that if this disclosure references any user data and personal information (including but not limited to device information, behavioral data, location information, etc.) and before applying the technical solutions described in the embodiments of this disclosure, the relevant products or services should comply with the laws and regulations concerning the protection of user data and personal information, strictly process users' personal information and data in accordance with the provisions of applicable laws and regulations throughout the entire data processing lifecycle, follow the principles of legality, legitimacy, necessity, good faith, openness, and transparency, and adopt reasonable privacy design schemes and technical measures to ensure the security of user data and personal information, protect users' legitimate rights and interests, and prevent the risks of leakage, theft, or tampering of user data and personal information.

[0257] Specifically, the company must publish and display its privacy policy in a prominent position on the user interface, clearly informing users of the types, purposes, uses, and methods of processing personal information, as well as other matters that should be disclosed as required by laws and regulations; obtain users' prior informed consent or explicit authorization for data processing through user-initiated interaction (such as confirmation pop-ups); process or store user data securely within the legally required timeframe; adopt a series of security technologies and management measures, including but not limited to data encryption and access control; share and transfer user data within the scope permitted by law and in a legally required manner; and process user rights, including the rights to query, access, correct, delete, withdraw authorization and consent, cancel registration, and obtain copies of personal information, within the legally required timeframe.

Claims

1. A method for skill scheduling, characterized in that, The method includes, Acquire multimodal interaction instructions and write the multimodal interaction instructions into the shared state area; The cockpit agent reads the multimodal interaction instructions from the shared state area, performs intent parsing on the multimodal interaction instructions, and determines the intent parsing result corresponding to the multimodal interaction instructions. Based on the intent parsing result, determine the target cockpit skill corresponding to the intent parsing result from the cockpit skill pool; The risk level of the target cockpit skills is assessed to obtain the risk level assessment result; The target cockpit skill is executed according to the target processing path corresponding to the risk level assessment result, and the skill execution result is obtained and output.

2. The method according to claim 1, characterized in that, The step of executing the target cockpit skill according to the target processing path corresponding to the risk level assessment result, and obtaining the skill execution result, includes: If the risk level assessment result is Level 1, the target cockpit skill is executed through the first processing path corresponding to Level 1 to obtain the skill execution result; the first processing path includes at least file permission verification, vehicle environment isolation, tamper interception, execution of the target cockpit skill in the sandbox, and desensitization processing of the skill execution result; If the risk level assessment result is Level 2, the target cockpit skill is executed through the second processing path corresponding to Level 2 to obtain the skill execution result; The skill execution result is written into the shared state area.

3. The method according to claim 1, characterized in that, Before determining the target cockpit skill matching the intent parsing result from the cockpit skill pool based on the intent parsing result, the method further includes: The multimodal interaction command is subjected to scene recognition to determine the scene recognition result corresponding to the multimodal interaction command; Based on the scene recognition results, the initialization operation corresponding to the scene recognition results is executed. The scene recognition results include at least children's scenes, movie-watching scenes, parking scenes, parking scenes, and traffic warning scenes.

4. The method according to any one of claims 1-3, characterized in that, The method further includes: If the cockpit agent identifies an abnormal skill execution result, data comparison and constraint verification are performed based on the operation data of the target node in the shared state area to determine the abnormal problem point. The target node is the node related to the skill execution result. The abnormal issues are corrected, and the corrected skill execution results are obtained and output.

5. The method according to claim 4, characterized in that, The method also includes, If the stored data in the shared state region is greater than or equal to a preset threshold, the stored data in the shared state region is pruned according to preset pruning parameters to obtain pruned data. The state pruning includes at least session context cleanup, temporary variable deletion, historical log archiving, and target data retention, wherein the target data includes at least user preference data and vehicle status data.

6. The method according to claim 5, characterized in that, The method further includes: The preset trimming parameters are generated based on the user's session preference data.

7. The method according to any one of claims 1-3, characterized in that, Before performing intent parsing on the multimodal interaction instructions, the method further includes: The cockpit agent performs security verification on the multimodal interaction commands to determine a first security verification result; the security verification includes at least voice desensitization verification, target keyword interception, and target information recognition. If the first security verification result indicates that the verification has passed, the security verification identifier corresponding to the multimodal interaction instruction is updated in the shared state area; If the first security verification result indicates that the verification fails, a response result is output according to the first protection template, and first record data is formed based on the multimodal interaction instructions.

8. The method according to claim 7, characterized in that, After obtaining the skill execution result, the method further includes, A second security check is performed on the skill execution result to determine the second security check result; The secondary security verification includes at least content screening and vehicle safety verification; If the second security verification result indicates that the verification has passed, the skill execution result is output; If the second security verification result indicates that the verification fails, a response result is output according to the second protection template, and second record data is formed based on the skill execution result.

9. The method according to any one of claims 1-3, characterized in that, The cockpit skill pool includes at least cockpit hardware control skills, navigation map update skills, multimedia playback skills, vehicle status diagnosis skills, cockpit configuration skills, interactive Q&A skills, children's scenario skills, and sleep mode skills.

10. The method according to any one of claims 1-3, characterized in that, The method further includes: If no feedback instruction regarding the skill execution result is received within the first duration, a sleep mode is triggered, and a cleanup operation is performed via the reducer.

11. A device for skill scheduling, characterized in that, The device includes, The acquisition module is used to acquire multimodal interaction instructions and write the multimodal interaction instructions into the shared state area; The processing module is used to read the multimodal interaction instructions from the shared state area through the cockpit intelligent agent, and to perform intent parsing on the multimodal interaction instructions to determine the intent parsing result corresponding to the multimodal interaction instructions; Based on the intent parsing result, determine the target cockpit skill corresponding to the intent parsing result from the cockpit skill pool; The risk level of the target cockpit skills is assessed to obtain the risk level assessment result; And execute the target cockpit skill according to the target processing path corresponding to the risk level assessment result, obtain the skill execution result and output it.

12. A vehicle, characterized in that, The vehicle includes a vehicle controller, a memory, and a computer program stored in the memory and capable of running on the vehicle controller, wherein the computer program, when executed by the vehicle controller, implements the skill scheduling method as described in any one of claims 1 to 10.

Images