A Multimodal Closed-Loop Control Method for Vehicle State Mapping in an Intelligent Interactive Vehicle Model

By verifying the status data of vehicles and car models to generate joint status information, abstract scene commands that are not bound to hardware are generated, which solves the problems of inaccurate status display and high hardware coupling in intelligent interactive car model products, achieves higher system scalability and operational security, and ensures reliable feedback of execution results and adaptability of interactive experience.

CN123113518APending Publication Date: 2026-11-17WEIFANG NAYIN ELECTRONIC TECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202611275190.7
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2026-08-21
Publication Date
2026-11-17

AI Technical Summary

Technical Problem

Existing intelligent interactive vehicle model products suffer from problems such as inaccurate display of vehicle status information, high coupling between cloud and terminal hardware, insufficient control over operational risks, unclear feedback of execution results, and inflexible adjustment of interactive parameters. They are unable to simultaneously ensure the accuracy of status display, the flexibility of terminal adaptation, the safety of the operation process, the perceptibility of execution results, and the continuous optimization of the interactive experience.

Method used

By acquiring state data from real vehicles and intelligent interactive vehicle model terminals for consistency verification, joint state information is generated. Based on this, abstract scenario commands that are not bound to specific hardware are generated. Scenario matching is performed in conjunction with user interaction and environmental information to determine execution conditions and risk levels. Multimodal expressions are generated and output, thereby achieving decoupling between the cloud and the terminal and risk management.

Benefits of technology

It improves the accuracy and real-time performance of vehicle status mapping, reduces the coupling between the cloud and the terminal, enhances the scalability and operational security of the system, and ensures reliable feedback of execution results and adaptability of the interactive experience.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN123113518A_ABST
    Figure CN123113518A_ABST
Patent Text Reader

Abstract

This application discloses a multimodal closed-loop control method for vehicle state mapping in an intelligent interactive vehicle model, belonging to the field of digital twin and intelligent interactive control technology. It includes: generating joint state information based on verified state data; performing scene matching based on the joint state information, user interaction information, and environmental context information to determine the target interaction scene, and determining scene risk based on the interaction object, execution action, and associated environmental information corresponding to the target interaction scene; determining scene execution conditions based on scene risk and user confirmation status, and generating an abstract scene command corresponding to the target interaction scene when the scene execution conditions are met. The abstract scene command is used to represent the target interaction scene and its execution action and is not bound to a specific hardware execution object; sending the abstract scene command to the intelligent interactive vehicle model terminal, where the intelligent interactive vehicle model terminal maps the abstract scene command into a corresponding multimodal expression and outputs it according to its own state and expression capabilities.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This application relates to the field of digital twin and intelligent interactive control technology, and more specifically, to a multimodal closed-loop control method for vehicle state mapping of an intelligent interactive vehicle model. Background Technology

[0002] With the development of vehicle-to-everything (V2X) technology, more and more automakers and third-party manufacturers are launching intelligent interactive car model products that work in conjunction with real vehicles. Users can intuitively perceive the vehicle's status or interact with it through these models. However, existing intelligent interactive car model products and related control solutions suffer from the following technical problems: Firstly, most existing intelligent interactive car models adopt Bluetooth speaker-style or simple remote control-style interaction methods. Their light, voice and other expressions are usually local preset fixed effects, or are directly displayed based on vehicle status data received at a single moment. They do not jointly verify the vehicle status data and the car model's own status data. In the case of network latency, message disorder, or data loss, they are prone to displaying outdated or distorted vehicle status information to users, causing users to make misjudgments based on incorrect information.

[0003] Secondly, when existing solutions send control commands from the cloud to the car model terminal, they usually directly carry execution parameters related to specific hardware (such as specific light channel numbers, specific audio file identifiers, etc.). This results in a high degree of coupling between the cloud business logic and the specific hardware form of the car model terminal. When the car model, car model hardware version, or expression capabilities change, the relevant cloud processing logic needs to be adapted and adjusted synchronously. This leads to poor scalability and maintenance costs, making it difficult to support the same cloud service to simultaneously adapt to multiple car models and multiple generations of car model hardware products.

[0004] Third, existing solutions typically lack differentiated processing mechanisms for different risk levels when generating operation commands involving vehicle unlocking and door lock control. They also fail to adequately differentiate based on the user's identity and permissions. In scenarios where non-owner users such as visitors or children use the smart interactive vehicle model, accidental triggering or lack of necessary user confirmation steps can easily lead to misoperation of the vehicle or home devices, posing certain security risks.

[0005] Fourth, existing solutions, after receiving control commands and outputting corresponding expressions at the vehicle model terminal, typically do not establish an effective feedback loop with the actual execution results of real vehicles or home devices. Users find it difficult to accurately know whether the command has been actually executed or whether the execution was successful. They may mistakenly believe that the operation has been completed after the command is issued, which can lead to a decline in user experience when the device is offline or execution fails.

[0006] Fifth, the scenario triggering conditions and multimodal expression parameters of existing solutions are usually fixed configurations, without dynamic adjustment based on the user's actual interaction behavior such as confirmation, ignoring, and cancellation of prompts. In actual use, the prompts may cause unnecessary disturbance to the user due to excessive frequency, or affect the user's attention to important prompts due to insufficient expression intensity. Moreover, when adjusting relevant parameters, existing solutions often do not set up corresponding adjustment exemption mechanisms for security-related scenarios, which may reduce the reliability of response in security scenarios due to adaptive parameter adjustment.

[0007] In summary, existing intelligent interactive vehicle models still have shortcomings in areas such as accurate interaction of vehicle status information, cloud-terminal hardware compatibility, operational risk management, execution result feedback, and interaction parameter adjustment. They struggle to simultaneously ensure the accuracy of status display, the flexibility of terminal adaptation, the security of the operation process, the perceptibility of execution results, and the continuous optimization of the interactive experience. Therefore, improving the accuracy of interaction between vehicle status and vehicle model status, reducing the coupling between cloud-based business logic and terminal hardware, achieving differentiated management for different operational risks and user permissions, enhancing the reliability of control command execution feedback, and improving the adaptability of interaction parameters while ensuring the reliability of safety-related interactions have become urgent technical challenges. Summary of the Invention

[0008] To overcome a series of shortcomings in existing technologies, the purpose of this application is to provide a multimodal closed-loop control method for vehicle state mapping in an intelligent interactive vehicle model, comprising: Acquire status data from real vehicle terminals and intelligent interactive vehicle model terminals, perform consistency verification on the status data, and generate joint status information based on the verified status data; Scene matching is performed based on joint state information, user interaction information, and environmental context information to determine the target interaction scene, and scene risk is determined based on the interaction object, execution action, and associated environmental information corresponding to the target interaction scene. The scenario execution conditions are determined based on the scenario risk and the user confirmation status. When the scenario execution conditions are met, an abstract scenario command corresponding to the target interaction scenario is generated. The abstract scenario command is used to represent the target interaction scenario and its execution action, but is not bound to a specific hardware execution object. Abstract scene commands are sent to the intelligent interactive vehicle model terminal, which then maps the abstract scene commands into corresponding multimodal expressions and outputs them based on its own state and expressive capabilities.

[0009] In some embodiments, the abstract scenario command includes at least a scenario identifier, a command identifier, a sequence identifier, a target set, an action set, a scenario risk, a priority, a validity period, and a script version requirement. The scenario identifier identifies the business scenario type corresponding to the target interaction scenario; the command identifier identifies the current abstract scenario command; the sequence identifier represents the execution order of the current abstract scenario command in the corresponding command sequence; the target set indicates one or more execution objects; the action set describes the behavior to be executed using abstract action types decoupled from specific hardware devices; the scenario risk represents the risk level of the target interaction scenario; the priority determines the execution order or preemption relationship between concurrent abstract scenario commands; the validity period limits the allowed execution time range of the abstract scenario command; and the script version requirement limits the compatible version range of the local expression scripts on which the abstract scenario command depends.

[0010] In some embodiments, the method for mapping abstract scene commands to corresponding multimodal expressions and outputting them is as follows: Analyze the target interaction scenario, execution action, and scenario risk level corresponding to the abstract scenario command; Acquire the status information and expressive capability information of the intelligent interactive vehicle model terminal. The status information and expressive capability information include at least one of the following: vehicle model identification, vehicle model capability set, local expressive script version, user preference settings, current lighting expressive status, and current voice output volume. Based on the target interaction scenario, the executed action, the scenario risk level, the status information and the expressive ability information, the local expression script is matched to determine the multimodal expression parameters; Expression control commands are generated based on multimodal expression parameters, and the intelligent interactive vehicle model terminal is controlled to output the corresponding multimodal expressions.

[0011] In some embodiments, the method for determining scenario risk is as follows: Obtain the interaction object type, execution action, action consequence level, and associated environment information corresponding to the target interaction scenario, and obtain the user mode information and corresponding user permission level of the user who initiated the target interaction scenario; Based on the type of interactive object, the level of action consequences, the associated environmental information, and the user's permission level, the initial risk level is determined according to the preset risk assessment rules. Adjust the initial risk level based on user pattern information to determine the scenario risk level corresponding to the target interaction scenario; When the user mode information corresponds to the guest mode and the execution action involves vehicle control or home device control, the initial risk level is increased. When the user mode information corresponds to the child mode, determine whether the action to be performed belongs to the preset restricted actions; When an action falls under a preset restriction, the generation of the corresponding abstract scene command is prohibited. When an action does not fall under the preset restrictions, the scenario risk level is determined to be low risk.

[0012] In some embodiments, the method for determining scenario execution conditions based on scenario risk and user confirmation status is as follows: Based on the scenario risks corresponding to the target interaction scenario, determine the corresponding execution condition judgment rules; Based on the execution condition determination rules, obtain the validity of the joint state information, the online status of the intelligent interactive vehicle model terminal, and the user confirmation status; When the intelligent interactive vehicle model terminal is online, the corresponding scenario execution conditions are determined based on the scenario risks: When the scenario risk level is low, the scenario execution conditions are determined to be met when the joint state information is valid. When the scenario risk level is medium risk, the scenario execution condition is determined to be met when the user confirmation status includes at least one of voice confirmation, touch confirmation and application confirmation. When the scenario risk level is high, the validity of user identity information, vehicle binding relationship information and interactive object device certificate is verified. When the verification result meets the preset conditions and the user confirmation status meets the multi-factor confirmation conditions or secondary confirmation conditions, the scenario execution conditions are determined to be met. When the intelligent interactive vehicle model terminal is offline, the scene execution conditions are determined to be unmet.

[0013] In some embodiments, the method for obtaining the user confirmation status is as follows: Based on the scenario execution conditions corresponding to the target interaction scenario, determine whether a user confirmation requirement exists; When a user confirmation request is required, a confirmation prompt message corresponding to the target interaction scenario is generated, and the intelligent interactive vehicle model terminal is controlled to output the confirmation prompt message. Within a preset waiting time, obtain the confirmation interaction information entered by the user in response to the confirmation prompt; The user's confirmation status is determined based on the confirmation interaction information, and the user's confirmation status is matched with the preset confirmation conditions; When the user's confirmation status meets the preset confirmation conditions, it is determined that the user's confirmation is valid, and it is determined that the scenario execution conditions are met. If a user confirmation status that meets the preset confirmation conditions is not obtained within the preset waiting time, the user confirmation is determined to be invalid, the scenario execution conditions are determined to be unmet, and the corresponding confirmation failure event is recorded.

[0014] In some embodiments, the method for scene matching based on joint state information, user interaction information, and environmental context information is as follows: When user interaction information is received or a preset change in the joint state information is detected, the scene matching process is triggered. Obtain current environmental context information, and generate scene matching input information based on joint state information, user interaction information, and environmental context information; The scene matching input information is matched with multiple scene rules in a preset scene rule base to obtain candidate interaction scenes; wherein, the scene rules include at least triggering conditions, necessary preconditions, prohibition conditions and corresponding execution actions; Determine whether the triggering conditions and necessary preconditions corresponding to each candidate interaction scenario are met, and whether any corresponding prohibition conditions are triggered. When there is a candidate interaction scenario that meets the triggering condition and the necessary preconditions but does not trigger the prohibition condition, the candidate interaction scenario is determined as the target interaction scenario. Specifically: when there are multiple valid candidate scenarios, the multiple valid candidate scenarios are sorted according to the preset scenario category priority, and the valid candidate scenario with the highest priority is determined as the target interaction scenario; when there is only one valid candidate scenario, the valid candidate scenario is determined as the target interaction scenario.

[0015] In some embodiments, the method for generating joint state information is as follows: Update the verified vehicle status data to the vehicle equipment status record maintained in the cloud, and update the most recent valid timestamp corresponding to the vehicle status data. Update the verified model car status data to the model car device status record maintained in the cloud; When the preset scenario matching trigger conditions are met, the vehicle equipment status record and the vehicle model equipment status record corresponding to the current moment are obtained; Based on the vehicle equipment status records and the model vehicle equipment status records, a joint state fusion is performed to generate a joint state snapshot, and the joint state snapshot is determined as the joint state information.

[0016] In some embodiments, it also includes: When an abstract scene command is associated with the control of an external device, the corresponding external device is controlled to execute according to the abstract scene command, and the corresponding execution result is obtained. The feedback status is determined based on the execution result, and a feedback instruction is generated based on the feedback status to control the intelligent interactive vehicle model terminal to output the corresponding multimodal feedback. Collect user interaction behaviors related to multimodal expression and multimodal feedback, and adjust scene matching parameters and multimodal expression parameters based on the interaction behaviors for subsequent scene matching and multimodal expression control.

[0017] In some embodiments, a feedback state is determined based on the execution result, and a feedback instruction is generated based on the feedback state to control the intelligent interactive vehicle model terminal to output corresponding multimodal feedback, including the following steps: Obtain the execution results or overall execution results corresponding to external devices, and determine the corresponding execution status; The feedback status and corresponding feedback instructions are determined based on the execution status, where: When the execution status is "all executed successfully", the feedback status is determined to be "successful", and a first feedback instruction including confirmation lighting effect and confirmation voice is generated. When the execution status is partially successful, the feedback status is determined to be partially successful, and a second feedback instruction including local prompt light effects and voice description of the execution object status is generated; When the execution status is at least one of the following: all executions failed, device offline, insufficient permissions, execution timeout, or not supported, the feedback status is determined to be a failure status, and a third feedback instruction including error prompt light effects and voice broadcast of the failure reason is generated. Based on the first feedback instruction, the second feedback instruction, or the third feedback instruction, the intelligent interactive vehicle model terminal is controlled to output corresponding multimodal feedback.

[0018] Compared with the prior art, this application has the following beneficial effects: First, by verifying the timeliness, source reliability, and sequence consistency of vehicle status data and car model status data respectively, and generating joint status information based on the verified status data, it is possible to effectively avoid status display errors caused by network latency, message out-of-order, or data loss, improve the accuracy and real-time performance of the mapping between vehicle status and car model status, and enable users to obtain a more reliable vehicle status perception experience.

[0019] Secondly, by generating abstract scene commands that are not bound to specific hardware execution objects, and having the intelligent interactive vehicle model terminal map these commands to specific multimodal expression parameters based on its own state and expressive capabilities, the cloud-based business logic and the specific hardware form of the vehicle model terminal are effectively decoupled. This allows the same set of cloud-based processing logic to be adapted to intelligent interactive vehicle model terminals of different models and hardware versions, significantly improving the system's scalability and reducing the maintenance costs of adapting to multiple models and hardware versions.

[0020] Third, by combining the type of interactive object, the level of action consequences, the associated environmental information, and the user mode information to classify and determine the risks of the scenario, and setting corresponding execution conditions and user confirmation requirements for different risk levels, it can ensure the ease of operation in low-risk scenarios while implementing control measures such as identity verification and multi-factor confirmation for high-risk operations involving vehicle control and home device control. In particular, in visitor mode and child mode, it can differentiate and restrict the corresponding operation permissions, effectively reducing the security risks caused by misoperation and improving the overall operational security of the system. Attached Figure Description

[0021] Figure 1 A schematic diagram of the overall architecture of the intelligent interactive vehicle model vehicle state mapping and multimodal feedback closed-loop control system provided in the embodiments of this application; Figure 2 A schematic diagram of the overall process of the vehicle state mapping and multimodal feedback closed-loop control method provided in the embodiments of this application; Figure 3 A flowchart illustrating the method for determining scenario risks and scenario execution conditions provided in the embodiments of this application; Figure 4 A flowchart illustrating the abstract scene command generation and intelligent interactive vehicle model terminal multimodal expression mapping method provided in this application embodiment; Figure 5 A flowchart illustrating the parameter adaptive optimization method based on user interaction behavior provided in this application embodiment. Detailed Implementation

[0022] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of the embodiments of this invention will be described in more detail below with reference to the accompanying drawings. In the drawings, the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The described embodiments are some embodiments of this invention, but not all embodiments.

[0023] Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0024] The embodiments and directional terms described below with reference to the accompanying drawings are exemplary and intended to explain the present invention, and should not be construed as limiting the present invention.

[0025] In a broad embodiment of the present invention, a multimodal closed-loop control method for vehicle state mapping of an intelligent interactive vehicle model includes: Acquire status data from real vehicle terminals and intelligent interactive vehicle model terminals, perform consistency verification on the status data, and generate joint status information based on the verified status data; Scene matching is performed based on joint state information, user interaction information, and environmental context information to determine the target interaction scene, and scene risk is determined based on the interaction object, execution action, and associated environmental information corresponding to the target interaction scene. The scenario execution conditions are determined based on the scenario risk and the user confirmation status. When the scenario execution conditions are met, an abstract scenario command corresponding to the target interaction scenario is generated. The abstract scenario command is used to represent the target interaction scenario and its execution action, but is not bound to a specific hardware execution object. Abstract scene commands are sent to the intelligent interactive vehicle model terminal, which then maps the abstract scene commands into corresponding multimodal expressions and outputs them based on its own state and expressive capabilities. When an abstract scene command is associated with the control of an external device, the corresponding external device is controlled to execute according to the abstract scene command, and the corresponding execution result is obtained. The feedback status is determined based on the execution result, and a feedback instruction is generated based on the feedback status to control the intelligent interactive vehicle model terminal to output the corresponding multimodal feedback. Collect user interaction behaviors related to multimodal expression and multimodal feedback, and adjust scene matching parameters and multimodal expression parameters based on the interaction behaviors for subsequent scene matching and multimodal expression control.

[0026] In some embodiments, the abstract scenario command includes at least a scenario identifier, a command identifier, a sequence identifier, a target set, an action set, a scenario risk, a priority, a validity period, and a script version requirement. The scenario identifier identifies the business scenario type corresponding to the target interaction scenario; the command identifier identifies the current abstract scenario command; the sequence identifier represents the execution order of the current abstract scenario command in the corresponding command sequence; the target set indicates one or more execution objects; the action set describes the behavior to be executed using abstract action types decoupled from specific hardware devices; the scenario risk represents the risk level of the target interaction scenario; the priority determines the execution order or preemption relationship between concurrent abstract scenario commands; the validity period limits the allowed execution time range of the abstract scenario command; and the script version requirement limits the compatible version range of the local expression scripts on which the abstract scenario command depends.

[0027] In some embodiments, the method for mapping abstract scene commands to corresponding multimodal expressions and outputting them is as follows: Analyze the target interaction scenario, execution action, and scenario risk level corresponding to the abstract scenario command; Acquire the status information and expressive capability information of the intelligent interactive vehicle model terminal. The status information and expressive capability information include at least one of the following: vehicle model identification, vehicle model capability set, local expressive script version, user preference settings, current lighting expressive status, and current voice output volume. Based on the target interaction scenario, the executed action, the scenario risk level, the status information and the expressive ability information, the local expression script is matched to determine the multimodal expression parameters; Expression control commands are generated based on multimodal expression parameters, and the intelligent interactive vehicle model terminal is controlled to output the corresponding multimodal expressions.

[0028] In some embodiments, it also includes: Receive abstract scenario commands and parse the corresponding command identifier and sequence identifier; The command identifier is matched with the local set of processed commands to determine whether the abstract scenario command has been processed. When the abstract scenario command has been processed, return the historical execution result corresponding to the command identifier; When the abstract scenario command is not processed, the sequence identifier is compared with the latest local sequence identifier; When the sequence identifier is earlier than the latest sequence identifier, the abstract scenario command is rejected and a command conflict information is sent to the cloud. When the sequence identifier is not earlier than the latest sequence identifier, execute the abstract scenario command and update the set of processed commands and the latest sequence identifier; Based on the validity period of abstract scenario commands, expired command records in the processed command set are cleaned up.

[0029] In some embodiments, a feedback state is determined based on the execution result, and a feedback instruction is generated based on the feedback state to control the intelligent interactive vehicle model terminal to output corresponding multimodal feedback, including the following steps: Obtain the execution results or overall execution results corresponding to external devices, and determine the corresponding execution status; The feedback status and corresponding feedback instructions are determined based on the execution status, where: When the execution status is "all executed successfully", the feedback status is determined to be "successful", and a first feedback instruction including confirmation lighting effect and confirmation voice is generated. When the execution status is partially successful, the feedback status is determined to be partially successful, and a second feedback instruction including local prompt light effects and voice description of the execution object status is generated; When the execution status is at least one of the following: all executions failed, device offline, insufficient permissions, execution timeout, or not supported, the feedback status is determined to be a failure status, and a third feedback instruction including error prompt light effects and voice broadcast of the failure reason is generated. Based on the first feedback instruction, the second feedback instruction, or the third feedback instruction, the intelligent interactive vehicle model terminal is controlled to output corresponding multimodal feedback.

[0030] In some embodiments, the method for adjusting scene matching parameters and multimodal expression parameters based on interaction behavior is as follows: Obtain the scene identifier and user interaction behavior data corresponding to the target interaction scenario, and perform statistics based on the scene identifier to obtain the corresponding number of confirmation operations, number of ignored operations, number of canceled operations, number of repeated operations, and response time; Based on the number of confirmed operations, the number of ignored operations, the number of canceled operations, the number of repeated operations, and the response time, the results of scene matching parameter adjustment and multimodal expression parameter adjustment are determined. Determine whether the target interaction scenario belongs to a security alarm scenario or a high-risk scenario; When the scenario does not fall under the category of safety alarms or high-risk scenarios, the trigger time threshold of the corresponding scenario rules is adjusted according to the scenario matching parameter adjustment results, and the lighting expression parameters, voice broadcast parameters, and prompt expression parameters are adjusted according to the multimodal expression parameter adjustment results. When the scenario falls under the category of security alerts or high-risk scenarios, the triggering conditions and user confirmation requirements of the corresponding scenario rules remain unchanged.

[0031] In some embodiments, the method for determining scenario risk is as follows: Obtain the interaction object type, execution action, action consequence level, and associated environment information corresponding to the target interaction scenario, and obtain the user mode information and corresponding user permission level of the user who initiated the target interaction scenario; Based on the type of interactive object, the level of action consequences, the associated environmental information, and the user's permission level, the initial risk level is determined according to the preset risk assessment rules. Adjust the initial risk level based on user pattern information to determine the scenario risk level corresponding to the target interaction scenario; When the user mode information corresponds to the guest mode and the execution action involves vehicle control or home device control, the initial risk level is increased. When the user mode information corresponds to the child mode, determine whether the action to be performed belongs to the preset restricted actions; When an action falls under a preset restriction, the generation of the corresponding abstract scene command is prohibited. When an action does not fall under the preset restrictions, the scenario risk level is determined to be low risk.

[0032] In some embodiments, the method for determining scenario execution conditions based on scenario risk and user confirmation status is as follows: Based on the scenario risks corresponding to the target interaction scenario, determine the corresponding execution condition judgment rules; Based on the execution condition determination rules, obtain the validity of the joint state information, the online status of the intelligent interactive vehicle model terminal, and the user confirmation status; When the intelligent interactive vehicle model terminal is online, the corresponding scenario execution conditions are determined based on the scenario risks: When the scenario risk level is low, the scenario execution conditions are determined to be met when the joint state information is valid. When the scenario risk level is medium risk, the scenario execution condition is determined to be met when the user confirmation status includes at least one of voice confirmation, touch confirmation and application confirmation. When the scenario risk level is high, the validity of user identity information, vehicle binding relationship information and interactive object device certificate is verified. When the verification result meets the preset conditions and the user confirmation status meets the multi-factor confirmation conditions or secondary confirmation conditions, the scenario execution conditions are determined to be met. When the intelligent interactive vehicle model terminal is offline, the scene execution conditions are determined to be unmet.

[0033] In some embodiments, the method for obtaining the user confirmation status is as follows: Based on the scenario execution conditions corresponding to the target interaction scenario, determine whether a user confirmation requirement exists; When a user confirmation request is required, a confirmation prompt message corresponding to the target interaction scenario is generated, and the intelligent interactive vehicle model terminal is controlled to output the confirmation prompt message. Within a preset waiting time, obtain the confirmation interaction information entered by the user in response to the confirmation prompt; The user's confirmation status is determined based on the confirmation interaction information, and the user's confirmation status is matched with the preset confirmation conditions; When the user's confirmation status meets the preset confirmation conditions, it is determined that the user's confirmation is valid, and it is determined that the scenario execution conditions are met. If a user confirmation status that meets the preset confirmation conditions is not obtained within the preset waiting time, the user confirmation is determined to be invalid, the scenario execution conditions are determined to be unmet, and the corresponding confirmation failure event is recorded.

[0034] In some embodiments, the method for scene matching based on joint state information, user interaction information, and environmental context information is as follows: When user interaction information is received or a preset change in the joint state information is detected, the scene matching process is triggered. Obtain current environmental context information, and generate scene matching input information based on joint state information, user interaction information, and environmental context information; The scene matching input information is matched with multiple scene rules in a preset scene rule base to obtain candidate interaction scenes; wherein, the scene rules include at least triggering conditions, necessary preconditions, prohibition conditions and corresponding execution actions; Determine whether the triggering conditions and necessary preconditions corresponding to each candidate interaction scenario are met, and whether any corresponding prohibition conditions are triggered. When there is a candidate interaction scenario that meets the triggering condition and the necessary preconditions but does not trigger the prohibition condition, the candidate interaction scenario is determined as the target interaction scenario. Specifically: when there are multiple valid candidate scenarios, the multiple valid candidate scenarios are sorted according to the preset scenario category priority, and the valid candidate scenario with the highest priority is determined as the target interaction scenario; when there is only one valid candidate scenario, the valid candidate scenario is determined as the target interaction scenario.

[0035] In some embodiments, the method for generating joint state information is as follows: Update the verified vehicle status data to the vehicle equipment status record maintained in the cloud, and update the most recent valid timestamp corresponding to the vehicle status data. Update the verified model car status data to the model car device status record maintained in the cloud; When the preset scenario matching trigger conditions are met, the vehicle equipment status record and the vehicle model equipment status record corresponding to the current moment are obtained; Based on the vehicle equipment status records and the model vehicle equipment status records, a joint state fusion is performed to generate a joint state snapshot, and the joint state snapshot is determined as the joint state information.

[0036] In some embodiments, the method for performing consistency verification on state data is as follows: The system parses the first timestamp, first data source identifier, and first serial number corresponding to the vehicle status data, as well as the second timestamp, second data source identifier, and second serial number corresponding to the vehicle model status data. The timeliness of vehicle status data and vehicle model status data is determined based on the first timestamp and the second timestamp, respectively. The first data source identifier and the second data source identifier are matched with preset source verification rules to determine the reliability of the data source of vehicle status data and car model status data; The first serial number is compared with the vehicle status serial number stored in the cloud, and the second serial number is compared with the car model status serial number stored in the cloud to determine the serial consistency between the vehicle status data and the car model status data. When the timeliness, source reliability, and sequence consistency of the vehicle status data and the model car status data all meet the preset conditions, the status data verification is deemed successful.

[0037] In some embodiments, the method for obtaining the execution result is as follows: The external device control object is determined based on the abstract scenario command, and the corresponding control instructions are generated; Based on the command identifier corresponding to the abstract scenario command, control instructions are sent to the external device control object, and the corresponding execution result data is received. When the external device control object includes multiple external devices, obtain the sub-execution result corresponding to each external device respectively; Aggregation is performed based on preset result aggregation rules to generate the overall execution result corresponding to the command identifier.

[0038] In some embodiments, the intelligent interactive vehicle model terminal includes a main control module, a communication module, a security and storage module, a voice interaction module, a multi-zone lighting module, and a power supply and charging module; The main control module is used to perform abstract scenario command parsing, idempotency judgment, priority arbitration, and multimodal expression mapping. The communication module is used to receive abstract scene commands issued by the cloud and report vehicle model status data and execution feedback information, and supports at least one communication method among cellular network, wireless local area network and near field wireless communication. The security and storage module is used to store device certificates, user binding relationship summaries, local expression scripts, command processing records, and execution logs. The voice interaction module is used to collect voice input information and output voice broadcasts and prompts. The multi-zone lighting module includes two or more lighting zones whose brightness, color, and change sequence can be controlled independently, for outputting lighting expressions; The power supply and charging module is used for power supply, charging management, and low battery protection.

[0039] In some embodiments, the user identity information includes user mode information, and the user mode includes owner mode, guest mode and child mode.

[0040] The preferred embodiments of the present invention are described below with reference to the accompanying drawings, and the present invention will be further described in detail.

[0041] like Figure 1As shown, the vehicle state mapping and multimodal feedback closed-loop control method provided in this application is applied to a system architecture including a real vehicle terminal, an intelligent interactive vehicle model terminal, and a cloud-based collaborative layer, and optionally also includes a home device terminal. The real vehicle terminal is an actual vehicle equipped with a vehicle-to-everything (V2X) communication module, capable of reporting status data such as doors, windows, remaining battery power, driving range, and location to the cloud-based collaborative layer, and receiving control commands from the cloud-based collaborative layer to execute corresponding actions. The intelligent interactive vehicle model terminal is a scaled-down model with the same or similar appearance as the real vehicle, containing a main control module, communication module, security and storage module, voice interaction module, multi-zone lighting module, and power and charging module. It is used to report its online status, capability set, local expression script version, and other information, and to receive abstract scene commands from the cloud-based collaborative layer, outputting corresponding lighting, voice, and sound effects. The home device terminal accesses the cloud-based collaborative layer through a home IoT platform, and is used to execute control actions for home devices such as lighting and air conditioning in linked scenarios. The cloud-based collaboration layer, as the core processing layer, undertakes tasks such as state data consistency verification, joint state information generation, scenario matching, scenario risk determination, abstract scenario command generation, execution result aggregation, and parameter optimization. It can be implemented based on a device access and rule engine platform, or on a self-built microservice architecture. The following section, in conjunction with specific application scenarios, further elaborates on... Figure 2 The implementation of each processing step in the method shown is explained in detail.

[0042] The specific structure of the intelligent interactive car model terminal is as follows: The main control module is used to perform abstract scene command parsing, command idempotency judgment, priority arbitration between multiple candidate scenes, and multimodal expression mapping processing, and can be implemented using an embedded processor; the communication module is used to receive abstract scene commands issued by the cloud collaboration layer and report car model status data and execution feedback information, supporting at least one of the following communication methods: cellular network, wireless LAN, and near-field wireless communication, to adapt to the network access needs of different usage scenarios such as home and vehicle; the security and storage module is used to store device certificates, user binding relationship summaries, local expression scripts, command processing records, and execution logs, providing a data foundation for identity and certificate verification in high-risk scenarios; the voice interaction module is used to collect user voice input information and output voice broadcasts and prompts, and the voice recognition process can be completed locally or in the cloud; the multi-zone lighting module includes two or more lighting zones that can independently control brightness, color, and change sequence, such as corresponding to the headlights, ambient lights, and taillights of the car model, used to output the specific expression effects corresponding to the aforementioned lighting expression parameters; the power supply and charging module is used to power the intelligent interactive car model terminal and realize charging management and low battery protection. The above modules work together to provide the hardware implementation foundation for the local mapping and execution of abstract scene commands into multimodal expressions.

[0043] Taking the specific application scenario of a user driving home from work as an example: When the real vehicle detects events such as unlocking, starting, or location change, it reports the corresponding vehicle status data to the cloud collaboration layer. This vehicle status data carries a first timestamp representing the time the data was generated, a first data source identifier representing the data's origin, and a first sequence number representing the data's order. When the intelligent interactive vehicle model terminal detects changes in its online status, light status, or battery status, it synchronously reports the corresponding vehicle model status data. This vehicle model status data carries a second timestamp, a second data source identifier, and a second sequence number. After receiving the above status data, the cloud collaboration layer first performs a consistency check: Let the current system time in the cloud be... The timestamp corresponding to the status data is The preset effective time limit threshold is The timeliness of the status data is determined as shown in formula (1).

[0044] Formula (1) in, This indicates the generation delay of the status data, which is the time interval from the generation of the status data to its arrival at the cloud. The value is derived from the device's local clock and the cloud's reception time. For preset thresholds based on specific state types, such as door status, alarm status, etc. The possible value is 3 seconds, for location-related states. The possible value is 10 seconds. When... Not greater than If the data is deemed to meet timeliness requirements, it is considered timed out. If timeliness is deemed acceptable, the cloud collaboration layer further matches the first and second data source identifiers with a pre-configured source whitelist to confirm the data source's credibility. Simultaneously, it compares the first sequence number with the vehicle status sequence number already saved in the cloud, and the second sequence number with the vehicle model status sequence number already saved in the cloud. Only if the current sequence number is not less than the saved sequence number is the data deemed to meet sequence consistency requirements, preventing old status data from overwriting new status data due to network retransmissions or message out-of-order delivery. If the timeliness, source credibility, and sequence consistency criteria all meet preset conditions, the status data verification is deemed successful; otherwise, the verification is deemed unsuccessful. In this case, the cloud collaboration layer only records the status data and does not use it for subsequent processing or as a basis for triggering high-risk abstract scenario commands. For example, when vehicle status data is affected by network fluctuations... =5 seconds, exceeded the preset threshold At 3 seconds, when the intelligent interactive vehicle model terminal responds to the user's voice inquiry "Is the vehicle locked?", it only displays the most recently saved valid locking status and announces "Vehicle status cannot be confirmed temporarily, please try again later" to avoid outputting a potentially distorted definitive conclusion.

[0045] After the status data verification is successful, the cloud collaboration layer updates the vehicle status data to the vehicle device status record. This record stores fields such as door lock status, door status, trunk status, remaining battery power, driving range, charging status, air conditioning status, location information, and alarm information, and synchronously updates the corresponding most recent valid timestamp. Simultaneously, it updates the vehicle model status data to the vehicle model device status record, storing fields such as online status, capability set, local expression script version, current expression status, and firmware version. When the scene matching trigger condition is met, such as receiving the user's voice interaction message "I'm almost home," or detecting that the vehicle's location has entered a preset geofence range, the cloud collaboration layer reads and merges the current vehicle device status record and vehicle model device status record in real time, generating a joint status snapshot, which is then identified as the joint status information. The cloud-based collaboration layer further acquires current environmental context information, including outdoor temperature, cabin temperature, current time period, and vehicle's geographical location. It combines this information with the combined status information, user interaction information, and environmental context information to form scene matching input. This input is then matched against multiple scene rules in a pre-defined scene rule base. Each scene rule includes triggering conditions (e.g., the vehicle is less than 500 meters from the home's geofence), necessary preconditions (e.g., the vehicle model terminal is online and the battery level is above 10%), prohibiting conditions (e.g., the vehicle is in maintenance mode), and corresponding actions (e.g., turning on welcome lights and pre-starting the air conditioning). When a scene rule simultaneously meets the triggering conditions, necessary preconditions, and does not trigger the prohibiting conditions, it is identified as a candidate interaction scene. When multiple candidate interaction scenes exist simultaneously, they are sorted according to a pre-defined scene category priority order (in order: safety alarms, vehicle anomalies, user-initiated commands, proactive reminders, and decorative lighting effects), with the highest priority scene being identified as the target interaction scene. In the aforementioned "coming home" scenario, because the user interaction information clearly expresses proactive intent, this scenario is typically categorized as a user-initiated command scenario.

[0046] After identifying the target interaction scenario, the cloud collaboration layer further performs scenario risk assessment and processing, the specific process of which is as follows: Figure 3As shown. The cloud collaboration layer first obtains the type of interactive object (real vehicle, intelligent interactive car model terminal or home device) to which the target interactive scene is directed, the level of the action consequence corresponding to the action, and the associated environmental information (including the current vehicle status or home status of the interactive object, the identity of the user who initiated the scene matching, the binding relationship between the user and the interactive object, the validity of the interactive object's device certificate, the online status of the interactive object and the environmental conditions in which it is located), and obtains the user mode information (owner mode, visitor mode or child mode) and the corresponding user permission level, and calculates the initial risk value according to the weighted summation model shown in formula (2).

[0047] Formula (2) in, This represents the initial risk value corresponding to the target interaction scenario, with a value range of [0,1]. (i=1,2,3,4) represent the quantitative values ​​of the interaction object type, the quantitative value of the action consequence level, the quantitative value of the associated environmental risk, and the quantitative value of the user permission deficiency, respectively. Each quantitative value is pre-mapped to the [0,1] interval according to a preset rule. For example, the quantitative value of the interaction object type The corresponding values ​​for intelligent interactive vehicle model terminals are 0.2, home devices are 0.4, and real vehicles are 0.6, representing the quantitative values ​​for the consequences of actions. The corresponding values ​​for display-type actions are 0.2, for home device control actions are 0.5, and for vehicle control actions are 0.9. Preset weighting coefficients and satisfying It is acceptable =0.3、 =0.3、 =0.2、 =0.2. After the initial risk value is determined, the cloud collaboration layer adjusts it according to the user pattern information to obtain the scenario risk value, as shown in formula (3).

[0048] Formula (3) in, This indicates the adjusted scenario risk value. User mode adjustment factor: In master mode Value 1.0; when in guest mode and the action involves controlling vehicle or home devices. Values ​​greater than 1 (e.g., 1.3) are used to increase the risk level. In child mode, it is further determined whether the action being performed belongs to a preset restricted action. If it does, the generation of abstract scene commands is directly prohibited; otherwise, the risk level is set to low risk. Taking the homecoming scene as an example, let... =0.6、 =0.5、 =0.3、 =0.4, substituting into formula (2) yields =0.3×0.6+0.3×0.5+0.2×0.3+0.2×0.4=0.47. If this scenario is initiated by a guest mode user, take... =1.3, substituting into formula (3) yields =0.47 × 1.3 = 0.611. The cloud-based collaboration layer has preset risk level classification thresholds. <0.35 indicates low risk, 0.35≤ <0.7 indicates medium risk. A value ≥0.7 indicates high risk. =0.611 corresponds to a medium risk level, and user confirmation is required before execution.

[0049] Furthermore, when the scenario risk is low-risk, the scenario execution conditions are directly determined to be met, provided the joint status information is valid and the intelligent interactive vehicle model terminal is online. When the risk level is medium-risk, the user confirmation status corresponding to at least one of the confirmation methods—voice confirmation, touch confirmation, or application confirmation—must be obtained. When the risk level is high-risk, the validity of the user's identity information, vehicle binding relationship information, and the interactive object's device certificate must also be verified, and the scenario execution conditions are only met when the verification passes and the user confirmation status meets the multi-factor confirmation conditions or secondary confirmation conditions. When the intelligent interactive vehicle model terminal is offline, the scenario execution conditions are determined to be unmet. The process of obtaining the user confirmation status is as follows: based on the scenario execution conditions, it is determined whether there is a confirmation requirement. If so, a confirmation prompt message is generated and the vehicle model terminal is controlled to output it. Within a preset waiting time (e.g., 10 seconds), the user confirmation interaction information is obtained and matched with the preset confirmation conditions. If no valid confirmation is obtained within the time limit, the confirmation is determined to be invalid, the scenario execution conditions are determined to be unmet, and the confirmation failure event is recorded for subsequent parameter optimization.

[0050] In the specific application scenario of the children's mode, if the user mode information of the user who initiates the scene matching is children's mode, and the execution action corresponding to the target interaction scene is a preset restricted action such as vehicle unlocking or door lock control, then the cloud collaboration layer directly determines that the corresponding abstract scene command is prohibited from being generated in the scene risk determination stage, and controls the intelligent interactive car model terminal to output prompt voice (such as "This operation is temporarily unavailable"), and no longer enters the subsequent scene execution condition determination stage; if the execution action is not a preset restricted action, such as only checking the current battery level of the vehicle or playing entertainment sound effects, then the risk level of the scene is directly determined to be low risk level, and it is automatically executed under the condition that the joint state information is valid and the intelligent interactive car model terminal is online, without user confirmation, so as to preserve the ability of children to have daily entertainment interaction with the intelligent interactive car model terminal while ensuring the safety of vehicle and home equipment operation. In the application scenario of the visitor mode, if the visitor user initiates an action involving home air conditioning or vehicle window control, the cloud collaboration layer follows the formula (3) Adjustment rules with values ​​greater than 1 increase the risk level of the corresponding scenario. This would raise a scenario that might be considered low or medium risk in owner mode to medium or high risk in visitor mode, thereby triggering corresponding user confirmation or multi-factor verification requirements and achieving differentiated control over visitor operation permissions.

[0051] Once the scenario execution conditions are met, the cloud-based collaboration layer generates an abstract scenario command corresponding to the target interaction scenario. This command carries at least a scenario identifier, command identifier, sequence identifier, target set, action set, scenario risk, priority, validity period, and script version requirements. The scenario identifier identifies the business scenario type to which the target interaction scenario belongs (e.g., "welcome home"). The command identifier uniquely identifies the current command. The sequence identifier represents the order of commands within the command sequence corresponding to the same scenario. The target set indicates one or more execution objects. The action set describes the behavior to be executed using abstract action types decoupled from specific hardware (e.g., "welcome lighting effect," "pre-start air conditioning"). After receiving the abstract scenario command, the intelligent interactive vehicle model terminal parses its command identifier and sequence identifier and matches them with the locally processed command set. If the command identifier already exists, the historical execution result is returned directly. If it does not exist, the sequence identifier is compared with the latest local sequence identifier. If it is earlier, execution is rejected and conflict information is reported. If it is not earlier, execution is performed and the processed command set and the latest local sequence identifier are updated. Simultaneously, expired records in the processed command set are cleaned up based on the command validity period, thus handling network retransmissions or out-of-order messages in an idempotent manner. During the execution phase, the main control module parses the target interaction scenario, execution action, and scenario risk level corresponding to the abstract scenario command, and obtains the vehicle model identifier, vehicle model capability set, local expression script version, user preference settings, current lighting expression status, and current voice output volume. Based on this, it matches the multimodal expression parameters from the local expression script. Taking the lighting expression brightness parameter as an example, its calculation method is shown in formula (4), and the corresponding processing flow is as follows: Figure 4 As shown.

[0052] Formula (4) in, This represents the final output light brightness parameter, with a value range of [0, 100], and the unit is percentage; This indicates the default brightness baseline value corresponding to the scene, which is pre-configured by the local expression script, such as the "Welcome Home" scene. 60 is acceptable; This represents the brightness enhancement coefficient related to the scene's risk level, used to improve visual appeal when the risk level is high. Low risk corresponds to 1.0, medium risk to 1.15, and high risk to 1.3. This represents the user preference weight coefficient, with values ​​ranging from [0,1], and a maximum value of 0.4. This represents the user's preset brightness preference value in the application. Continuing with the aforementioned homecoming scenario, let's assume the risk level is medium risk (…). =1.15), user preference value =80, substituting into formula (4) yields =60×1.15+0.4×(80-60)=69+8=77, meaning the final output light brightness parameter is 77%. The method for determining the voice broadcast content and sound effect playback parameters is similar to the above, both based on the target interaction scenario, scenario risk level, and user preference settings. Pre-configured content is matched from the local expression script and the intensity is adjusted in combination with the risk level, which will not be elaborated here.

[0053] When the action set of the abstract scene command indicates that a real vehicle or home device needs to perform a corresponding action, the cloud collaboration layer determines the external device control object based on the command and generates the corresponding control instruction. This instruction is sent to the real vehicle and / or home device through a method associated with the command identifier, and the layer receives the execution result data returned by the real vehicle and / or home device. This data is used to represent at least one of the following situations: success, failure, offline, insufficient permissions, timeout, or lack of support. Taking the simultaneous unlocking of the vehicle and turning on the living room lights after the user arrives home as an example, if the target set includes two external devices, the real vehicle and the living room lights, the cloud collaboration layer obtains the sub-execution results corresponding to each device and aggregates them according to the method shown in formula (6).

[0054] Formula (6) in, This indicates the success rate of this command execution. This indicates the number of external devices that returned a success status. This indicates the total number of external devices associated with this command. When =1, the execution status is determined to be all successful; 0 < A value less than 1 indicates partial execution success; A value of 0 indicates that all operations failed. For example, if a real vehicle unlocks successfully but the living room lights fail to activate due to being offline, then... =1、 =2, substituting into formula (6) gives =0.5, indicating partial execution success. The cloud-based collaboration layer determines the feedback status and generates feedback instructions based on this: A first feedback instruction is generated when all executions are successful, including a confirmation light effect and voice confirmation; a second feedback instruction is generated when some executions are successful, including a partial prompt light effect and a voice description of the execution object's status (e.g., "Vehicle unlocked, living room lights temporarily offline, please try again later"); a third feedback instruction is generated when all executions fail, the device is offline, permissions are insufficient, timeout occurs, or the system is not supported, including an error prompt light effect and a voice announcement of the failure reason. The intelligent interactive vehicle model terminal outputs corresponding multimodal feedback based on this, allowing users to accurately understand the actual execution results of external devices and avoid mistakenly believing that the command itself has been issued and executed.

[0055] In another specific application scenario, when a real vehicle detects a collision warning or abnormal tire pressure and reports the corresponding status data, the cloud collaboration layer, after consistency verification and scenario matching, classifies the scenario as a safety alarm scenario, and the corresponding scenario risk level is usually directly determined to be high-risk. At this time, before generating abstract scenario commands, the cloud collaboration layer needs to verify the validity of user identity information, vehicle binding relationship information, and interactive object device certificates, and requires obtaining the user confirmation status corresponding to multi-factor confirmation or secondary confirmation. If the intelligent interactive vehicle model terminal is offline, the scenario execution conditions are determined to be unmet according to the aforementioned rules. In this case, the cloud collaboration layer retains the cacheable low-risk level expression configuration and cancels or marks the real-time abstract scenario commands generated during this period as expired. After the intelligent interactive vehicle model terminal returns to online, the status synchronization operation is performed first, instead of directly replaying the marked expired commands, to avoid outputting delayed or distorted alarm prompts to the user. If the version of the expression script stored locally on the intelligent interactive vehicle model terminal does not match the version of the script carried by the abstract scene command, a compatible expression script or basic lighting effect will be used to output the corresponding expression, and the script version upgrade requirement will be reported to the cloud. If the remaining battery power of the intelligent interactive vehicle model terminal is lower than the preset battery threshold, the brightness of the light expression and the volume of the voice broadcast will be reduced, and the output of high-power expression content will be prohibited to ensure that basic alarm expression capabilities can still be maintained under resource-constrained conditions.

[0056] To further enhance the interactive experience and response efficiency, the method provided in this application also includes a process for adaptively optimizing scene matching parameters and multimodal expression parameters based on user interaction behavior, as detailed in the following steps: Figure 5As shown. The cloud collaboration layer continuously collects user interaction data such as confirmation, ignore, cancel and repeat operations for multimodal expression and multimodal feedback. According to the scene identifier corresponding to the target interaction scene, the number of operations and average response time of each type in the preset statistical window are obtained, and it is determined whether the scene belongs to the security alarm scene or the high-risk level scene: if it belongs, the triggering conditions and user confirmation requirements remain unchanged; if it does not belong, the triggering time threshold and multimodal expression parameters are adjusted according to the statistical results. The adjustment method is shown in formula (5).

[0057] Formula (5) in, This represents the trigger time threshold for the scenario before the k-th adjustment, in seconds. This indicates the adjusted trigger time threshold; This represents the learning step size coefficient, used to control the magnitude of a single adjustment. Its unit is the same as T, and it can be taken as... =20 seconds; This represents the preset target confirmation rate, which physically means the proportion of prompts in this scenario that are expected to be effectively confirmed by the user. =0.8; This represents the actual confirmation rate for that scenario within the statistics window. The calculation method is the number of confirmation operations divided by the sum of the number of confirmation, ignore, and cancel operations. Taking a non-security-related proactive alert scenario as an example, let... =15 seconds, =0.8, =20 seconds, =0.55, substituting into formula (5) yields =15 + 20 × (0.8 - 0.55) = 15 + 20 × 0.25 = 20, meaning the trigger time threshold is adjusted from 15 seconds to 20 seconds. This indicates that because the actual confirmation rate is lower than the target value, the system automatically extends the trigger interval to reduce invalid interference; conversely, when the actual confirmation rate is higher than the target value, the trigger time threshold is shortened accordingly to improve response timeliness. The adjustment methods for parameters such as light brightness, voice broadcast frequency, and prompt expression are similar, all based on statistically obtained operational data adjusted according to preset rules, and will not be elaborated here. After the above adaptive optimization of parameters, the adjusted scene matching parameters and multimodal expression parameters are used for subsequent scene matching and multimodal expression control, forming a complete closed loop of data acquisition, processing analysis, result output, and feedback optimization.

[0058] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A multimodal closed-loop control method for vehicle state mapping of an intelligent interactive vehicle model, characterized in that, include: Acquire status data from real vehicle terminals and intelligent interactive vehicle model terminals, perform consistency verification on the status data, and generate joint status information based on the verified status data; Scene matching is performed based on joint state information, user interaction information, and environmental context information to determine the target interaction scene, and scene risk is determined based on the interaction object, execution action, and associated environmental information corresponding to the target interaction scene. The scenario execution conditions are determined based on the scenario risk and the user confirmation status. When the scenario execution conditions are met, an abstract scenario command corresponding to the target interaction scenario is generated. The abstract scenario command is used to represent the target interaction scenario and its execution action, but is not bound to a specific hardware execution object. Abstract scene commands are sent to the intelligent interactive vehicle model terminal, which then maps the abstract scene commands into corresponding multimodal expressions and outputs them based on its own state and expressive capabilities.

2. The vehicle state mapping multimodal closed-loop control method according to claim 1, characterized in that, The abstract scenario command includes at least a scenario identifier, a command identifier, a sequence identifier, a target set, an action set, a scenario risk, a priority, a validity period, and a script version requirement. Specifically, the scenario identifier identifies the business scenario type corresponding to the target interaction scenario; the command identifier identifies the current abstract scenario command; the sequence identifier represents the execution order of the current abstract scenario command in the corresponding command sequence; the target set indicates one or more execution objects; the action set describes the behavior to be executed using abstract action types decoupled from specific hardware devices; the scenario risk represents the risk level of the target interaction scenario; the priority determines the execution order or preemption relationship between concurrent abstract scenario commands; the validity period limits the allowed execution time range of the abstract scenario command; and the script version requirement limits the compatible version range of the local expression scripts on which the abstract scenario command depends.

3. The vehicle state mapping multimodal closed-loop control method according to claim 2, characterized in that, The method for mapping abstract scene commands to their corresponding multimodal representations and outputting them is as follows: Analyze the target interaction scenario, execution action, and scenario risk level corresponding to the abstract scenario command; Acquire the status information and expressive capability information of the intelligent interactive vehicle model terminal. The status information and expressive capability information include at least one of the following: vehicle model identification, vehicle model capability set, local expressive script version, user preference settings, current lighting expressive status, and current voice output volume. Based on the target interaction scenario, the executed action, the scenario risk level, the status information and the expressive ability information, the local expression script is matched to determine the multimodal expression parameters; Expression control commands are generated based on multimodal expression parameters, and the intelligent interactive vehicle model terminal is controlled to output the corresponding multimodal expressions.

4. The vehicle state mapping multimodal closed-loop control method according to claim 1, characterized in that, The method for determining scenario risks is as follows: Obtain the interaction object type, execution action, action consequence level, and associated environment information corresponding to the target interaction scenario, and obtain the user mode information and corresponding user permission level of the user who initiated the target interaction scenario; Based on the type of interactive object, the level of action consequences, the associated environmental information, and the user's permission level, the initial risk level is determined according to the preset risk assessment rules. Adjust the initial risk level based on user pattern information to determine the scenario risk level corresponding to the target interaction scenario; When the user mode information corresponds to the guest mode and the execution action involves vehicle control or home device control, the initial risk level is increased. When the user mode information corresponds to the child mode, determine whether the action to be performed belongs to the preset restricted actions; When an action falls under a preset restriction, the generation of the corresponding abstract scene command is prohibited. When an action does not fall under the preset restrictions, the scenario risk level is determined to be low risk.

5. The vehicle state mapping multimodal closed-loop control method according to claim 1, characterized in that, The method for determining scenario execution conditions based on scenario risks and user confirmation status is as follows: Based on the scenario risks corresponding to the target interaction scenario, determine the corresponding execution condition judgment rules; Based on the execution condition determination rules, obtain the validity of the joint state information, the online status of the intelligent interactive vehicle model terminal, and the user confirmation status; When the intelligent interactive vehicle model terminal is online, the corresponding scenario execution conditions are determined based on the scenario risks: When the scenario risk level is low, the scenario execution conditions are determined to be met when the joint state information is valid. When the scenario risk level is medium risk, the scenario execution condition is determined to be met when the user confirmation status includes at least one of voice confirmation, touch confirmation and application confirmation. When the scenario risk level is high, the validity of user identity information, vehicle binding relationship information and interactive object device certificate is verified. When the verification result meets the preset conditions and the user confirmation status meets the multi-factor confirmation conditions or secondary confirmation conditions, the scenario execution conditions are determined to be met. When the intelligent interactive vehicle model terminal is offline, the scene execution conditions are determined to be unmet.

6. The vehicle state mapping multimodal closed-loop control method according to claim 1, characterized in that, The method for obtaining the user's confirmation status is as follows: Based on the scenario execution conditions corresponding to the target interaction scenario, determine whether a user confirmation requirement exists; When a user confirmation request is required, a confirmation prompt message corresponding to the target interaction scenario is generated, and the intelligent interactive vehicle model terminal is controlled to output the confirmation prompt message. Within a preset waiting time, obtain the confirmation interaction information entered by the user in response to the confirmation prompt; The user's confirmation status is determined based on the confirmation interaction information, and the user's confirmation status is matched with the preset confirmation conditions; When the user's confirmation status meets the preset confirmation conditions, it is determined that the user's confirmation is valid, and it is determined that the scenario execution conditions are met. If a user confirmation status that meets the preset confirmation conditions is not obtained within the preset waiting time, the user confirmation is determined to be invalid, the scenario execution conditions are determined to be unmet, and the corresponding confirmation failure event is recorded.

7. The vehicle state mapping multimodal closed-loop control method according to claim 1, characterized in that, The method for scene matching based on joint state information, user interaction information, and environmental context information is as follows: When user interaction information is received or a preset change in the joint state information is detected, the scene matching process is triggered. Obtain current environmental context information, and generate scene matching input information based on joint state information, user interaction information, and environmental context information; The scene matching input information is matched with multiple scene rules in a preset scene rule base to obtain candidate interaction scenes; wherein, the scene rules include at least triggering conditions, necessary preconditions, prohibition conditions and corresponding execution actions; Determine whether the triggering conditions and necessary preconditions corresponding to each candidate interaction scenario are met, and whether any corresponding prohibition conditions are triggered. When there is a candidate interaction scenario that meets the triggering condition and the necessary preconditions but does not trigger the prohibition condition, the candidate interaction scenario is determined as the target interaction scenario. Specifically: when there are multiple valid candidate scenarios, the multiple valid candidate scenarios are sorted according to the preset scenario category priority, and the valid candidate scenario with the highest priority is determined as the target interaction scenario; when there is only one valid candidate scenario, the valid candidate scenario is determined as the target interaction scenario.

8. The vehicle state mapping multimodal closed-loop control method according to claim 1, characterized in that, The method for generating joint state information is as follows: Update the verified vehicle status data to the vehicle equipment status record maintained in the cloud, and update the most recent valid timestamp corresponding to the vehicle status data. Update the verified model car status data to the model car device status record maintained in the cloud; When the preset scenario matching trigger conditions are met, the vehicle equipment status record and the vehicle model equipment status record corresponding to the current moment are obtained; Based on the vehicle equipment status records and the model vehicle equipment status records, a joint state fusion is performed to generate a joint state snapshot, and the joint state snapshot is determined as the joint state information.

9. The vehicle state mapping multimodal closed-loop control method according to claim 1, characterized in that, Also includes: When an abstract scene command is associated with the control of an external device, the corresponding external device is controlled to execute according to the abstract scene command, and the corresponding execution result is obtained. The feedback status is determined based on the execution result, and a feedback instruction is generated based on the feedback status to control the intelligent interactive vehicle model terminal to output the corresponding multimodal feedback. Collect user interaction behaviors related to multimodal expression and multimodal feedback, and adjust scene matching parameters and multimodal expression parameters based on the interaction behaviors for subsequent scene matching and multimodal expression control.

10. The vehicle state mapping multimodal closed-loop control method according to claim 1, characterized in that, The feedback status is determined based on the execution result, and a feedback command is generated based on the feedback status to control the intelligent interactive vehicle model terminal to output the corresponding multimodal feedback, including the following steps: Obtain the execution results or overall execution results corresponding to external devices, and determine the corresponding execution status; The feedback status and corresponding feedback instructions are determined based on the execution status, where: When the execution status is "all executed successfully", the feedback status is determined to be "successful", and a first feedback instruction including confirmation lighting effect and confirmation voice is generated. When the execution status is partially successful, the feedback status is determined to be partially successful, and a second feedback instruction including local prompt light effects and voice description of the execution object status is generated; When the execution status is at least one of the following: all executions failed, device offline, insufficient permissions, execution timeout, or not supported, the feedback status is determined to be a failure status, and a third feedback instruction including error prompt light effects and voice broadcast of the failure reason is generated. Based on the first feedback instruction, the second feedback instruction, or the third feedback instruction, the intelligent interactive vehicle model terminal is controlled to output corresponding multimodal feedback.