Vehicle-mounted emotional robot multi-modal interaction control method and vehicle-mounted emotional robot

Abstract

The application relates to the technical field of vehicle-mounted emotional robots, in particular to a vehicle-mounted emotional robot multi-modal interaction control method and a vehicle-mounted emotional robot, which comprises the following steps: acquiring monitoring indexes, wherein the monitoring indexes comprise at least one of time, weather type, holiday type, standby state, system state and vehicle driving state; identifying at least one target event based on the monitoring indexes; performing priority arbitration and conflict processing on the at least one target event according to preset priority judgment rules and arbitration rules, generating corresponding expression control instructions; calling an expression engine to render corresponding dynamic expressions and sound effects according to the expression control instructions, and performing visual output through a display module. Therefore, multiple source monitoring indexes can be effectively fused, expression rendering frequent jumps can be avoided through priority arbitration and conflict processing, driving safety can be improved, and a coherent emotional interaction experience can be provided.

Description

Technical Field

[0001] This application relates to the field of vehicle-mounted emotion robot technology, and in particular to a multimodal interactive control method for vehicle-mounted emotion robots and a vehicle-mounted emotion robot. Background Technology

[0002] With the rapid development of smart cockpit technology, in-vehicle emotion robots are gradually becoming an important carrier for in-vehicle interaction and emotional experience. Most existing in-vehicle interaction devices can only respond to a single trigger source and cannot simultaneously accommodate multi-dimensional information such as time, weather, holidays, system status, and vehicle driving status. Their interaction methods are limited and their emotional expression is insufficient.

[0003] When multiple events are triggered simultaneously, existing technologies lack a unified priority determination and conflict arbitration mechanism, which can easily lead to problems such as frequent switching of expressions, display confusion, and conflicting dynamic effects. This not only affects the continuity of interaction but may also interfere with driving safety, making it difficult to meet the personalized, stable, and compliant needs of in-vehicle interaction. Summary of the Invention

[0004] This application provides a multimodal interaction control method for an in-vehicle emotion robot and an in-vehicle emotion robot, which realizes intelligent recognition of multiple events and orderly arbitration of conflicts, thereby improving the stability, personalization and emotional experience of in-vehicle interaction.

[0005] To achieve the above objectives, this application adopts the following technical solution: Firstly, a multimodal interaction control method for an in-vehicle emotion robot is provided, including: Acquire monitoring indicators, which include at least one of time, weather type, holiday type, standby status, system status, and vehicle driving status; Based on the monitoring indicators, identify at least one target event; Based on preset priority determination rules and arbitration rules, priority arbitration and conflict handling are performed on the at least one target event to generate corresponding facial expression control instructions; The expression engine is invoked to render corresponding dynamic expressions and sound effects according to the expression control instructions, and the results are visualized through the display module.

[0006] Secondly, a vehicle-mounted emotion robot is provided, comprising: The acquisition module is used to acquire monitoring indicators, which include at least one of time, weather type, holiday type, standby status, system status and vehicle driving status; The identification module is used to identify at least one target event based on the monitoring indicators; The processing module is used to perform priority arbitration and conflict handling on the at least one target event according to preset priority determination rules and arbitration rules, and generate corresponding facial expression control instructions. The output module is used to call the expression engine to render corresponding dynamic expressions and sound effects according to the expression control instructions, and to output them visually through the display module.

[0007] Thirdly, embodiments of this application provide an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the multimodal interactive control method for an in-vehicle emotion robot as described in any one of the first aspects above.

[0008] Fourthly, embodiments of this application provide a computer-readable storage medium storing a computer program that, when executed by a processor, implements the multimodal interactive control method for an in-vehicle emotion robot as described in any one of the first aspects above.

[0009] Fifthly, embodiments of this application provide a computer program product that, when run on an electronic device, causes the electronic device to execute the multimodal interactive control method for an in-vehicle emotion robot described in any of the first aspects above.

[0010] It is understood that the beneficial effects of the second to fifth aspects mentioned above can be found in the relevant descriptions in the first aspect mentioned above, and will not be repeated here.

[0011] The embodiments of this application have the following beneficial effects: First, monitoring indicators are acquired, including at least one of time, weather type, holiday type, standby status, system status, and vehicle driving status. Then, based on these indicators, at least one target event is identified. Next, according to preset priority determination and arbitration rules, priority arbitration and conflict resolution are performed on the at least one target event, generating corresponding facial expression control commands. Finally, the facial expression engine is invoked to render the corresponding dynamic facial expressions and sound effects based on the facial expression control commands, and the results are visualized through the display module. Thus, the integration of multimodal monitoring indicators enables the device to comprehensively perceive the driving environment, the priority arbitration mechanism ensures orderly response when multiple events occur concurrently, and the adaptive facial expression rendering strategy achieves coherent output of emotional interaction.

[0012] The above description is only an overview of the technical solution of this application. In order to better understand the technical means of this application and to implement it in accordance with the contents of the specification, and to make the above and other objects, features and advantages of this application more obvious and understandable, the following are specific embodiments of this application. Attached Figure Description

[0013] Various other advantages and benefits will become apparent to those skilled in the art upon reading the detailed description of the preferred embodiments below. The accompanying drawings are for illustrative purposes only and are not intended to limit the scope of this application. Furthermore, the same reference numerals denote the same parts throughout the drawings. In the drawings: Figure 1 This is a flowchart illustrating a multimodal interaction control method for an in-vehicle emotion robot provided in an embodiment of this application; Figure 2 This is a structural block diagram of the vehicle-mounted emotion robot provided in the embodiments of this application; Figure 3 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0014] The embodiments of the technical solutions of this application will now be described in detail with reference to the accompanying drawings. The following embodiments are only used to more clearly illustrate the technical solutions of this application, and are therefore merely examples and should not be used to limit the scope of protection of this application. When the following description relates to the accompanying drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. Various changes, modifications, and equivalents of the methods, apparatus, and / or systems described herein will become apparent upon understanding this disclosure. For example, the order of operations described herein is merely illustrative and is not limited to those orders set forth herein, but can be changed as will become apparent upon understanding this disclosure, except for operations that must be performed in a specific order. Furthermore, for clarity and conciseness, descriptions of features known in the art may be omitted.

[0015] The embodiments described in the following examples of this disclosure are not representative of all embodiments consistent with this disclosure. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this disclosure as detailed in the appended claims.

[0016] It should be noted that in the field of in-vehicle interactive devices, existing technical solutions only execute response operations for a single trigger condition, and cannot achieve concurrent processing of multi-source heterogeneous event data such as time, weather, holidays, system status, and vehicle driving status. When multiple events are triggered simultaneously and have priority conflicts, the system lacks a unified arbitration logic, leading to inconsistent expression rendering command switching behavior and displayed content, which in turn causes user attention distraction and driving safety risks. This problem stems from the discrete design of the event processing mechanism, resulting in a lack of temporal coordination and logical coherence in multimodal interaction output, reducing the reliability of human-computer interaction. For example, during the morning rush hour, weekday indicators, rainstorm weather warnings, traffic congestion status prompts from the navigation system, and low battery warnings are detected simultaneously. At this time, weather-related events triggering rainy day care expressions, time-related events triggering commuting time reminders, sensor-related events triggering battery warnings, and system-related events triggering navigation status are all processed simultaneously. Because the priorities of each event are not clearly defined, the expression engine switches between rainy day expressions and battery warning icons, causing the content displayed on the central control screen to flicker. Users cannot clearly obtain key information, and the superimposed output of sound prompts increases the degree of interference in the cabin environment.

[0017] In response, this application proposes a multimodal interaction control method for an in-vehicle emotion robot and an in-vehicle emotion robot.

[0018] It should be noted that the executing entity of the multimodal interaction control method for the vehicle-mounted emotion robot in this embodiment can be the vehicle-mounted emotion robot, and this application embodiment does not limit this.

[0019] See Figure 1 This is a flowchart illustrating the multimodal interaction control method for an in-vehicle emotion robot provided in the first embodiment of this application. Figure 1 As shown, the multimodal interaction control method for an in-vehicle emotion robot may include the following steps: Step 101: Obtain monitoring indicators, which include at least one of the following: time, weather type, holiday type, standby status, system status, and vehicle driving status.

[0020] Monitoring indicators refer to various data used to assess the current vehicle environment, system status, or user context. These data can be discrete or continuous, such as current time information, external weather conditions, whether it is a specific holiday, the operating status of the in-vehicle emotion robot itself (e.g., whether it is in standby mode), the overall operating status of the in-vehicle system, and the vehicle's current driving mode or status.

[0021] In this embodiment, the vehicle driving state is calculated locally by collecting raw data from the Inertial Measurement Unit (IMU) and the Global Positioning System (GPS). The vehicle driving state specifically includes seven states: vehicle start-up, acceleration, deceleration, turning, bumping, idling, and cruising. By collecting multi-dimensional monitoring indicators, it can cover all interactive triggering conditions such as driving, parking, standby, holidays, and special weather, thus improving the scene adaptability.

[0022] For example, the current time can be obtained through the built-in clock module of the vehicle system. The current weather type, such as sunny, rainy, or cloudy, can be obtained by connecting to external weather services or onboard environmental sensors. Whether it is a holiday can be determined by querying a preset calendar database or network service. The standby status can be obtained by reading the power management status of the onboard emotion robot. The system status can be obtained by monitoring the vehicle operating system's logs or specific sensor data, such as whether the system is updating or experiencing an anomaly.

[0023] The following is an illustrative explanation of sensor status recognition, which includes seven status categories: start-up, acceleration, deceleration, turning, bumping, idling, and cruising.

[0024] Acceleration: The average X-axis acceleration is >0.8 m / s² and lasts for ≥0.6 s; Deceleration: X-axis acceleration < 0.8 m / s² and lasting ≥ 0.6 s; Turning: The absolute value of the Z-axis angular velocity is >15° / s and lasts for ≥0.5s; Bumping: Z-axis acceleration variance > T_bump and peak interval < 0.4s; Idle speed: speed ≈ 0 and |a| < 0.2 m / s² and lasts for ≥ 8 seconds; Cruise: The speed is stable within ±5km / h and the average value of |a| is <0.2m / s² for ≥20s; Start-up: from rest to an increase in velocity or a sudden change in acceleration.

[0025] Step 102: Identify at least one target event based on monitoring indicators.

[0026] Among them, the target event refers to the specific situation or triggering condition identified based on monitoring indicators that requires the in-vehicle emotion robot to respond.

[0027] For example, a specific time point, a weather change, or a system warning can all be identified as a target event. The identification of one or more target events forms the basis for the emotion robot's subsequent decision-making.

[0028] The target events include system events, sensor events, holiday events, weather events, time-related events, and standby events. System events include charging, abnormal temperature, low battery, and system upgrades. Sensor events include changes in driving status such as acceleration, deceleration, turning, and bumps. Holiday events include statutory holidays, lunar holidays, and user-defined anniversaries. Weather events include sunny, rainy, snowy, foggy, high temperature, and low temperature. Time-related events include the hour, commuting hours, and nighttime. Standby events include inactive sleep mode and breathing display.

[0029] Specifically, based on the acquired monitoring indicators, at least one target event is further identified. For example, when the monitoring time is during the early morning commute, a target event of "commuting begins" can be identified. When the weather type is "rainy," a target event of "rainy day alert" can be identified. When the system status is "low battery," a target event of "insufficient battery" can be identified. This identification process can be implemented through a preset set of rules or simple conditional judgments. For example, a threshold can be set, and when a certain monitoring indicator reaches the threshold, a specific target event is triggered. This identification process aims to transform the raw monitoring data into events with clear semantics for subsequent processing.

[0030] Step 103: Based on the preset priority determination rules and arbitration rules, priority arbitration and conflict handling are performed on at least one target event to generate corresponding facial expression control instructions.

[0031] The preset priority determination rule is as follows: from high to low priority, the events are system-related events, sensor-related events, holiday-related events, weather-related events, time-related events, and standby events.

[0032] The priority determination rule and arbitration rule are logical frameworks used to determine which event should be responded to first and how to resolve conflicts between events when multiple target events occur simultaneously. The priority determination rule defines the order of importance of different types of events, while the arbitration rule stipulates how to select and coordinate when multiple events simultaneously meet priority conditions to ensure that the emotional robot's behavioral output is consistent and as expected.

[0033] The facial expression control instructions are generated after priority arbitration and conflict resolution. These instructions contain detailed information about what dynamic facial expressions the in-vehicle emotion robot should display and what sound effects should be played.

[0034] Optionally, when multiple target events are triggered simultaneously, the target events to be executed are selected from high to low according to preset priorities. The selected target events are then used to resolve conflicts using arbitration rules to obtain the target events determined by arbitration. The arbitration rules include high-priority preemption, same-priority sticking, throttling of similar events, and overload protection. Based on the target events determined by arbitration, the corresponding facial expressions and sound effects are matched to generate facial expression control commands.

[0035] Among them, high-priority preemption means that high-priority events preempt low-priority events. Same-priority sticking means that events of the same priority stick for a short period of time (default 1.5s) to avoid flickering. Same-event throttling means that the same event will be triggered at most once within 10 seconds (configurable). Overload protection means that if it is triggered ≥5 times within 5 seconds, it will enter noise reduction mode (only the highest priority event is passed through).

[0036] Example: At exactly 12:00 and all users simultaneously connect to charge: Events generated: Time-based (on the hour), System-based (charging) Arbitration: System Category > Time Category → First display the "Charging Emoji" for 2.5 seconds → If it is still in the 12:00 window and the "Hourly Reminder" has not expired (valid for 10 seconds by default), then play the "12:00 Hourly" emoji for 1.5 seconds; if it has expired, discard the time category.

[0037] Optionally, in response to the user's QR code configuration command, the device can be bound and configured via Bluetooth and Wi-Fi dual links. The lightweight configuration includes theme switching, reminder volume, silent period, energy-saving threshold and geographical permissions.

[0038] The user QR code configuration command refers to the instruction that a user initiates the device configuration process by scanning the QR code displayed on the in-vehicle emotion robot's display module or on the product packaging. This command can be recognized and parsed by a specific application on a mobile terminal (such as a smartphone), thereby establishing a communication connection with the in-vehicle emotion robot for subsequent device binding and parameter configuration.

[0039] The dual-link Bluetooth and Wi-Fi technology means that the in-vehicle emotion robot simultaneously supports both Bluetooth and Wi-Fi wireless communication technologies and can coordinate or switch between these two links to complete specific tasks. Bluetooth links are typically used for short-range, low-power data transmission, such as initial handshakes, device discovery, and the transmission of small amounts of configuration data. Wi-Fi links are suitable for high-speed, high-capacity data transmission, such as firmware updates, large-scale configuration data synchronization, or remote control. By combining the advantages of these two links, the stability and efficiency of the device binding and configuration process can be ensured.

[0040] Device binding refers to the process of associating an in-vehicle emotion robot with a specific user account or mobile terminal. This process typically involves authentication and security key exchange to ensure that only authorized users can control and configure the device. Device binding is fundamental to enabling personalized settings and data synchronization; it connects the physical device to the user's digital identity, allowing users to remotely manage and customize the device.

[0041] Lightweight configuration refers to the ability of users to perform a series of core and commonly used personalized settings on the in-vehicle emotion robot during or after device pairing, aiming to provide sufficient customization freedom without increasing the user's burden. Theme switching allows users to change the interface style, color scheme, font, or emoji style displayed on the in-vehicle emotion robot's display module according to personal preferences or scenario needs, thereby enhancing the aesthetics and personalization of the user interface.

[0042] The reminder volume refers to the user's ability to adjust the volume of the in-car emotion robot's prompts, alarms, or interactive sounds. This allows users to adjust the volume to a suitable level based on the noise level in the car or their individual sensitivity to sound. The silent period refers to the user's ability to preset the in-car emotion robot to automatically enter silent mode and stop playing any sound effects during specific time periods.

[0043] The energy-saving threshold refers to the parameters that users can set for the in-vehicle emotion robot to enter a low-power mode under specific conditions. For example, when the vehicle is turned off for a period of time, or when the battery level is lower than a certain preset percentage, the device can automatically reduce screen brightness, decrease the frequency of dynamic expression rendering, or shut down some non-core functions to extend the device's battery life.

[0044] Geographic permissions refer to the ability of users to authorize or restrict the in-vehicle emotion robot's access to or use of its geographic location information. For example, users can choose to allow the device to provide location-based services within a specific area, or to trigger specific facial expressions or sound effects when the user enters a specific area.

[0045] Step 104: Call the expression engine to render the corresponding dynamic expressions and sound effects according to the expression control instructions, and output them visually through the display module.

[0046] The facial expression engine is responsible for receiving facial expression control commands and rendering corresponding dynamic facial expressions and sound effects based on those commands. This engine can include components such as an facial expression library, rendering algorithms, and a sound effects player.

[0047] Dynamic facial expressions refer to the continuously changing or animated graphic expressions displayed on the display module of the in-vehicle emotion robot, simulating human emotional changes such as happiness, surprise, and confusion to enhance the vividness of the interaction. Sound effects refer to the audio effects played in sync with the dynamic facial expressions to assist in expressing emotions or providing prompts.

[0048] The display module is a hardware component used by the in-vehicle emotion robot to visually output dynamic facial expressions. Visual output refers to presenting dynamic facial expressions to the user in a graphical way through the display module, allowing the user to intuitively perceive the "emotional" feedback of the emotion robot.

[0049] For example, if the facial expression control command instructs the display of a "happy" animated emoticon and the playback of a "cheerful" sound effect, the facial expression engine will extract the corresponding animation sequence and audio file from its internal material library and render them in real time. The rendered animated emoticon will then be presented to the user through the display module of the in-vehicle emotion robot, while the sound effect will be played through the built-in speaker.

[0050] The following example will provide a more detailed explanation of the above technical solution: Suppose user A is driving a vehicle in a city. An in-vehicle emotion robot continuously acquires various monitoring indicators. For example, the in-vehicle system detects the current time as 8:00 AM (peak commuting time), external sensors detect the weather as "light rain," and the in-vehicle system reports itself as "normal operation" and the vehicle's driving status as "in motion." Based on these monitoring indicators, the in-vehicle emotion robot identifies multiple target events. For example, based on the time 8:00 AM, it identifies the "commuting begins" event; based on the weather "light rain," it identifies the "rainy day reminder" event. In this case, if a traditional single-response mechanism is used, the emotion robot might frequently switch expressions, for example, first displaying a "commuting" expression and then immediately switching to a "rainy day" expression, causing display confusion and potentially distracting the driver. However, in the method of this embodiment, when both the "commuting begins" and "rainy day reminder" target events are identified simultaneously, the system processes them according to preset priority determination rules and arbitration rules. For example, the priority of "weather-related events" can be preset to be higher than that of "time-related events." Therefore, the "rainy day reminder" event is determined to have a higher priority. Regarding the arbitration rules, if the rule of "high-priority events directly override low-priority events" is adopted, then the "rainy day reminder" event will gain the right of execution. Therefore, the system generates corresponding expression control instructions, such as instructing the expression engine to render a dynamic expression with "raindrop" elements or a "reminder of slippery roads" meaning, and play a soft "rain sound" or "cautionary sound effect." Upon receiving the instruction, the expression engine immediately renders the expression and visualizes it through the display module of the in-vehicle emotion robot, while simultaneously playing the corresponding sound effect.

[0051] Optionally, user profiles corresponding to vehicle drivers can also be obtained. These user profiles include at least commuter car owners, family car owners, and professional drivers. Based on these user profiles, the display mode of the in-vehicle emotion robot can be adjusted.

[0052] The proposed solution introduces a user profiling mechanism, enabling the interactive control of the in-vehicle emotion robot to move beyond passive responses to environmental and system events and proactively adapt to the personalized needs of different users.

[0053] For example, commuter drivers (18-35 years old): prefer personalized, anime-themed, and themed environments; they also want to improve their mood during rush hour. Family drivers (28-45 years old): desire a friendly in-car atmosphere that their children enjoy; they also want a festive atmosphere and reminders. Professional drivers (ride-hailing / taxi drivers): require gentle reminders and a fatigue-preventing environment for long drives; they prefer minimal distractions.

[0054] Specifically, after the system acquires monitoring indicators and identifies the target event, and before or after generating preliminary expression control commands according to preset rules, it first acquires the user profile of the current vehicle driver. This user profile is used to fine-tune the display mode of the emotion robot. For example, when a target event (such as a "holiday event") is identified that requires triggering a specific dynamic expression, the system will determine the specific presentation method of the dynamic expression based on the current user's profile type (e.g., whether it is a "commuter driver" or a "family driver"). For "commuter drivers," the system may choose to present holiday greetings in a simpler, less disruptive way, such as displaying only a static icon or a short text prompt; while for "family drivers," it may render more vivid and interactive holiday animations and sound effects. This user profile-based adjustment of the display mode allows the emotion robot's output to better match the user's driving scenario and psychological expectations, thereby providing emotional interaction while avoiding unnecessary interference and significantly improving user satisfaction and acceptance of the emotion robot service.

[0055] The emoji engine is equipped with a high-temperature downgrading protection strategy: When the device temperature exceeds the first preset threshold, the rendering frame rate is gradually reduced. When the device temperature exceeds the second preset threshold, dynamic facial expression rendering is stopped, and only the static prompt interface is retained. The second preset threshold is greater than the first preset threshold.

[0056] Among them, the high-temperature downgrade protection strategy is a thermal management mechanism designed to monitor the temperature of the equipment during operation and reduce power consumption and heat generation by adjusting the equipment performance when the temperature reaches a preset safety threshold, thereby preventing the equipment from overheating.

[0057] Among them, device temperature refers to the real-time operating temperature of key components inside the vehicle-mounted emotion robot, which is usually monitored in real time through built-in temperature sensors.

[0058] The first preset threshold is a critical point for device temperature. When the device temperature reaches or exceeds this threshold for the first time, the system will trigger the first stage of downgrading protection measures. Gradually reducing the rendering frame rate means that after the device temperature reaches the first preset threshold, the expression engine gradually reduces the number of image frames rendered and displayed per second. This can be achieved by adjusting the waiting time of the rendering loop or by skipping the rendering of some frames to reduce the amount of computation.

[0059] The second preset threshold is a higher-level critical point for device temperature, exceeding the first preset threshold. When the device temperature reaches or exceeds this threshold, it indicates that the device is under high heat load. Stopping dynamic facial expression rendering and retaining only a static prompt interface means that after the device temperature reaches the second preset threshold, the facial expression engine completely stops generating and displaying continuously changing dynamic images.

[0060] At this point, the display module will switch to displaying only preset static images or text information. The setting that the second preset threshold is greater than the first preset threshold reflects a tiered protection strategy. The first preset threshold is used to gently reduce performance to preventatively control temperature, while the second preset threshold is used to take more aggressive measures when the temperature rises further to avoid equipment damage.

[0061] For example, no interaction for 5 minutes / ACC OFF → sleep (screen dark, rendering 0 / 5FPS, sampling rate reduced to ≤10Hz); Equipment temperature > 45℃ → Rendering speed downgraded; Equipment temperature >50℃ → Stop animation, only static display will be displayed; Device temperature >55℃ → Only display the overheat icon and prompt for heat dissipation.

[0062] Optionally, the system can receive user-uploaded emoticons and theme packs, review the content of the uploaded emoticons and theme packs, configure pricing and register copyrights for approved target materials, and implement DRM copyright protection for the priced target materials.

[0063] Emoticons and theme packs refer to collections of digital content, including graphics, animations, and sound effects, used in the display module of the in-vehicle emotion robot for visualization output. Emoticons typically refer to sequences of dynamic or static images expressing specific emotions or situations, often accompanied by sound effects. Theme packs, on the other hand, refer to collections that alter the robot's overall interface style, background, icons, and other visual elements.

[0064] Specifically, data can be uploaded via a mobile application that accompanies the in-vehicle emotion robot, a specific web-based management platform, or imported via a connection between the in-vehicle system and an external storage device.

[0065] Furthermore, user-uploaded emojis and theme packs will be reviewed to ensure they comply with relevant laws and regulations, platform policies, content guidelines, and security standards. For example, artificial intelligence technology can be used for automated review, employing algorithms such as image recognition, text analysis, and audio analysis to quickly screen for inappropriate content.

[0066] Pricing configuration refers to setting a price for the use or purchase of approved emoji packs and theme packs. This can include various pricing models such as free provision, one-time purchase, and subscription.

[0067] Copyright registration refers to recording and registering the intellectual property information of approved materials to clarify the creator and ownership of the materials.

[0068] Specifically, pricing can be set by the content uploader and reviewed and confirmed by the platform, or the platform can recommend a price based on factors such as content quality and popularity. Copyright registration can be integrated with national or regional copyright management agencies to achieve an automated or semi-automated registration process.

[0069] DRM (Digital Rights Management) protection refers to implementing digital rights management technology on priced emoji and theme pack materials to prevent unauthorized copying, distribution, use, or alteration. DRM protection can include various technical means, such as encrypting the material files so that they can only be decrypted and used on authorized devices or applications. Digital watermarking technology can be used to embed invisible copyright information into the materials, and a license management system can be integrated to strictly control the usage rights, validity period, and device binding of the materials.

[0070] Optionally, the unique device identifier of the in-vehicle emotion robot can be bound to the purchased paid materials, and digital signature verification can be performed on the purchased paid materials to verify their legality and integrity. The materials can be authorized and activated in an offline environment using an offline activation code.

[0071] The following is a specific example to illustrate this.

[0072] When a user purchases a holiday-themed package through the accompanying mobile app or in-vehicle system interface of the in-vehicle emotion robot, the system first obtains the unique device identifier of the in-vehicle emotion robot, such as its internally stored serial number. Subsequently, this serial number is stored in the cloud server's database along with the authorization information of the holiday-themed package purchased by the user, thus completing the binding of the in-vehicle emotion robot's unique device identifier with the purchased paid content.

[0073] When a user attempts to apply the holiday theme pack to the in-car mood robot, the system first checks the digital signature of the theme pack file. For example, the theme pack file might contain a metadata file signed by the content provider using their private key. The in-car mood robot's built-in verification module uses a pre-set public key to verify the validity of this signature, thus verifying the legality and integrity of the purchased paid content. If the signature verification fails, it indicates that the theme pack may have been tampered with or is not an official release, and the system will refuse to load it. Furthermore, if the in-car mood robot is located in an area with no network signal, such as an underground parking lot or mountainous region, the user can still use the theme pack through offline activation. At the time of purchase, the system provides the user with an offline activation code, such as a string of alphanumeric characters. The user can manually enter this offline activation code in the in-car mood robot's settings interface. The in-car mood robot's internal authorization module parses the activation code and completes the authorization locally based on the authorization information it contains, thus completing the authorization activation of the content in an offline environment.

[0074] Optionally, a content lifecycle management mechanism can be adopted to clean up local redundant cache using the Least Recently Used (LRU) algorithm.

[0075] Content lifecycle management mechanisms are used to manage the entire process of data or content from creation, storage, and use to final archiving or deletion. The aim is to ensure effective data management at different stages, optimize storage resource utilization, and maintain system performance.

[0076] Optionally, management can be based on preset strategies (e.g., based on content type, importance, or storage time) or dynamically adjusted according to real-time usage. For example, rules can be set to automatically mark specific types of content as ready for cleanup after a certain period, or to manage content in tiers based on access frequency. The Least Recently Used (LRU) algorithm is a commonly used cache eviction strategy. If data has not been used recently, it is unlikely to be used in the future. When cache space is insufficient, the LRU algorithm prioritizes evicting the least recently used data.

[0077] The embodiments of this application have the following beneficial effects: First, monitoring indicators are acquired, including at least one of time, weather type, holiday type, standby status, system status, and vehicle driving status. Then, based on these indicators, at least one target event is identified. Next, according to preset priority determination and arbitration rules, priority arbitration and conflict resolution are performed on the at least one target event, generating corresponding facial expression control commands. Finally, the facial expression engine is invoked to render the corresponding dynamic facial expressions and sound effects based on the facial expression control commands, and the results are visualized through the display module. Thus, the integration of multimodal monitoring indicators enables the device to comprehensively perceive the driving environment, the priority arbitration mechanism ensures orderly response when multiple events occur concurrently, and the adaptive facial expression rendering strategy achieves coherent output of emotional interaction.

[0078] Corresponding to the multimodal interaction control method for the vehicle-mounted emotion robot in the above embodiment, Figure 2 This is a structural block diagram of the vehicle-mounted emotion robot 200 provided in the embodiments of this application.

[0079] The acquisition module 210 is used to acquire monitoring indicators, which include at least one of time, weather type, holiday type, standby status, system status and vehicle driving status. The identification module 220 is used to identify at least one target event based on the monitoring indicators; The processing module 230 is used to perform priority arbitration and conflict handling on the at least one target event according to preset priority determination rules and arbitration rules, and generate corresponding facial expression control instructions. The output module 240 is used to call the expression engine to render the corresponding dynamic expressions and sound effects according to the expression control instructions, and to output them visually through the display module.

[0080] in addition, Figure 2 The in-vehicle emotion robot shown can be a software unit, hardware unit, or a combination of software and hardware built into existing electronic devices. It can also be integrated into the electronic devices as an independent accessory, or exist as an independent electronic device.

[0081] The embodiments of this application have the following beneficial effects: First, monitoring indicators are acquired, including at least one of time, weather type, holiday type, standby status, system status, and vehicle driving status. Then, based on these indicators, at least one target event is identified. Next, according to preset priority determination and arbitration rules, priority arbitration and conflict resolution are performed on the at least one target event, generating corresponding facial expression control commands. Finally, the facial expression engine is invoked to render the corresponding dynamic facial expressions and sound effects based on the facial expression control commands, and the results are visualized through the display module. Thus, the integration of multimodal monitoring indicators enables the device to comprehensively perceive the driving environment, the priority arbitration mechanism ensures orderly response when multiple events occur concurrently, and the adaptive facial expression rendering strategy achieves coherent output of emotional interaction.

[0082] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this application. The specific working process of the units and modules in the above system can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0083] Figure 3 This is a schematic diagram of the structure of the electronic device provided in an embodiment of this application. For example... Figure 3 As shown, the electronic device 5 of this embodiment includes: at least one processor 50 ( Figure 3 (Only one is shown in the diagram) a processor, a memory 51, and a computer program 52 stored in the memory 51 and executable on the at least one processor 50, wherein the processor 50 executes the computer program 52 to implement the steps in any of the above embodiments of the vehicle-mounted emotion robot multimodal interaction control method.

[0084] The electronic device may be a desktop computer, laptop, handheld computer, or cloud server, etc. This electronic device may include, but is not limited to, a processor and memory. Those skilled in the art will understand that... Figure 3 This is merely an example of electronic device 5 and does not constitute a limitation on electronic device 5. It may include more or fewer components than shown in the figure, or combine certain components, or different components. For example, it may also include input / output devices, network access devices, etc.

[0085] The processor 50 may be a central processing unit, or it may be other general-purpose processors, digital signal processors, application-specific integrated circuits, off-the-shelf programmable gate arrays or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor.

[0086] In some embodiments, the memory 51 may be an internal storage unit of the electronic device 5, such as a hard disk or memory of the electronic device 5. In other embodiments, the memory 51 may be an external storage device of the electronic device 5, such as a plug-in hard disk, smart memory card, secure digital card, flash memory card, etc., equipped on the electronic device 5. Further, the memory 51 may include both internal storage units and external storage devices of the electronic device 5. The memory 51 is used to store operating systems, applications, boot loaders, data, and other programs, such as the program code of the computer program. The memory 51 can also be used to temporarily store data that has been output or will be output.

[0087] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, can implement the steps in the above-described method embodiments.

[0088] This application provides a computer program product that, when run on an electronic device, enables the electronic device to implement the steps described in the various method embodiments above.

[0089] If the integrated unit is implemented as a software functional unit and used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments of this application can be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include at least: any entity or device capable of carrying computer program code to a device / electronic device, a recording medium, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical carrier signal, a telecommunication signal, and a software distribution medium. Examples include USB flash drives, portable hard drives, magnetic disks, or optical disks.

[0090] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0091] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0092] In the embodiments provided in this application, it should be understood that the disclosed devices / electronic devices and methods can be implemented in other ways. For example, the device / electronic device embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual couplings or direct couplings or communication connections may be through some interfaces; indirect couplings or communication connections between devices or units may be electrical, mechanical, or other forms.

[0093] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0094] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains; the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the application; the terms “comprising” and “having”, and any variations thereof, in the specification, claims, and foregoing description of the drawings are intended to cover non-exclusive inclusion.

[0095] In the description of the embodiments of this application, technical terms such as "first" and "second" are used only to distinguish different objects and should not be construed as indicating or implying relative importance or implicitly specifying the number, specific order, or primary and secondary relationship of the indicated technical features. In the description of the embodiments of this application, "multiple" means two or more, unless otherwise explicitly defined.

[0096] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0097] In the description of the embodiments in this application, the term "and / or" is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. Additionally, the character " / " in this document generally indicates that the preceding and following related objects have an "or" relationship.

[0098] In the description of the embodiments of this application, the term "multiple" refers to two or more (including two), similarly, "multiple sets" refers to two or more (including two sets), and "multiple pieces" refers to two or more (including two pieces).

[0099] In the description of the embodiments of this application, the technical terms "center," "longitudinal," "lateral," "length," "width," "thickness," "upper," "lower," "front," "rear," "left," "right," "vertical," "horizontal," "top," "bottom," "inner," "outer," "clockwise," "counterclockwise," "axial," "radial," and "circumferential" indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are only for the convenience of describing the embodiments of this application and simplifying the description, and are not intended to indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on the embodiments of this application.

[0100] In the description of the embodiments of this application, unless otherwise expressly specified and limited, technical terms such as "installation," "connection," "joining," and "fixing" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral part; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; they can refer to the internal communication of two components or the interaction between two components. For those skilled in the art, the specific meaning of the above terms in the embodiments of this application can be understood according to the specific circumstances.

[0101] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. A multimodal interactive control method for an in-vehicle emotion robot, characterized in that, include: Acquire monitoring indicators, which include at least one of time, weather type, holiday type, standby status, system status, and vehicle driving status; Based on the monitoring indicators, identify at least one target event; Based on preset priority determination rules and arbitration rules, priority arbitration and conflict handling are performed on the at least one target event to generate corresponding facial expression control instructions; The expression engine is invoked to render corresponding dynamic expressions and sound effects according to the expression control instructions, and the results are visualized through the display module.

2. The method according to claim 1, characterized in that, The target events include system events, sensor events, holiday events, weather events, time events, and standby events; the preset priority determination rule is as follows: the priority from high to low is system events, sensor events, holiday events, weather events, time events, and standby events.

3. The method according to claim 2, characterized in that, Also includes: Obtain user profiles corresponding to vehicle drivers, and the user profiles include at least commuter car owners, family car owners, and professional drivers; Based on the user profile, the display mode of the in-vehicle emotion robot is adjusted.

4. The method according to claim 1, characterized in that, The facial expression engine is equipped with a high-temperature downgrading protection strategy: When the device temperature exceeds the first preset threshold, the rendering frame rate is gradually reduced. When the device temperature exceeds the second preset threshold, dynamic facial expression rendering is stopped, and only the static prompt interface is retained. The second preset threshold is greater than the first preset threshold.

5. The method according to claim 1, characterized in that, Also includes: In response to the user's QR code configuration command, the device is bound and lightweight configured via Bluetooth and Wi-Fi dual links. The lightweight configuration includes theme switching, reminder volume, silent period, energy saving threshold and geographical permissions.

6. The method according to claim 1, characterized in that, Also includes: Receive user-uploaded emoji packs and theme packs; Content review will be conducted on the uploaded emoticons and theme packs. Pricing and copyright registration will be carried out on the approved target materials; DRM copyright protection is applied to the target material after pricing is completed.

7. The method according to claim 6, characterized in that, Also includes: Bind the unique device identifier of the in-vehicle emotion robot to the purchased paid materials; Perform digital signature verification on the purchased paid materials to verify their legality and integrity; Activate material authorization in an offline environment using an offline activation code.

8. The method according to claim 1, characterized in that, Also includes: A content lifecycle management mechanism is adopted, and local redundant cache is cleaned up using the Least Recently Used (LRU) algorithm.

9. The method according to claim 1, characterized in that, The step of performing priority arbitration and conflict handling on the at least one target event according to preset priority determination rules and arbitration rules, and generating corresponding facial expression control instructions, includes: When multiple target events are triggered simultaneously, the target events currently being executed are selected from high to low according to a preset priority. The arbitration rules are used to handle conflicts of the currently selected target events to obtain the target events determined by arbitration. The arbitration rules include high-priority preemption, same-priority sticking, throttling of similar events, and overload protection. Based on the target event determined by arbitration, the corresponding facial expression and sound effect are matched to generate facial expression control instructions.

10. A vehicle-mounted emotion-generating robot, characterized in that: include: Acquire monitoring indicators, which include at least one of time, weather type, holiday type, standby status, system status, and vehicle driving status; Based on the monitoring indicators, identify at least one target event; Based on preset priority determination rules and arbitration rules, priority arbitration and conflict handling are performed on the at least one target event to generate corresponding facial expression control instructions; The expression engine is invoked to render corresponding dynamic expressions and sound effects according to the expression control instructions, and the results are visualized through the display module.

Images