Interaction methods, in-vehicle robots, interaction systems, electronic devices and vehicles

By enabling in-vehicle robots to perform various interactive behaviors based on their unlocked permissions and driving status, the in-vehicle interaction methods are enriched, solving the problem of limited interaction methods in existing technologies and improving user experience and driving safety.

CN122633046BActive Publication Date: 2026-11-10BYD CO LTD +1
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202611114765.7
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2026-07-27
Publication Date
2026-11-10
Estimated Expiration
2046-07-27

AI Technical Summary

Technical Problem

Existing in-vehicle interaction methods mainly focus on simple functional information feedback, lacking diversity and interactivity, and are unable to meet the diverse interaction needs of users.

Method used

By using an onboard robot as an interaction node, various interactive behaviors, including actions and control commands, can be executed based on its unlocked permissions, thus enriching the interaction methods. The sensor module detects user interaction operations, and the execution module executes the corresponding target interactive behaviors, which are dynamically adjusted in combination with the permission unlocking mechanism and driving status.

Benefits of technology

It has diversified the in-vehicle interaction methods, enhanced the fun and stickiness of the user experience, taken into account driving safety, and dynamically adjusted the interaction permissions to adapt to different driving scenarios.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN122633046B_ABST
    Figure CN122633046B_ABST
Patent Text Reader

Abstract

This application discloses an interaction method, an in-vehicle robot, an interaction system, an electronic device, and a vehicle, applied to a vehicle's smart cockpit. Upon detecting a preset interaction operation, it can execute a target interaction behavior corresponding to that preset operation, wherein the target interaction behavior is determined at least by the unlocked permissions of the in-vehicle robot. Thus, using the in-vehicle robot as an interaction node, multiple interaction behaviors can be executed within the vehicle's smart cockpit based at least on the robot's unlocked permissions, overcoming the limitations of existing single-feedback screen text / icons and enriching in-vehicle interaction methods.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This application relates to the field of vehicle interaction technology, and in particular to an interaction method, vehicle robot, interaction system, electronic device and vehicle. Background Technology

[0002] Existing in-vehicle interaction methods typically focus on simple, functional information feedback. For example, the interaction method of displaying text and corresponding icons on the vehicle screen needs improvement. Summary of the Invention

[0003] This application provides an interaction method, an in-vehicle robot, an interaction system, a storage medium, a program product, an electronic device, and a vehicle. By using the in-vehicle robot as an interaction node, multiple interactive behaviors can be performed based on at least the unlocked permissions of the in-vehicle robot, thereby breaking through the limitations of the single feedback of existing screen text / icons and enriching the in-vehicle interaction methods.

[0004] This application provides an interaction method applied to an in-vehicle robot, the method comprising: Upon detecting a preset interactive operation, the target interactive behavior corresponding to the preset interactive operation is executed, wherein the target interactive behavior is determined at least by the unlocked permissions of the vehicle robot.

[0005] Furthermore, this application also provides a vehicle-mounted robot, which is used to execute any of the interaction methods provided in this application.

[0006] In addition, this application embodiment also provides an interactive system, which includes the above-mentioned vehicle-mounted robot.

[0007] Furthermore, embodiments of this application also provide an electronic device, including one or more processors and a memory, wherein the memory stores a computer program, and the processor is used to run the computer program in the memory to implement any of the interactive methods provided in embodiments of this application.

[0008] Furthermore, embodiments of this application also provide a computer-readable storage medium storing a computer program. When the computer program is run on an electronic device, the computer program is used to cause the electronic device to execute any of the interaction methods provided in the embodiments of this application.

[0009] Furthermore, embodiments of this application also provide a computer program product, including a computer program or instructions, which, when executed by a processor, implement any of the interaction methods provided in embodiments of this application.

[0010] Furthermore, this application also provides a vehicle including the aforementioned electronic device or processor, wherein the processor is used to implement any of the interaction methods provided in this application when executing.

[0011] In this embodiment, upon detecting a preset interactive operation, a target interactive behavior corresponding to the preset interactive operation can be executed. The target interactive behavior is determined at least by the unlocked permissions of the in-vehicle robot. Thus, using the in-vehicle robot as the interaction node, multiple interactive behaviors can be executed based at least on the unlocked permissions of the in-vehicle robot, overcoming the limitations of existing single-feedback screen text / icons and enriching in-vehicle interaction methods. Attached Figure Description

[0012] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0013] To gain a more complete understanding of this application and its beneficial effects, the following description will be provided in conjunction with the accompanying drawings, wherein the same reference numerals in the following description denote the same parts.

[0014] Figure 1 This is a flowchart illustrating the interaction method provided in an embodiment of this application; Figure 2 This is a system architecture diagram of the interactive system provided in the embodiments of this application; Figure 3 This is another system architecture diagram of the interactive system provided in the embodiments of this application; Figure 4 This is a system architecture diagram of the system to which the interaction method provided in the embodiments of this application is applied; Figure 5 This is a schematic diagram illustrating the specific process of the interaction method provided in the embodiments of this application; Figure 6 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application; Figure 7 This is a schematic diagram of the vehicle provided in the embodiments of this application. Detailed Implementation

[0015] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0016] Furthermore, in the embodiments of this application, "multiple" refers to two or more. The terms "first" and "second," etc., in the embodiments of this application are used for distinguishing descriptions and should not be construed as implying relative importance.

[0017] This application provides an interaction method, an in-vehicle robot, an interaction system, an electronic device, a storage medium, a computer program product, and a vehicle. The electronic device may be the in-vehicle robot itself, a server controlling the in-vehicle robot, or a terminal controlled by the in-vehicle robot.

[0018] The server can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, network acceleration services (Content Delivery Network, CDN), as well as big data and artificial intelligence platforms.

[0019] The terminal controlled by the in-vehicle robot can refer to various in-vehicle functional modules or external devices that send control commands to the in-vehicle robot via wired or wireless communication and execute corresponding operations. For example, in an in-vehicle scenario, the terminal can be an in-vehicle air conditioning system, in-vehicle entertainment system, window / sunroof control module, seat adjustment module, headlight control module, door lock control module, in-vehicle fragrance / ambient lighting system, or other vehicle internal subsystems, but is not limited to these. In scenarios where the vehicle is interconnected with external devices, the terminal can also be a user mobile terminal (such as a smartphone or smartwatch), smart home device (such as a smart door lock or home air conditioner), or other external device that can establish a connection with the in-vehicle robot and receive its control commands through vehicle networking or near-field communication, but is not limited to these.

[0020] Terminals and servers can be connected directly or indirectly via wired or wireless communication, and this application does not impose any restrictions on this.

[0021] Taking the vehicle-mounted robot itself as an example, when a preset interactive operation is detected, the target interactive behavior corresponding to the preset interactive operation is executed. The target interactive behavior is determined at least by the unlocked permissions of the vehicle-mounted robot. Thus, using the vehicle-mounted robot as the interaction node, multiple interactive behaviors can be executed based on at least the unlocked permissions of the vehicle-mounted robot, breaking through the limitations of the existing single feedback of screen text / icons, thereby enriching the in-vehicle interaction methods.

[0022] It should be noted that the implementation environment scenarios of the interaction methods described in the embodiments of this application are for the purpose of more clearly illustrating the technical solutions of the embodiments of this application, and do not constitute a limitation on the technical solutions provided in the embodiments of this application. Those skilled in the art will understand that with the evolution of data processing and the emergence of new business scenarios, the technical solutions provided in this application are also applicable to similar technical problems.

[0023] The solutions provided in this application are specifically illustrated through the following embodiments. It should be noted that the order of description of the following embodiments is not intended to limit the preferred order of the embodiments.

[0024] This embodiment will be described from the perspective of an in-vehicle robot. The electronic device may be a terminal controlled by the in-vehicle robot and / or a server that controls the in-vehicle robot. This application does not impose any limitations on this.

[0025] This application provides an interaction method applied to an in-vehicle robot. The in-vehicle robot can be configured to perform various interactive behaviors corresponding to different permissions. The in-vehicle robot can also be referred to as an interaction device or an in-vehicle device.

[0026] It should be noted that this in-vehicle robot is a physical interactive device, and its placement within the vehicle is not limited. For example, it can be placed in a position visible from the driver's seat, existing as a decorative piece, or it can be placed on the center console, dashboard, roof, seat back, etc., without restriction. The robot's appearance is also not limited. For example, it can be anthropomorphic, animal-shaped, geometric, or any other arbitrary shape. Its specific size, materials, and the configuration of interactive components (such as displays, speakers, robotic arms, lighting modules, etc.) can be flexibly set according to the actual application scenario.

[0027] It should be noted that vehicle-mounted robots can be configured with multiple levels of permissions, including action permissions, data permissions, function call permissions, and device control permissions. For example, the permissions configured for a vehicle-mounted robot may include, but are not limited to, simple command execution permissions (such as time announcement and weather query), basic voice wake-up permissions, and scene linkage permissions. Each permission corresponds to at least one interactive behavior. For example, simple command execution permissions correspond to displaying and broadcasting the queried weather.

[0028] As is understood, an in-vehicle robot refers to a physical device that can support interaction with a vehicle. This in-vehicle robot can be a robot installed on the vehicle itself, or it can be another aftermarket product; there are no restrictions here.

[0029] Please see Figure 1 , Figure 1This is a flowchart illustrating an embodiment of an interaction method provided in this application. The interaction method may include the following step S101: Step S101: When a preset interactive operation is detected, the target interactive behavior corresponding to the preset interactive operation is executed, wherein the target interactive behavior is determined at least by the unlocked permissions of the vehicle robot.

[0030] Among them, the preset interactive operation refers to the operation used to trigger interactive behavior, such as the interactive command input by the user to the vehicle robot, which can be recognized by the system and trigger subsequent responses.

[0031] There are multiple preset interactive operations, and their specific configurations can be made according to actual conditions. No restrictions are imposed here.

[0032] Optionally, the interactive operation may include voice interaction. For example, the user saying "turn on the air conditioner" is a voice interaction operation.

[0033] Optionally, the interactive operation may include gesture interaction, such as the user waving or making a heart shape with their hand at the on-board robot.

[0034] Optionally, the interactive operation may include touch interaction operation, such as the user touching the touch-sensing area on the vehicle robot or pressing a physical button.

[0035] It should be noted that preset interactive operations can be applied to the onboard robot. "Applying to" the onboard robot means that the preset interactive operations have an effective influence on the onboard robot that can be perceived or received by the robot, and are sufficient to cause the robot to produce a corresponding technical response. This influence includes both mechanical inputs such as touch, pressing, and flicking applied directly to the onboard robot by the user through physical contact, and information inputs such as voice, gestures, light signals, electromagnetic signals, or remote communication commands sent to the onboard robot through non-contact methods. The specifics can be adjusted according to the actual situation and are not limited here.

[0036] The target interactive behavior refers to the interactive behavior triggered by a preset interactive operation, that is, the corresponding behavior executed or output by the vehicle robot after the preset interactive operation is detected.

[0037] In some embodiments, the target interactive behavior may include performing an action, and / or the vehicle robot outputting control commands to the vehicle in which the vehicle robot is located.

[0038] Specifically, the target interactive behavior includes at least two categories: first, actions performed by the vehicle-mounted robot itself, such as nodding, shaking, slow rotation, rapid rotation, displaying specific facial expressions, and broadcasting voice commands; second, control commands output by the vehicle-mounted robot to its vehicle, such as controlling the vehicle's air conditioning system to turn on and adjust the temperature, controlling the windows to rise and fall, controlling the in-vehicle entertainment system to play specific music, and controlling seat ventilation or heating. The specific content of the target interactive behavior is determined at least by the unlocked permissions of the vehicle-mounted robot; that is, the behavior can only be executed or output when the vehicle-mounted robot has the corresponding permissions.

[0039] The unlocked permissions refer to the set of permissions corresponding to the interactive behaviors that the vehicle robot is currently allowed to perform.

[0040] The interaction method of this application can execute a target interaction behavior corresponding to a preset interaction operation when a preset interaction operation is detected. The target interaction behavior is determined at least by the unlocked permissions of the in-vehicle robot. Thus, by using the in-vehicle robot as the interaction node, multiple interaction behaviors can be executed based on at least the unlocked permissions of the in-vehicle robot, overcoming the limitations of existing single-feedback screen text / icons and enriching in-vehicle interaction methods.

[0041] It should be noted that there are multiple ways to determine the unlocked permissions of the vehicle robot, and the specific method can be adjusted according to the actual situation. No restrictions are imposed here.

[0042] Optionally, the unlocked permissions of the vehicle-mounted robot can be determined by statistical parameters of the detected interaction operations.

[0043] Statistical parameters refer to indicators used to quantitatively evaluate user interaction performance and quality, including but not limited to interaction accuracy, cumulative effective interaction count, effective interaction percentage, task completion rate, and false trigger rate. For example, interaction accuracy can refer to the proportion of correctly recognized voice commands out of the total number of voice interactions; cumulative effective interaction count can refer to the cumulative number of effective interactions successfully triggered and completed by the user; and false trigger rate can refer to the proportion of non-intent-related erroneous triggers out of the total number of triggers.

[0044] For example, by statistical parameters such as the accuracy rate of the user's historical interaction operations, the cumulative number of valid interactions, and the percentage of valid interactions, it can be determined whether the user meets the permission unlocking conditions, and if the permission unlocking conditions are met, the user's permission to the vehicle robot is unlocked.

[0045] In this way, the unlocked permissions of the in-vehicle robot can be determined by the statistical parameters of the detected interactive operations, thereby enriching the permission unlocking methods. Then, with the in-vehicle robot as the interaction node, multiple interactive behaviors can be performed based on the unlocked permissions of the in-vehicle robot, thus breaking through the limitations of the existing single feedback of screen text / icons and enriching the in-vehicle interaction methods.

[0046] Optionally, the unlocked permissions can be determined by the association between the user identifier and the unlocked permissions of the vehicle robot, thus identifying the unlocked permissions corresponding to the target user identifier that initiated the interaction.

[0047] Specifically, by identifying the target user identifier of the user initiating the interactive operation, and based on the association between the user identifier and the different user permissions that have been unlocked in the vehicle robot, the unlocked permissions corresponding to the target user identifier initiating the interactive operation are determined.

[0048] In this way, unlocked permissions can be linked to users, improving the security of interactions based on in-vehicle robots.

[0049] In some embodiments, the unlocked permissions for the vehicle-mounted robot include at least some of the multi-level permissions available to the robot. Different levels of permissions require certain conditions to be met before they can be unlocked.

[0050] The unlocked permissions of the vehicle-mounted robot include at least some of the permissions in a multi-level action permission system. These permissions are determined by the parameter values ​​and threshold values ​​of statistical parameters of detected interactive operations. Examples include Level 1 permissions (basic voice interaction, simple command execution), Level 2 permissions (multi-turn dialogue, preset gestures, scene linkage), and Level 3 permissions (custom commands, complex gestures, large-amplitude rotation movements, emotional interaction), etc. Each level of permission corresponds to a different set of target interactive behaviors.

[0051] Among them, the parameter threshold refers to the preset numerical standard used to determine whether the conditions for granting or reverting permissions are met. For example, "the cumulative number of valid wake-ups is greater than or equal to 20 and the command recognition accuracy is greater than or equal to 90%" is used as the parameter threshold for unlocking from the initial state to level 1 permissions; "the false touch rate is greater than or equal to 30%" is used as the parameter threshold for triggering permission reverting.

[0052] Specifically, if the parameter value of the statistical parameter acting on the interactive operation of the vehicle robot meets the corresponding parameter threshold condition, the permission corresponding to the parameter threshold condition is unlocked.

[0053] In some embodiments, the above-described interaction method may further include: adjusting the unlocked permissions of the vehicle-mounted robot.

[0054] Optionally, the step "Adjust the unlocked permissions of the vehicle robot" may include: reverting the unlocked permissions if the permission reversion conditions are met.

[0055] The permission rollback condition refers to the set of conditions that trigger the revocation or downgrading of the unlocked permissions of the vehicle robot.

[0056] In one possible implementation, the permission fallback condition includes the default interaction operation violating the interaction constraints set for the security scenario.

[0057] Among them, safe scenarios refer to situations where there are driving safety risks or where the driver needs to maintain a high level of attention during vehicle operation. Examples include high-speed scenarios where the vehicle is traveling at a speed higher than a preset threshold, rapid acceleration or braking scenarios where the absolute value of the vehicle's longitudinal acceleration is greater than a preset threshold, lane-changing scenarios where the vehicle's lateral acceleration or steering angular velocity is greater than a preset threshold, and high-risk following scenarios where the distance between the vehicle and the vehicle in front is less than a preset threshold.

[0058] Interaction constraints refer to rules set for different security scenarios to limit the types and intensity of user interaction behaviors. For example, in high-speed scenarios, the interaction constraints may be limited to "only voice interaction is allowed, gesture interaction is prohibited", "only small-amplitude basic movements are allowed, high-amplitude or high-frequency rotational movements are prohibited", or "interactions that require visual attention shifts are prohibited", etc.

[0059] For example, when the vehicle is traveling at high speed (e.g., speed greater than or equal to 80 km / h), if the user initiates a complex gesture interaction that requires both hands, the preset interaction operation violates the interaction constraint that "distracting interaction is not allowed in high-speed scenarios," thereby triggering permission rollback.

[0060] Optionally, the step "Adjust the unlocked permissions of the vehicle robot" may also include: adding the unlocked permissions of the vehicle robot if the conditions for adding permissions are met.

[0061] The permission increase condition refers to the set of conditions that trigger the on-board robot to obtain higher-level or more functional permissions.

[0062] In one possible implementation, the permission addition condition includes completing a preset interactive task based on the preset interactive operation.

[0063] For example, if a user completes a certain number of valid interactions under the current permission status, and the accuracy of the interactions remains above a preset level, it is considered to have completed the preset interaction task, thereby satisfying the conditions for increasing permissions.

[0064] For example, the unlocked permissions of the vehicle robot include at least the first permission among the multi-level permissions that matches the current interaction state of the user who triggered the interaction operation.

[0065] When a user performs an advanced interactive task associated with the current interactive state, the unlocked permissions of the in-vehicle robot also include: at least a portion of the second permissions that match the interactive state after the advanced interactive task is completed in the multi-level permissions.

[0066] If a user performs an advanced interactive task related to the current interaction state, and the conditions for permission rollback are met, the second permission among the unlocked permissions will be rolled back.

[0067] The conditions for permission rollback include at least one of the following: failure of the advanced interaction task; abnormal user interaction behavior during the user's performance of the advanced interaction task.

[0068] When a user completes an advanced interactive task associated with their current interactive state, the user's current interactive state is updated to unlock the third permission in the multi-level permissions that matches the updated current interactive state; the third permission includes the second permission.

[0069] In some embodiments, the target interactive behavior can also be determined by the driving status of the vehicle in which the onboard robot is located.

[0070] Vehicle driving status refers to a set of parameters reflecting the current operating conditions of the vehicle, including but not limited to vehicle speed, longitudinal acceleration, lateral acceleration, steering angular velocity, distance to the vehicle in front, and lane departure status. The target interactive behavior can also be determined by the driving status of the vehicle in which the onboard robot is located. For example, when the vehicle driving status indicates an unsafe scenario, even if the onboard robot has unlocked permission for high-amplitude rotational movements, the system still restricts it from performing such movements and only allows it to output small-amplitude basic movements or voice feedback.

[0071] To facilitate better implementation of the interaction method provided in the embodiments of this application, this application also provides a vehicle-mounted robot for implementing the above-mentioned interaction method. The meanings of the terms used are the same as in the above-described interaction method, and specific implementation details can be found in the descriptions within the method embodiments.

[0072] In-vehicle robots refer to physical devices installed in vehicles that can interact with users and the vehicle. Their placement within the vehicle is not limited; for example, they can be placed on the center console within the driver's field of vision as decorative items, or on the dashboard, roof, or seatbacks. Their appearance is also not limited; they can be anthropomorphic, animal-shaped, or geometric in design, and can be equipped with interactive components such as displays, speakers, movable mechanical structures (such as heads, torsos, and robotic arms), and lighting modules as needed.

[0073] The vehicle-mounted robot may include a sensing module and an execution module. The sensing module is used to acquire preset interactive operations applied to the vehicle-mounted robot, and the vehicle-mounted robot performs steps of any of the aforementioned interactive methods. The execution module is used to execute a target interactive behavior that matches the preset interactive operations based on the vehicle-mounted robot, wherein the target interactive behavior is determined at least through the unlocked permissions of the vehicle-mounted robot.

[0074] The sensing module includes multiple sensing devices for acquiring preset interactive operations, such as voice receivers and touchpads, etc., which are not limited here.

[0075] In some embodiments, the execution module is further configured to control the execution actions of the vehicle-mounted robot, and / or control the vehicle-mounted robot to output control commands to the vehicle in which the vehicle-mounted robot is located.

[0076] The in-vehicle robot using this application can acquire preset interactive operations applied to the robot through a sensing module, and execute a target interactive behavior matching the preset interactive operations based on the robot through an execution module. The target interactive behavior is determined at least by the unlocked permissions of the in-vehicle robot. Thus, by using the in-vehicle robot as an interaction node, multiple interactive behaviors can be executed based on at least the unlocked permissions of the robot, overcoming the limitations of existing single-feedback screen text / icons and enriching in-vehicle interaction methods.

[0077] In some embodiments, the vehicle-mounted robot may further include a permission unlocking module, wherein the permission unlocking module is used to determine the unlocked permissions of the vehicle-mounted robot based on statistical parameters of the detected interactive operations.

[0078] In some embodiments, the permission unlocking module can also be used to adjust the unlocked permissions of the vehicle-mounted robot.

[0079] In practice, each of the above modules can be implemented as an independent entity or can be combined arbitrarily to be implemented as the same or several entities. For the specific implementation methods and corresponding beneficial effects of each of the above modules, please refer to the previous method embodiments, which will not be repeated here.

[0080] To facilitate better implementation of the interaction method provided in the embodiments of this application, the embodiments of this application also provide an interaction system for implementing the above-described interaction method. The meanings of the terms used are the same as in the above-described interaction method, and specific implementation details can be found in the descriptions within the method embodiments.

[0081] Please see Figure 2 The interactive system includes the aforementioned vehicle-mounted robot 201.

[0082] It is understood that the aforementioned vehicle-mounted robot 201 is also used to: output control commands to the vehicle where the vehicle-mounted robot 201 is located based on the target interactive behavior, so as to control the vehicle based on the control commands.

[0083] Please see Figure 3 In some embodiments, the above-described interactive system further includes a server 202; The aforementioned server 202 is communicatively connected to the vehicle-mounted robot 201 and is used to receive and respond to the interaction requests from the vehicle-mounted robot 201, so that the vehicle-mounted robot 201 can complete any of the above-mentioned interaction methods.

[0084] In some embodiments, the server 202 is further configured to obtain statistical parameters of the interactive operation detected by the vehicle robot 201, so as to trigger the vehicle robot 201 to unlock the permission matching the interactive operation when the statistical parameters of the interactive operation meet the preset unlocking conditions.

[0085] In some embodiments, the server 202 is further configured to construct a user behavior profile that matches the unlocked permissions of the vehicle-mounted robot 201.

[0086] In this way, by using the in-vehicle robot as an interaction node, multiple interactive behaviors can be performed based on the unlocked permissions of the in-vehicle robot, thus breaking through the limitations of the single feedback of existing screen text / icons and enriching the in-vehicle interaction methods.

[0087] To better understand the interaction method, vehicle-mounted robot, and interaction system provided in the embodiments of this application, a specific embodiment will be used for explanation and description below.

[0088] Existing in-vehicle robot interaction methods primarily use user and passenger status, vehicle status, and objective information as touchpoints to convey emotional information through voice and facial expressions. Based on different touchpoints and driving scenario data, they define the types of facial expressions on the interface and the display order and priority strategy for different types of expressions within the same node. However, this method suffers from fixed and monotonous action patterns, lacking growth attributes and making it difficult to cultivate stable interaction stickiness during long-term user engagement. Traditional motion control also lacks hierarchical logic, failing to adapt to the anthropomorphic interaction needs at different developmental stages.

[0089] To address at least some of the aforementioned problems, this application provides a gradient unlocking and dynamic control method for in-vehicle robots, belonging to the field of in-vehicle intelligent interaction technology. This method generates two core values—a development level and an intimacy level—by collecting user voice, touch, and driving behavior data. Based on preset mapping rules, it performs graded gradient unlocking of multi-degree-of-freedom postures such as nodding, shaking, and rotating on the in-vehicle robot (i.e., the in-vehicle device). As the development value increases, the system gradually unlocks large-amplitude, combined movements and personalized expressions (including hidden attributes), while simultaneously optimizing the smoothness of movements and the linkage effect of expressions. This application enables the in-vehicle robot's movement capabilities to dynamically grow with the interaction process, enhancing the fun and user stickiness of human-vehicle interaction while meeting in-vehicle driving safety requirements, and is applicable to various in-vehicle emotional interaction devices.

[0090] This application provides an interactive system that adopts a layered architecture of "interactive terminal layer – core control unit layer – vehicle execution layer – cloud service layer", with each layer working together through a standardized vehicle communication protocol.

[0091] Specifically, please refer to Figure 4 The core control unit layer integrates six functional modules to realize the entire process of perception, judgment, control and iteration. Its functional composition is shown in Table 1: Table 1

[0092] like Figure 4 As shown, the main interactive device (i.e., the vehicle-mounted terminal) and the core control unit employ dual-link communication via CAN and WIFI: CAN ensures real-time command transmission, while WIFI supports large-volume data exchange. The vehicle-mounted microphone array and camera connect to the core control unit through dedicated interfaces, with the camera using a GMSL interface to achieve low-latency, high-interference-resistance audio and video signal transmission. The core control unit communicates with the vehicle control system via the CANFD bus, whose bandwidth is significantly higher than traditional CAN, enabling the simultaneous issuance of multi-dimensional control commands. This communication architecture enables hierarchical control of cockpit functions and vehicle safety-related functions, thus balancing system interaction needs with vehicle safety management requirements.

[0093] Based on such Figure 4 The system architecture shown allows for the configuration of a gradual, incremental unlocking process. This process can be adjusted based on actual circumstances; no restrictions are imposed here.

[0094] For example, such as Figure 5The illustrated gradient unlocking process can adopt a "five-level state, three-level unlock, and closed-loop rollback" design. It divides the interaction actions into three levels: basic, intermediate, and advanced. It achieves the gradual opening of permissions through task-driven approaches, while ensuring interaction security through a rollback mechanism.

[0095] Phase 1, Initial Basic State (S1).

[0096] Specifically, when the system is first started or the user initializes the interactive device, the in-vehicle device is in its initial basic state, with only basic interaction permissions enabled and all advanced and high-level actions prohibited.

[0097] For example, the scope of permissions in Phase 1 only supports basic voice wake-up and simple command execution (such as time announcement and weather query), and does not provide custom actions or scene linkage capabilities. The core objective of Phase 1 is to complete the initial data collection of user interaction behavior and guide users to become familiar with the device's basic interaction logic.

[0098] Phase 2: Basic Interaction Tasks and Level 1 Action Unlocking (S2).

[0099] Specifically, if a user completes the preset basic interaction tasks in the initial state (such as accumulating ≥20 effective wake-ups, command recognition accuracy ≥90%, and no accidental touches / invalid interaction records), then the user's development level will be upgraded (S5) and the first-level action will be unlocked (S2).

[0100] Among them, the gradient unlocking module of the core control unit (equivalent to the permission unlocking module of the subsequent vehicle robot) determines that the task is completed, triggers the development level to be upgraded from level 0 to level 1, and updates the user behavior profile data.

[0101] After unlocking the first-level action (S2), first-level interactive action permissions can be granted. For example, first-level interactive action permissions may include: multi-turn dialogue function of basic voice interaction; preset gesture interaction (such as waving to wake up, heart gesture for thumbs up); basic scene linkage (such as automatically raising the air conditioning temperature when "I'm cold"); door opening to welcome guests, when the driver and passenger open the door, the interactive device will turn to the side of the door and display a smiley face.

[0102] In Phase 2, if the user meets the rollback permission conditions within the task cycle of the basic interaction task, a rollback due to unlocking failure (S6) is triggered, and the initial permission state is maintained. At the same time, a guidance prompt is pushed to the user. The rollback permission conditions for the basic interaction task may include a high accidental touch rate (≥30%) or a violation of the command execution rules (such as initiating a distracting interaction in a high-speed scenario).

[0103] Phase 3: Advanced interactive tasks and secondary actions unlocked (S3).

[0104] Specifically, if a user completes the corresponding advanced interaction task under the first-level permission status (such as accumulating ≥50 valid multi-turn dialogues, gesture recognition accuracy ≥95%, and no record of unsafe interaction violations), the user's level will be upgraded (S5) and the second-level action will be unlocked (S3).

[0105] Among them, the gradient unlocking module of the core control unit determines that the advanced interactive task is completed, the development level is upgraded from level 1 to level 2, and the cloud user profile is updated simultaneously.

[0106] After unlocking the second-level action (S3), second-level interactive action permissions can be granted. For example, second-level interactive action permissions may include: custom voice command settings (such as a user-defined "I'm sleepy" triggering a driver fatigue reminder + air conditioning cooling); compound gesture interaction (such as making a heart shape with both hands to trigger ambient light color change + music playback); multi-device linkage control in the cabin (such as simultaneously controlling windows, air conditioning, and seat ventilation); and unlocking nodding and shaking gestures when interactive devices interact with the user.

[0107] In Phase 3, if the user meets the rollback permission conditions within the task cycle of the advanced interaction task, a rollback due to unlocking failure (S6) will be triggered, and the permission will be rolled back to the first-level state. At the same time, interactive guidance and security prompts will be pushed to the user. The rollback permission conditions for the advanced interaction task may include a decrease in interaction accuracy (<90%), violation of security scenarios (such as initiating a compound gesture interaction while driving at high speed), etc.

[0108] Phase 4: Unlocking advanced interactive tasks and level 3 actions (S4).

[0109] Specifically, if a user completes a high-level interactive task while in a level 2 permission state (such as executing ≥30 valid custom commands, achieving a composite gesture recognition accuracy of ≥98%, having no security violation interaction records for a long period of time, and the cloud model determining that the user's interactive behavior is stable), then the user's level will be upgraded (S5) and level 3 action unlocked (S4).

[0110] Among them, the gradient unlocking module of the core control unit determines the completion of the high-level interaction task, triggering the development level to be upgraded from level 2 to level 3, and the cloud model completes the training of personalized interaction rules for users.

[0111] Once the Level 3 action unlock (S4) is achieved, all advanced interactive action permissions can be granted. For example, advanced interactive action permissions may include: emotional interactive actions (such as the device proactively triggering comforting words and playing soothing music when the user's voice is in a low mood); personalized scene linkage (such as automatically triggering the full cabin configuration of "commuter mode" and "driving mode" based on the user's driving habits); low wake-up word interaction (such as triggering the device response softly without needing to wake it up loudly); and in addition to nodding and shaking the head, the interactive device will unlock slow rotation and fast rotation.

[0112] It's important to note that even after unlocking Level 3 permissions, the system will continue to monitor user interactions, meaning continuous management and iteration. If long-term violations or a sharp drop in accuracy occur, permission downgrades can still be triggered. Simultaneously, cloud services will continuously update the user interaction model and optimize the execution logic of higher-level actions, achieving continuous iteration of the user experience.

[0113] Stage 5, Hidden Attributes Unlocked (S7).

[0114] Specifically, once a user completes all previous interactive tasks and achieves perfect behavioral evaluation, the cloud-based model will select the top 30% of the competitive rankings (the specific rankings can be dynamically adjusted based on the size of the group). Users at this stage are considered VIPs and will receive periodic targeted benefits such as personalized backgrounds, exclusive badges, and special emoticons.

[0115] It should be noted that the evaluation at each stage will collect users' driving behavior habits, such as rapid acceleration, rapid deceleration, speeding, arbitrary lane changes, crossing the line, running yellow lights, and fatigued driving, which will account for a preset percentage of the overall scoring mechanism (e.g., 40%).

[0116] It should be noted that the gradient unlocking module of the core control unit has a built-in multi-dimensional judgment model. Only when preset conditions are met can the level be upgraded and the corresponding action unlocked: Optionally, preset conditions may include interaction validity conditions, such as command / gesture recognition accuracy, effective interaction ratio, and task completion rate.

[0117] Optionally, the preset conditions may include safety and compliance conditions, such as whether the interaction scenario meets vehicle safety requirements (e.g., high-speed / low-speed scenarios, driving / parking scenarios) and whether there is a risk of distracted driving.

[0118] Optionally, preset conditions may include behavioral stability conditions, such as consistency of interactive behavior, no long-term violation records, and exclusion of interference from occasional operations.

[0119] It should be noted that the core control unit's numerical and action mapping mechanism achieves precise control from interactive behavior to vehicle actions through a two-way mapping of "numerical features - control commands." Mapping can include forward mapping, which maps the numerical features of user interaction (such as command accuracy, gesture trajectory, and emotion index) to vehicle control commands according to preset rules. For example, "the user says 'I'm cold' three times in a row" can be mapped to "automatically raise the air conditioning temperature by 2°C and turn on the seat heating." Mapping can also include reverse mapping, which maps vehicle execution results (such as air conditioning adjustment feedback and safety system status) to the basis for updating the user behavior profile. For example, "the user repeatedly adjusts the air conditioning temperature in ways that do not match the voice command," triggering cloud model optimization and adjusting the command-action mapping rules.

[0120] It should be noted that the cloud-based nurturing and iteration mechanism can utilize the nurturing mechanism cloud service to achieve a data closed loop, supporting the continuous evolution of the interactive experience. This can include data collection (i.e., uploading user interaction behavior data, unlock / rollback records, and vehicle execution feedback data), model training (i.e., training personalized interaction models based on user data to optimize unlock judgment thresholds and action mapping rules), and rule push (i.e., periodically pushing updated models and rules to the core unit to achieve iterative upgrades of local interaction logic without requiring manual user operation).

[0121] For example, the personalized interaction model can be trained according to Table 2 below: Table 2

[0122] For example, during the training of the M3 model, dynamic threshold adjustment can be performed using the formula Ni = N baseline × α cluster × β history. N baseline can be 20, α cluster ∈ [0.6, 1.4], and can be obtained by adjusting the cluster (novice cluster × 1.4, experienced cluster × 0.6). β history ∈ [0.5, 1.5], and can be adjusted using historical data (high accuracy × 0.5, low accuracy × 1.5).

[0123] For example, the security classification of the M4 model can be referred to in Table 3 below: Table 3

[0124] Thus, by constructing a dual numerical system of development levels and intimacy levels, multi-degree-of-freedom postures such as nodding, shaking, and rotating are divided into multi-level tiered permissions based on amplitude, frequency, and complexity. This allows actions to be unlocked in a gradient manner according to development values, creating a companion-like interactive experience. The growth mechanism drives continuous user interaction, significantly enhancing the fun of human-vehicle interaction and user stickiness. Furthermore, by unlocking actions in layers, lower levels correspond to basic, gentle movements, while higher levels unlock complex combinations of rotating movements, resulting in a more natural anthropomorphic effect and avoiding mechanical stiffness.

[0125] Furthermore, traditional interaction solutions lack safety adaptation for in-vehicle scenarios, failing to balance interactive enjoyment with driving safety. This application's embodiment, however, incorporates a dynamic safety control mechanism for action permissions based on vehicle driving status. In unsafe driving scenarios such as high speeds, sudden braking, and lane changes, it automatically blocks high-amplitude, high-frequency rotational actions, retaining only small-amplitude basic actions. This balances interactive entertainment with driving safety, avoiding the risk of actions interfering with driving and better complying with in-vehicle safety regulations. By assigning training value bonuses to safe driving behaviors, it guides users to drive safely, achieving a two-way promotion of interaction and safety.

[0126] This application also provides an electronic device, such as... Figure 6 As shown, it illustrates a structural schematic diagram of the electronic device involved in the embodiments of this application, specifically: The electronic device may include components such as a processor 301 with one or more processing cores, a memory 302 with one or more storage media, a power supply 303, and an input unit 304. Those skilled in the art will understand that... Figure 6 The electronic device structure shown does not constitute a limitation on the electronic device and may include more or fewer components than shown, or combine certain components, or have different component arrangements. Wherein: The processor 301 is the control center of the electronic device, connecting various parts of the device via various interfaces and lines. It executes computer programs and / or modules stored in the memory 302, and calls data stored in the memory 302 to perform various functions and process data. Optionally, the processor 301 may include one or more processing cores; preferably, the processor 301 may integrate an application processor and a modem processor, wherein the application processor mainly handles the operating system, user interface, and applications, and the modem processor mainly handles wireless communication. It is understood that the modem processor may not be integrated into the processor 301.

[0127] The memory 302 can be used to store computer programs and modules. The processor 301 executes various functional applications and interactions by running the computer programs and modules stored in the memory 302. The memory 302 may mainly include a program storage area and a data storage area. The program storage area may store the operating system, computer programs required for at least one function (such as audio-visual prompts, interactive functions, etc.), etc.; the data storage area may store data created based on the use of the electronic device. In addition, the memory 302 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device. Accordingly, the memory 302 may also include memory electronics to provide the processor 301 with access to the memory 302.

[0128] The electronic device also includes a power supply 303 that supplies power to various components. Preferably, the power supply 303 can be logically connected to the processor 301 through a power management system, thereby enabling functions such as charging, discharging, and power consumption management through the power management system. The power supply 303 may also include one or more DC or AC power supplies, recharging systems, power fault detection circuits, power converters or inverters, power status indicators, and other arbitrary components.

[0129] The electronic device may also include an input unit 304, which can be used to receive input digital or character information and generate keyboard, mouse, joystick, optical or trackball signal inputs related to user settings and function control.

[0130] Although not shown, the electronic device may also include a display unit, etc., which will not be described in detail here. Specifically, in this embodiment, the processor 301 in the electronic device loads the executable files corresponding to the processes of one or more computer programs into the memory 302 according to the following instructions, and the processor 301 runs the computer programs stored in the memory 302 to realize various functions, such as: If a preset interactive operation is detected, the target interactive behavior corresponding to the preset interactive operation is executed, wherein the target interactive behavior is determined at least by the unlocked permissions of the vehicle robot.

[0131] Therefore, the electronic device provided in this application, upon detecting a preset interactive operation, executes a target interactive behavior corresponding to that preset interactive operation, wherein the target interactive behavior is determined at least by the unlocked permissions of the in-vehicle robot. Thus, by using the in-vehicle robot as the interaction node, multiple interactive behaviors are executed based at least on the unlocked permissions of the in-vehicle robot, overcoming the limitations of existing single feedback from screen text / icons and enriching in-vehicle interaction methods.

[0132] For details on the specific implementation methods and corresponding beneficial effects of the above operations, please refer to the detailed description of the interaction methods above, which will not be repeated here.

[0133] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be performed by a computer program, or by a computer program controlling related hardware. The computer program can be stored in a storage medium and loaded and executed by a processor.

[0134] Therefore, embodiments of this application provide a computer-readable storage medium storing a computer program that can be loaded by a processor to execute steps in any of the interactive methods provided in embodiments of this application. For example, the computer program can execute the following steps: If a preset interactive operation is detected, the target interactive behavior corresponding to the preset interactive operation is executed, wherein the target interactive behavior is determined at least by the unlocked permissions of the vehicle robot.

[0135] Therefore, the storage medium provided in this application embodiment executes the target interactive behavior corresponding to the preset interactive operation when a preset interactive operation is detected. The target interactive behavior is determined at least by the unlocked permissions of the in-vehicle robot. Thus, by using the in-vehicle robot as the interaction node, multiple interactive behaviors are executed based at least on the unlocked permissions of the in-vehicle robot, overcoming the limitations of existing single feedback from screen text / icons and enriching in-vehicle interaction methods.

[0136] For details on the specific implementation methods and corresponding beneficial effects of the above operations, please refer to the previous embodiments, which will not be repeated here.

[0137] The storage medium may include: read-only memory (ROM), random access memory (RAM), disk or optical disk, etc.

[0138] Since the computer program stored in the storage medium can execute the steps in any of the interactive methods provided in the embodiments of this application, the beneficial effects that any of the interactive methods provided in the embodiments of this application can achieve can be realized, as detailed in the preceding embodiments, and will not be repeated here.

[0139] According to one aspect of this application, a computer program product or computer program is provided, comprising computer instructions stored in a storage medium. A processor of a computer device reads the computer instructions from the storage medium and executes the computer instructions, causing the computer device to perform the aforementioned interactive method.

[0140] This application also provides a vehicle that includes the aforementioned vehicle-mounted robot, processor, electronic device, or computer program product.

[0141] For example, please refer to Figure 7 The vehicle 400 may include the aforementioned electronic device, which, upon detecting a preset interactive operation, executes a target interactive behavior corresponding to the preset interactive operation. The target interactive behavior is determined at least by the unlocked permissions of the onboard robot. Thus, by using the onboard robot as the interaction node, multiple interactive behaviors can be executed based at least on the unlocked permissions of the onboard robot, overcoming the limitations of existing single-feedback screen text / icons and enriching the onboard interaction methods.

[0142] This application does not limit the specific structure of the vehicle. The specific implementation methods and corresponding beneficial effects of the above-described operations of the control device are also applicable to this vehicle. For details, please refer to the detailed description of the interaction method above, which will not be repeated here.

[0143] The foregoing has provided a detailed description of an interaction method, an in-vehicle robot, an interaction system, an electronic device, a storage medium, a computer program product, and a vehicle provided in the embodiments of this application. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the embodiments above are only for the purpose of helping to understand the method and core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. An interaction method, characterized in that, Applied to an in-vehicle robot, the in-vehicle robot being used to direct physical devices that can support interaction with a vehicle, the method includes: When a user inputs a preset interactive operation to the vehicle-mounted robot and acts on the robot, the target interactive behavior corresponding to the preset interactive operation is executed. The target interactive behavior is determined at least by the unlocked permissions of the vehicle-mounted robot, which include at least one of the following: action permissions, data permissions, function call permissions, and device control permissions. The unlocked permissions of the vehicle-mounted robot are determined by statistical parameters of the detected interactive operation. These statistical parameters include at least one of the following: interaction accuracy rate, cumulative effective interaction count, effective interaction percentage, and false touch rate. The interaction accuracy rate can be used to indicate the proportion of correctly recognized voice commands to the total number of voice interactions, and the false touch rate can be used to indicate the proportion of non-intentional erroneous triggers to the total number of triggers. The unlocked permissions of the vehicle-mounted robot include at least some of the permissions in the multi-level permissions of the vehicle-mounted robot; wherein, the at least some permissions are determined by the parameter values ​​and parameter thresholds of the statistical parameters of the detected interactive operations; If the permission rollback conditions are met, the unlocked permissions of the vehicle-mounted robot are rolled back, wherein the permission rollback conditions include the preset interaction operation violating the interaction constraints set for the safety scenario; and / or, If the conditions for adding permissions are met, the unlocked permissions of the vehicle robot are added, wherein the conditions for adding permissions include completing a preset interactive task based on the preset interactive operation.

2. The interaction method according to claim 1, characterized in that, The preset interactive operations include at least one of voice, gesture, and touch.

3. The interaction method according to claim 1, characterized in that, The target interactive behavior includes performing actions and / or outputting control commands to the vehicle in which the onboard robot is located.

4. The interaction method according to claim 1, characterized in that, The target interactive behavior is also determined by the driving status of the vehicle in which the onboard robot is located.

5. A vehicle-mounted robot, characterized in that, The vehicle-mounted robot is used to perform the steps of the interaction method as described in any one of claims 1 to 4.

6. An interactive system, characterized in that, The interactive system includes the vehicle-mounted robot as described in claim 5.

7. The interactive system according to claim 6, characterized in that, The vehicle-mounted robot is also used to: output control commands to the vehicle where the vehicle-mounted robot is located based on the target interactive behavior, so as to control the vehicle based on the control commands.

8. The interactive system according to claim 6, characterized in that, The system also includes a server; The server is communicatively connected to the vehicle-mounted robot and is used to receive and respond to the interaction requests from the vehicle-mounted robot, so that the vehicle-mounted robot can complete the interaction method as described in any one of claims 1 to 4.

9. The interactive system according to claim 8, characterized in that, The server is also used to obtain statistical parameters of the interactive operation detected by the vehicle robot, so as to trigger the vehicle robot to unlock the permission matching the interactive operation when the statistical parameters of the interactive operation meet the preset unlocking conditions.

10. The interactive system according to claim 8, characterized in that, The server is also used to construct a user behavior profile that matches the unlocked permissions of the vehicle-mounted robot.

11. An electronic device having a computer program or instructions stored thereon, characterized in that, When the computer program or instructions are executed by a processor, they implement the steps of the interactive method according to any one of claims 1 to 4.

12. A vehicle, characterized in that, include: The electronic device as claimed in claim 11; Alternatively, a processor, the processor being configured to perform the interaction method according to any one of claims 1 to 4.

Citation Information

Patent Citations

  • Vehicle and vehicle-mounted data processing method and device

    CN110871756A

  • Humanoid robot interaction system based on big data

    CN121615680A