A control method of a car machine assistant, an electronic device, and a vehicle

Abstract

The application discloses a control method of a car machine assistant, an electronic device and a vehicle, relates to the technical field of intelligent cockpits, and comprises the following steps: monitoring the real-time vehicle speed of a target vehicle and the current display position of a car machine assistant in a car machine screen; when the real-time vehicle speed is greater than a preset threshold and the current display position is in a non-safe display area, migrating the car machine assistant from the current display position to a safe display area in the car machine screen; and when the real-time vehicle speed is less than or equal to the preset threshold, collecting the line-of-sight information of a driver to a target object, determining a target display position based on the line-of-sight information, and migrating the car machine assistant from the current display position to the target display position. The application can take into account driving safety and interaction convenience, and improve the intelligent level and user experience of a virtual assistant on a vehicle.

Description

Technical Field

[0001] This application relates to the field of smart cockpit technology, and in particular to a control method, electronic device and vehicle for a vehicle assistant. Background Technology

[0002] With the development of smart cockpit technology, in-vehicle virtual personal assistants (VPAs) have become an important interface for human-computer interaction, providing drivers with services such as voice interaction, status feedback, and emotional companionship.

[0003] However, current in-vehicle VPAs adjust the display position of the virtual avatar based on user commands, such as sound source localization or voice commands triggering the avatar's migration. But this approach relies on active user commands and cannot achieve seamless adaptive migration based on changes in the driving scenario. Furthermore, when the vehicle is traveling at high speeds, if the virtual avatar is displayed in the center of the screen for an extended period, it can easily distract the driver and affect driving safety.

[0004] Therefore, how to achieve adaptive control of the display position of the in-vehicle virtual assistant while ensuring driving safety has become an urgent technical problem to be solved. Summary of the Invention

[0005] This application provides a control method, electronic device, and vehicle for a vehicle-mounted assistant, which can balance driving safety and interactive convenience, and improve the intelligence level and user experience of the in-vehicle virtual assistant.

[0006] This application provides a control method for a vehicle-mounted assistant, including: The system monitors the real-time speed of the target vehicle and the current display position of the vehicle assistant on the vehicle screen. When the real-time speed is greater than a preset threshold and the current display position is in a non-safe display area, the system moves the vehicle assistant from the current display position to a safe display area on the vehicle screen. When the real-time speed is less than or equal to the preset threshold, the system collects the driver's line-of-sight information about the target object and determines the target display position based on the line-of-sight information, so as to move the vehicle assistant from the current display position to the target display position.

[0007] This application embodiment monitors the real-time speed of the target vehicle and the current display position of the vehicle assistant. When the vehicle speed exceeds a preset threshold and the current display position is in an unsafe area, the vehicle assistant is automatically moved to a safe display area. This effectively avoids the problem of the virtual image obscuring the core field of view of the screen and thus distracting the driver's attention when driving at high speeds, ensuring driving safety. At the same time, when the vehicle speed is less than or equal to the preset threshold, the vehicle assistant is moved to the target display position desired by the driver based on the driver's line of sight information. This enables convenient interaction in low-speed or parking scenarios, where the virtual image position can be adjusted without manual operation. By integrating real-time vehicle speed and line of sight information for control, the display strategy of the virtual assistant is adaptively switched according to different driving states, improving the interactive convenience and intelligence level of the in-vehicle virtual assistant while ensuring driving safety.

[0008] Optionally, the step of collecting the driver's gaze information on the target object and determining the target display position based on the gaze information includes: acquiring the driver's gaze duration and gaze direction on the target object; and determining the target display position based on the gaze direction if the gaze duration is greater than or equal to a preset duration threshold.

[0009] This optional implementation collects the driver's gaze duration and gaze direction for the target object, and determines the target display position only when the gaze duration meets the condition based on the gaze direction. This can effectively filter out the driver's unconscious and brief eye scanning behavior, avoid frequent and meaningless changes in the virtual assistant's position, improve interaction stability, make the virtual assistant's position adjustment fit the driver's real gaze intention, and optimize the human-computer interaction experience.

[0010] Optionally, obtaining the driver's gaze duration and gaze direction towards the target object includes: acquiring a facial image of the driver through an image acquisition device; extracting the driver's eye feature information and / or head posture information based on the facial image; determining the driver's gaze direction on the vehicle screen based on the eye feature information and / or the head posture information; determining the driver's gaze point on the vehicle screen based on the gaze direction, and recording the duration of the gaze point as the gaze duration.

[0011] This optional implementation uses an image acquisition device installed inside the vehicle to capture the driver's facial image and extract eye feature information and / or head posture information. This allows for the determination of the driver's gaze direction, gaze point, and duration on the vehicle's infotainment screen, enabling precise acquisition of the driver's gaze information. Combining eye features and head posture information to determine the gaze direction improves the accuracy of gaze tracking. Recording the duration of the gaze point and comparing it with a preset duration threshold effectively distinguishes between the driver's conscious gaze and unconscious scanning, preventing frequent movement of the virtual avatar due to accidental eye contact. This provides accurate and reliable input for the subsequent location migration of the virtual assistant based on gaze information.

[0012] Optionally, the preset threshold includes a first vehicle speed threshold and a second vehicle speed threshold, wherein the first vehicle speed threshold is less than the second vehicle speed threshold; when the real-time vehicle speed is greater than the second vehicle speed threshold and the current display position is in a non-safe display area, the vehicle assistant is moved from the current display position to a safe display area on the vehicle screen; when the real-time vehicle speed is less than or equal to the second vehicle speed threshold and greater than the first vehicle speed threshold, collecting the driver's gaze information on the target object includes: collecting the driver's gaze information on the target object at a first collection frequency and using a first duration as the preset duration threshold; when the real-time vehicle speed is less than or equal to the first vehicle speed threshold, collecting the driver's gaze information on the target object includes: collecting the driver's gaze information on the target object at a second collection frequency and using a second duration as the preset duration threshold; wherein the first collection frequency is less than the second collection frequency, and the first duration is greater than the second duration.

[0013] This optional implementation divides the vehicle's driving state into three intervals: high speed, medium speed, and low speed by setting a first speed threshold and a second speed threshold, and configures different virtual assistant display control strategies for each interval. Compared with a single threshold control scheme, it can more precisely adapt to the safety and interaction needs at different speeds, and retain the convenience and usability of eye-to-eye interaction to the greatest extent while ensuring driving safety.

[0014] Optionally, migrating the vehicle assistant from the current display position to the safe display area includes: controlling the vehicle assistant to migrate from the current display position to the safe display area with a first animation effect; migrating the vehicle assistant from the current display position to the target display position includes: controlling the vehicle assistant to migrate from the current display position to the target display position with a second animation effect; wherein the migration speed of the first animation effect is greater than the migration speed of the second animation effect.

[0015] This optional implementation configures different migration animations for high-speed migration scenarios and low-speed line-of-sight interaction scenarios, allowing the virtual avatar's migration process to match the current driving state: rapid migration at high speeds to ensure safety, and smooth migration at low speeds or when stationary to enhance the interactive experience. This differentiated animation control method balances driving safety and interactive comfort, further improving the user experience of the in-vehicle virtual assistant in different driving scenarios.

[0016] Optionally, the method further includes: stopping the collection of the driver's line-of-sight information about the target object when the real-time vehicle speed is greater than the preset threshold.

[0017] This optional implementation dynamically starts and stops collecting driver line-of-sight information based on real-time vehicle speed. The collection work is suspended when driving at high speeds to reduce the computational load on the vehicle system and avoid distracting the driver's attention with frequent interface adjustments in high-speed scenarios. Line-of-sight monitoring is resumed in low-speed scenarios, taking into account interactive intelligence and achieving a balance between driving safety and efficient use of equipment computing power.

[0018] Optionally, the step of migrating the vehicle assistant from the current display position to a safe display area in the vehicle screen includes: obtaining driver seat side information, and determining a target safe display area from multiple safe display areas based on the driver seat side information; and migrating the vehicle assistant from the current display position to the target safe display area.

[0019] This optional implementation selects a target safety display area by acquiring information from the driver's seat and moves the vehicle assistant to that area. It can rationally plan the display position of the virtual assistant based on the driver's position, avoid blind spots and key observation areas, reduce the interference of the virtual image on the driver's line of sight, and improve driving safety while ensuring the continuity of interaction.

[0020] Optionally, after migrating the vehicle assistant from the current display position to a safe display area on the vehicle screen, or migrating the vehicle assistant from the current display position to the target display position, the method further includes: obtaining preset display parameters of the vehicle assistant corresponding to the safe display area or the target display position; and adjusting the display form of the vehicle assistant based on the preset display parameters.

[0021] This optional implementation adjusts the display form of the virtual image according to the preset display parameters corresponding to the target location after the virtual image migrates to the target location. This allows the size, range of motion, transparency, and display style of the virtual image to match the display characteristics of the current location. This not only ensures the low-interference characteristics of the safe display area but also enhances the visual appeal and interactive experience of the interactive location, further optimizing the adaptability of the in-vehicle virtual assistant in different display locations.

[0022] This application also provides a control device for a vehicle-mounted assistant, including: The monitoring unit is used to monitor the real-time speed of the target vehicle and the current display position of the vehicle assistant on the vehicle screen; The control unit is configured to, when the real-time vehicle speed exceeds a preset threshold and the current display position is in a non-safe display area, Move the vehicle assistant from its current display position to a secure display area on the vehicle screen. The control unit is also used to collect the driver's line-of-sight information about the target object when the real-time vehicle speed is less than or equal to the preset threshold, and determine the target display position based on the line-of-sight information so as to migrate the vehicle assistant from the current display position to the target display position.

[0023] This application also provides an electronic device, including: a memory for storing a computer program; and a processor for executing the computer program to implement the control method of any of the above-described vehicle assistants.

[0024] This application also provides a vehicle, including a memory and a processor, wherein the memory stores executable program code, and the processor is used to call and execute the executable program code to implement the control device of the above-mentioned vehicle assistant.

[0025] This application also provides a computer-readable storage medium storing a computer program, wherein when the computer program is executed by a processor, it implements the steps of any of the above-described vehicle assistant control methods.

[0026] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of any of the above-described vehicle assistant control methods. Attached Figure Description

[0027] To more clearly illustrate the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0028] Figure 1 One of the flowcharts of a vehicle-mounted assistant control method provided in this application embodiment; Figure 2 A schematic diagram of the interface of the vehicle display screen in a control method for a vehicle assistant provided in an embodiment of this application; Figure 3A second schematic flowchart illustrating a control method for a vehicle-mounted assistant provided in an embodiment of this application; Figure 4 A third schematic flowchart illustrating a control method for a vehicle-mounted assistant provided in this application embodiment; Figure 5 This is a schematic diagram of the structure of a vehicle assistant control device provided in an embodiment of this application. Detailed Implementation

[0029] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the protection scope of this application.

[0030] It should be noted that, in the description of this application, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. The terms "first," "second," etc., in this application are used to distinguish similar objects and are not used to describe a specific order or sequence.

[0031] To enable those skilled in the art to better understand the present application, the present application will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0032] The present invention will now be described in detail with reference to the accompanying drawings and specific embodiments.

[0033] The embodiments of this application provide a control method for a vehicle-mounted assistant, referring to... Figure 1 The diagram shown is a flowchart of a control method for a vehicle-mounted assistant, which includes the following steps: S11. Monitor the real-time speed of the target vehicle and the current display position of the vehicle assistant on the vehicle screen.

[0034] In this step, it is necessary to obtain the target vehicle's speed in real time and simultaneously obtain the current display position of the vehicle assistant on the vehicle screen.

[0035] Specifically, the real-time vehicle speed can be obtained by reading the real-time vehicle speed signal from the vehicle's Electronic Control Unit (ECU) through the vehicle's Controller Area Network (CAN) bus or On-Board Diagnostics (OBD) interface, and then directly obtaining the speed data displayed on the vehicle's instrument panel.

[0036] It should be noted that real-time vehicle speed monitoring is a continuous process. It begins monitoring vehicle speed changes as soon as the vehicle starts, so that it can respond promptly when the vehicle speed exceeds a preset threshold.

[0037] Simultaneously, it is also necessary to obtain the current display position of the vehicle assistant on the vehicle screen. In this embodiment, the vehicle assistant instance is a globally unique instance of the vehicle system, meaning that there is only one vehicle assistant instance in the entire vehicle system, and its corresponding target virtual image is displayed in only one host area of ​​the vehicle screen at any given time. By adopting a globally unique vehicle assistant instance architecture, the task interruption problem caused by the isolation of dialogue contexts and interaction states of each instance under the traditional multi-entry, multi-instance architecture can be avoided. At the same time, the interface logic confusion caused by the simultaneous display of multiple assistants can be avoided, and the consumption of vehicle system computing power by multiple instances residing can be effectively reduced, thereby improving the smoothness of virtual assistant interaction and the consistency of user experience.

[0038] The current display position refers to the real-time position of the target virtual image on the vehicle's infotainment screen. The vehicle system maintains the display parameters of the target virtual image in real time. These display parameters include at least the coordinates of the target virtual image on the vehicle's infotainment screen (e.g., two-dimensional coordinates with the top left corner of the screen as the origin) and size parameters (width and height). By reading these display parameters, the current display position of the target virtual image can be determined. In one embodiment, the current display position is characterized by the coordinates of the geometric center point, anchor point, or predetermined reference point of the target virtual image on the vehicle's infotainment screen.

[0039] In one implementation, the display area of ​​the vehicle infotainment screen is divided into multiple host areas. The target virtual image corresponding to the vehicle assistant instance is bound to one of these host areas for display at any given time. The multiple host areas include a first host area corresponding to the side function panel, the vehicle status bar area, and the dynamic island area, etc. The side function panel occupies a preset area of ​​the vehicle infotainment screen, for example, one-third of the screen's area; that is, when unfolded, the side function panel occupies approximately one-third of the screen's width or area.

[0040] When the side panel is expanded, the vehicle assistant instance is bound to the first host area corresponding to the side panel, such as the center position of the top edge of the side panel window. The display position of the target virtual image changes synchronously with the shape of the side panel, such as coordinate and size changes caused by dragging or scaling operations. When the side panel is collapsed or closed, the vehicle assistant instance migrates from its current first host area to a second host area, such as the vehicle status bar area or the dynamic island area, and adjusts the display form of the target virtual image based on the preset virtual image display parameters corresponding to the second host area. The vehicle assistant is then displayed as a virtual image on the vehicle screen.

[0041] It should also be noted that the real-time vehicle speed and location can be continuously monitored through a background service that runs resident in the vehicle system. This background service is activated after the vehicle starts and remains running during vehicle operation to ensure real-time response to changes in vehicle speed.

[0042] S12. When the real-time vehicle speed is greater than a preset threshold and the current display position is in a non-safe display area, the vehicle assistant is moved from the current display position to a safe display area in the vehicle screen.

[0043] In this step, when the real-time vehicle speed detected by S11 exceeds the preset threshold and the current display position of the vehicle assistant is not within the safe display area, the virtual image is triggered to migrate from its current display position to the safe display area.

[0044] The preset threshold is used to distinguish between high-speed driving and low-speed or stationary driving states. The specific value of the preset threshold can be flexibly configured according to actual needs.

[0045] For example, a preset threshold can be set to 60 km / h as a typical speed dividing line between urban roads and expressways; alternatively, it can be set to 40 km / h or other values ​​to adapt to different road types and driving scenarios. Specifically, in highway scenarios, the preset threshold can be set to 80 km / h to trigger the virtual avatar's migration only at higher speeds; in congested urban scenarios, the preset threshold can be set to 30 km / h to move the virtual avatar out of the driver's field of vision earlier. Furthermore, the preset threshold can be dynamically adjusted according to traffic regulations and driving habits in different regions.

[0046] It should be noted that the preset threshold can be set at the factory, or it can be customized by the user in the vehicle system settings interface, or it can be dynamically adjusted according to the speed limit information of the road where the vehicle is currently located.

[0047] The safe display area refers to the area on the vehicle's infotainment screen that will not interfere with the driver's primary line of sight. (See reference...) Figure 2 As shown, the safety display area is the status information bar area 22, the dynamic island area 23, or the lower right corner area 25 of the vehicle screen. The virtual image 21 corresponding to the vehicle assistant can be displayed in the above areas. These areas are usually located at the top or bottom of the screen, are narrow and located at the edge of the screen. Drivers rarely use these areas as the main information acquisition areas during driving.

[0048] Correspondingly, non-safe display areas refer to areas other than the aforementioned safe display areas, such as the center of the screen. The center of the screen is typically where navigation maps, multimedia information, and other content are displayed, and it is also the location that drivers frequently focus on while driving. If the virtual avatar is displayed in the center of the screen for an extended period, it can easily distract the driver and affect driving safety. Therefore, when the real-time vehicle speed exceeds a preset threshold and the virtual avatar is currently located in a non-safe display area, it is necessary to move the virtual avatar from its current position to a safe display area.

[0049] Furthermore, the implementation process of S12 above (moving the vehicle assistant from the current display position to the safe display area in the vehicle screen) can be further refined into the following steps 1 and 2: Step 1: Obtain driver seat side information and determine the target safety display area from multiple safety display areas based on the driver seat side information.

[0050] This step requires determining whether the driver's seat of the target vehicle is located on the left or right side. Depending on the target market, vehicles may be either left-hand drive or right-hand drive. In left-hand drive vehicles, the driver is on the left side, and when viewing the infotainment screen, the right side offers a relatively wider field of vision and less visual interference. Conversely, in right-hand drive vehicles, the driver is on the right side, and the situation is reversed.

[0051] Driver seat information can be pre-stored in the vehicle's infotainment system configuration file when the vehicle leaves the factory, or it can be automatically identified through the Vehicle Identification Number (VIN) or target market information.

[0052] Since there are multiple safety display areas, the degree of visual interference to the driver may vary depending on the area. For example, in a left-hand drive vehicle, the driver's line of sight is closer to the left side of the screen and farther to the right. In this case, moving the virtual avatar to the lower right corner of the screen, compared to moving it to the virtual island, is further away from the driver's direct line of sight and thus less disruptive.

[0053] Therefore, selecting the target safety display area based on the driver's seat side information can further move the migrated virtual image away from the driver's direct line of sight, further reducing interference with the driver's field of vision.

[0054] For example, if the driver's seat side information indicates that the driver is on the left side of the vehicle, the right side of the screen is selected from multiple safety display areas, and the lower right corner or status information bar is selected as the target safety display area; if the driver's seat side information indicates that the driver is on the right side of the vehicle, the left side of the screen is selected from multiple safety display areas, such as the dynamic island area, as the target safety display area.

[0055] Step 2: Move the vehicle assistant from the current display position to the target safety display area.

[0056] Therefore, after determining the target safe display area, the virtual image can be moved from the current display position to the target safe display area.

[0057] By using steps 1 and 2 above, the target safety display area is selected by obtaining information from the driver's seat side, and the vehicle assistant is moved to that area. This allows for a reasonable planning of the virtual assistant's display position based on the driver's orientation, avoiding blind spots and key observation areas, reducing the interference of the virtual image on the driver's line of sight, and improving driving safety while ensuring the continuity of interaction.

[0058] In some embodiments, this application embodiment further includes stopping the collection of the driver's line-of-sight information about the target object when the real-time vehicle speed is greater than the preset threshold.

[0059] Specifically, when the vehicle is traveling at high speed, even if the driver's gaze falls on a specific area of ​​the vehicle's infotainment screen, the system will not respond to the gaze information to move the infotainment assistant. This prevents the infotainment assistant from being accidentally moved to the center of the screen or other unsafe display areas due to accidental touches while traveling at high speed, thus ensuring driving safety.

[0060] When the real-time vehicle speed drops to less than or equal to the preset threshold, the locking state is released, thereby restoring normal response to the gaze information, allowing the driver to control the display position of the vehicle assistant by looking at the vehicle screen. If the real-time vehicle speed exceeds the preset threshold, the response function of relocating the vehicle assistant based on gaze information can also be locked.

[0061] This application embodiment collects driver line-of-sight information dynamically based on real-time vehicle speed. The collection work is paused when driving at high speeds to reduce the computational load on the vehicle system and avoid distracting the driver's attention with frequent interface adjustments in high-speed scenarios. Line-of-sight monitoring is resumed in low-speed scenarios, taking into account interactive intelligence and achieving a balance between driving safety and efficient use of equipment computing power.

[0062] S13. When the real-time vehicle speed is less than or equal to the preset threshold, collect the driver's line of sight information to the target object, and determine the target display position based on the line of sight information, so as to move the vehicle assistant from the current display position to the target display position.

[0063] Specifically, when the monitored real-time vehicle speed does not exceed a preset threshold, i.e. when the vehicle is in a low-speed driving or parked state, the interaction mode based on line-of-sight information is activated. That is, by collecting the driver's line-of-sight information, the system determines the user's desired target display position based on the line-of-sight information, and then moves the vehicle assistant from the current display position to the target display position.

[0064] The target object refers to the object that the driver is looking at on the vehicle's infotainment screen. The target object can be a specific area on the screen, such as a function icon, navigation information area, or multimedia playback control area; it can also be the vehicle assistant itself. For example, when the driver looks at the vehicle assistant, the system recognizes this gaze and moves the vehicle assistant to the position the driver is looking at, thus achieving a gaze-following interaction effect. The specific type of the target object can be flexibly set according to the actual application scenario, and this application does not impose any restrictions on it.

[0065] In one specific implementation, refer to Figure 3 As shown, step S13 (collecting the driver's line-of-sight information about the target object and determining the target display position based on the line-of-sight information) may specifically include the following steps S131 to S132: S131. Obtain the driver's gaze duration and gaze direction towards the target object.

[0066] Specifically, an image acquisition device, such as an infrared camera, installed inside the vehicle can capture the driver's facial image. An infrared camera emits infrared light to illuminate the driver's face and receives the reflected infrared light to form a facial image. Because infrared light is unaffected by changes in ambient visible light, this method can operate stably under various lighting conditions, including daytime, nighttime, and tunnels.

[0067] After acquiring the facial image, the driver's eye feature information and / or head posture information are extracted from the facial image.

[0068] Then, based on eye feature information and / or head posture information, the driver's gaze direction on the vehicle's infotainment screen is determined. The gaze direction can be characterized by the azimuth angle or coordinates of the point of contact of the line of sight on the screen.

[0069] Specifically, a mapping model between eye features / head posture and screen coordinates can be established in advance through a calibration process. During calibration, the driver is required to sequentially gaze at multiple known calibration points on the screen, thereby collecting eye feature and head posture data corresponding to each calibration point. A mapping function is then established using interpolation or regression methods. In actual use, the real-time extracted eye feature and / or head posture data are substituted into this mapping function to calculate the driver's gaze direction on the vehicle's infotainment screen.

[0070] Finally, the driver's gaze point on the vehicle's infotainment screen is determined based on the gaze direction, and the duration of the gaze at that point is recorded as the gaze duration.

[0071] It should be noted that the duration of fixation refers to the length of time the driver continuously focuses on that fixation point, which can be recorded and accumulated based on timestamps.

[0072] S132. If the gaze duration is greater than or equal to a preset duration threshold, determine the target display position according to the gaze direction.

[0073] Specifically, the preset duration threshold is a pre-configured time parameter used to distinguish between conscious gaze and unconscious squinting. If the gaze duration is shorter than the preset duration threshold, the gaze may simply be a brief visual pause by the driver while observing the road or scanning the screen, and should not be misjudged as an intention to transfer to the virtual image; if the gaze duration reaches or exceeds the preset duration threshold, it can be considered a valid gaze intention.

[0074] For example, the preset duration threshold can be set to any value between 500 milliseconds and 2 seconds. For instance, the preset duration threshold can be set to 800 milliseconds, where a driver's gaze at a specific location is considered a valid gaze if the gaze lasts for 800 milliseconds. This duration setting balances response speed and the need to prevent accidental touches: a threshold that is too short may lead to frequent accidental triggers, while a threshold that is too long will affect the smoothness of the interaction.

[0075] Furthermore, when the gaze duration reaches or exceeds a preset duration threshold, the target display position is determined based on the gaze direction.

[0076] Specifically, the mapping position of the gaze direction on the vehicle's infotainment screen can be determined first, and then this mapping position can be set as the target display position. For example, if the gaze direction is the central area of ​​the screen, the central area of ​​the screen can be set as the target display position; if the gaze direction is the left side of the screen, the left side of the screen can be set as the target display position.

[0077] In this embodiment, the target display location can be the coordinates of the gaze point corresponding to the gaze direction itself. For example, if the driver is looking at a point on the screen, the vehicle assistant can be directly moved to that gaze point location.

[0078] Optionally, the target display position can be a preset anchor position in the screen area corresponding to the gaze direction. For example, when the driver gazes at the left side of the screen, the vehicle assistant can be moved to the center point or a preset anchor point slightly above that area, rather than being moved directly to the gaze point coordinates, thus making the position of the virtual image more regular and stable.

[0079] After determining the target display location, the vehicle assistant is moved from the current display location to the target display location. In one specific implementation, the vehicle assistant can be controlled to move from the current display location to the target display location using a second animation effect. The migration speed of the second animation effect is less than that of the first animation effect, so as to create a smooth and fluid visual transition effect and enhance the interactive immersion in low-speed or parked scenarios.

[0080] Based on the above S131 and S132, by collecting the driver's gaze duration and gaze direction for the target object, the target display position is determined only when the gaze duration meets the condition based on the gaze direction. This can effectively filter out the driver's unconscious and brief eye scanning behavior, avoid frequent and meaningless changes in the virtual assistant's position, improve the stability of the interaction, make the virtual assistant's position adjustment fit the driver's real gaze intention, and optimize the human-computer interaction experience.

[0081] This application embodiment monitors the real-time speed of the target vehicle and the current display position of the vehicle assistant. When the vehicle speed exceeds a preset threshold and the current display position is in an unsafe area, the vehicle assistant is automatically moved to a safe display area. This effectively avoids the problem of the virtual image obscuring the core field of view of the screen and thus distracting the driver's attention when driving at high speeds, ensuring driving safety. At the same time, when the vehicle speed is less than or equal to the preset threshold, the vehicle assistant is moved to the target display position desired by the driver based on the driver's line of sight information. This enables convenient interaction in low-speed or parking scenarios, where the virtual image position can be adjusted without manual operation. By integrating real-time vehicle speed and line of sight information for control, the display strategy of the virtual assistant is adaptively switched according to different driving states, improving the interactive convenience and intelligence level of the in-vehicle virtual assistant while ensuring driving safety.

[0082] As an extension and refinement of the above embodiments, refer to Figure 4 As shown, S131 in the above embodiment can be further refined into S1311-S1314 as follows: S1311. Acquire the driver's facial image using an image acquisition device.

[0083] In this step, the driver's face is captured in real time by an image acquisition device installed inside the vehicle.

[0084] Specifically, the image acquisition device can be a camera installed in the dashboard area in front of the steering wheel, near the rearview mirror, or inside the A-pillar. In a preferred embodiment, the image acquisition device is an infrared camera. Because infrared cameras can actively emit infrared light and receive reflected signals, they are unaffected by changes in ambient visible light. Therefore, they can work stably in various scenarios with drastic light changes, such as daytime, nighttime, and tunnel entrances / exits, ensuring the continuity and reliability of facial image acquisition.

[0085] It should be noted that the image acquisition device continuously acquires facial images of the driver at a preset frame rate, such as 30 frames per second, forming a video stream. The vehicle's infotainment system acquires this video stream in real time and performs subsequent processing on each frame.

[0086] S1312. Extract the driver's eye feature information and / or head posture information based on the facial image.

[0087] Specifically, the eye feature information may include: pupil position, pupil diameter, iris edge information, iris offset, eyelid opening and closing degree, eye corner position, and eyeball center position; these eye features can reflect the direction of eyeball rotation and the angle of gaze, and are the core basis for determining the direction of gaze.

[0088] In one implementation, a deep learning-based facial landmark detection algorithm can be used to process the facial image. For example, a multi-task cascaded convolutional network (MTCNN) or a heatmap regression model (such as Hourglass Network or HRNet) can be used to process the driver's facial image in real time to extract key facial feature points. These key facial feature points include at least key points in the eye region (such as the inner and outer corners of the eyes, and the edges of the upper and lower eyelids) and the center point of the pupil. Based on these key points, further calculations can be made of eye feature information such as pupil position and eyelid opening / closing.

[0089] Specifically, the head pose information may include: head pitch angle, head yaw angle, and head roll angle. The head pose information can be obtained by extracting global feature points of the face (such as the tip of the nose, the left and right corners of the eyes, the left and right corners of the mouth, etc.), and based on the correspondence between the two-dimensional positions of these key points and the standard three-dimensional face model, the three-dimensional rotation angle of the head can be obtained by using the Perspective-n-Point (PnP) algorithm.

[0090] It should be noted that while individual eye features or head posture can be used to determine gaze direction, combining eye features with head posture yields higher accuracy. In one implementation, eye feature information and head posture information can be extracted simultaneously and fused for a more accurate gaze direction. Furthermore, head posture information is particularly important when the driver's head is turned (e.g., to observe road conditions to the side), as relying solely on eye features may not accurately determine the driver's actual line of sight in such situations.

[0091] S1313. Determine the driver's gaze direction on the vehicle screen based on the eye feature information and / or the head posture information.

[0092] It should be noted that the gaze direction refers to the direction in which the driver's line of sight lands on the vehicle's infotainment screen. Specifically, the gaze direction is characterized by the azimuth of the line of sight on the screen, such as the line of sight pointing to the upper left, center, or lower right area of ​​the screen; the gaze direction is also characterized by the two-dimensional coordinates of the point where the line of sight intersects the screen, which are the coordinates of the gaze point in subsequent steps.

[0093] In this embodiment, the conversion from eye features / head posture to gaze direction can be achieved through a pre-established mapping model. The process of establishing the mapping model includes a calibration phase and a usage phase.

[0094] During the calibration phase, the driver is required to sequentially look at multiple known calibration points on the vehicle screen (such as the top left corner, top right corner, center, bottom left corner, bottom right corner, etc., nine calibration points in total). Eye feature information and head posture information of the driver when looking at each calibration point are collected respectively, and the correspondence between feature data and screen coordinates is established.

[0095] In some embodiments, algorithms such as multinomial regression, support vector regression, or neural networks can be used to train a mapping model based on the above correspondence.

[0096] Then, during the usage phase, the real-time extracted eye feature information and / or head posture information are input into the mapping model, which can calculate the driver's gaze direction on the vehicle screen in real time.

[0097] In another implementation, a geometric calculation method based on an eye model can also be used to determine the gaze direction. Specifically, based on the positional relationship between the pupil center and the corneal reflector, combined with the intrinsic and extrinsic parameters of the camera, the direction vector of the gaze in three-dimensional space can be calculated through three-dimensional geometric reconstruction. Then, the intersection point of this direction vector with the plane where the vehicle screen is located can be calculated, thereby obtaining the mapped position of the gaze direction on the screen.

[0098] It should be noted that if the driver is wearing glasses or sunglasses, it may affect the accurate extraction of eye features. In this case, head posture information can be relied upon first to estimate the gaze direction, or multispectral imaging technology can be used to penetrate lens reflections to ensure the robustness of gaze direction calculation.

[0099] S1314. Determine the driver's gaze point on the vehicle screen based on the gaze direction, and record the duration of the gaze at the gaze point as the gaze duration.

[0100] Specifically, the gaze point refers to the specific location where the driver's line of sight lands on the vehicle's infotainment screen. When the gaze direction is represented by the two-dimensional coordinates of the intersection of the line of sight and the screen, the coordinates can be directly determined as the gaze point. When the gaze direction is represented by an azimuth angle, the coordinates of the intersection of the line of sight and the screen need to be calculated based on the azimuth angle and the spatial geometric relationship between the driver's eyes and the screen.

[0101] After determining the fixation point, the duration of the fixation at that point is recorded; the duration of fixation refers to the length of time the driver continuously looks at that fixation point; the time when the driver's fixation point changes can be recorded using a timestamp. When the driver continuously looks at the same position, the duration of fixation at that point can be continuously accumulated; when the driver's fixation point moves, the timing of the previous fixation point is stopped, and the duration of fixation at the new fixation point is started.

[0102] For example, gaze tracking can be performed frame by frame. If the gaze point calculated from multiple consecutive frames is located in the same area on the screen (e.g., within a preset radius centered on that location), it is determined that the driver is continuously looking at that location. The number of consecutive frames at that location is accumulated, and the dwell time is calculated by combining this with the frame rate. For example, if images are captured at a frame rate of 30 frames per second, and the gaze point is located in the same area for 30 consecutive frames, the dwell time at that gaze point is determined to be 1 second.

[0103] Therefore, in this embodiment of the application, a gaze point timer can be maintained, which is started when the gaze point begins to stay still and reset when the gaze point moves.

[0104] Considering the existence of minute eye movements (such as microsaccades, nystagmus, and other physiological eye movements), even when focusing on the same object, the fixation point may fluctuate within a small range. To address this, a tolerance radius can be set (e.g., a 50-pixel radius centered on the fixation point). When the fixation point changes within this tolerance radius, it is still considered the same fixation point, and the timer continues to increment, thus avoiding frequent timer resets due to minute eye movements.

[0105] It should be noted that the duration of the gaze is used for subsequent comparison with a preset duration threshold. When the duration reaches or exceeds the preset duration threshold, it is determined to be a valid gaze intent, which can trigger the virtual avatar's position migration. The preset duration threshold can be configured according to actual needs, for example, it can be set to any value between 500 milliseconds and 2 seconds. Comparing the duration with the preset duration threshold can effectively distinguish between conscious gaze and unconscious scanning, avoiding incorrect responses to unconscious gaze scanning by the driver, thereby improving the accuracy of interaction and user experience.

[0106] This application embodiment uses an image acquisition device installed inside the vehicle to acquire facial images of the driver and extract eye feature information and / or head posture information to determine the driver's gaze direction, gaze point, and duration on the vehicle screen, thereby accurately acquiring the driver's gaze information. Combining eye features and head posture information to determine the gaze direction can improve the accuracy of gaze tracking, while recording the duration of the gaze point and comparing it with a preset duration threshold can effectively distinguish between the driver's conscious gaze and unconscious scanning, avoiding frequent movement of the virtual image due to accidental gaze touch, and providing accurate and reliable input basis for the subsequent position migration of the virtual assistant based on gaze information.

[0107] It should be noted that the preset threshold includes a first vehicle speed threshold and a second vehicle speed threshold, and the first vehicle speed threshold is less than the second vehicle speed threshold.

[0108] By setting two different speed thresholds, the vehicle's driving state can be divided into three speed ranges: high speed, medium speed, and low speed. Different speed ranges correspond to different virtual assistant display and control strategies.

[0109] It should be noted that the specific values ​​of the first and second speed thresholds can be configured according to actual needs. In one embodiment, the first speed threshold can be set to 20 km / h, and the second speed threshold can be set to 60 km / h, thus forming a three-level speed range division of "low speed (0-20 km / h), medium speed (20-60 km / h), and high speed (>60 km / h)". In another embodiment, the first speed threshold can also be set to 30 km / h, and the second speed threshold can be set to 80 km / h to adapt to high-speed driving scenarios such as highways. The values ​​of the first and second speed thresholds can be flexibly adjusted according to the road speed limits of different countries and regions, vehicle types, and user driving habits, and this application does not impose any restrictions on this.

[0110] Specifically, different speed ranges correspond to different virtual assistant display control strategies, including the following three cases: (1) When the real-time vehicle speed is greater than the second vehicle speed threshold and the current display position is in a non-safe display area, the vehicle assistant is moved from the current display position to a safe display area in the vehicle screen.

[0111] If the real-time vehicle speed is greater than the second vehicle speed threshold, it indicates that the vehicle is traveling at high speed. In this state, if the current display position of the vehicle assistant is located in a non-safe display area (e.g., the center area of ​​the screen), a forced migration is triggered, moving the vehicle assistant from the current display position to a safe display area on the vehicle screen.

[0112] It is understandable that drivers need to focus most of their attention on the road ahead when driving at high speeds. If a virtual avatar is displayed for an extended period in the center of the screen or the core area of ​​the driver's field of vision, it can easily interfere with the driver's vision and increase the risk of driver distraction. Therefore, when the vehicle speed exceeds a second speed threshold, the virtual avatar is forcibly moved to a safe display area (such as the corner of the screen) to avoid obstructing the driver's view. Simultaneously, if the real-time vehicle speed exceeds the second speed threshold, the collection of gaze information can be stopped, or the response function for relocating the vehicle's assistant based on gaze information can be locked, to completely prevent the virtual avatar from being moved back to the center of the screen due to accidental touches of gaze information while driving at high speeds.

[0113] It should also be noted that if the real-time vehicle speed exceeds the second vehicle speed threshold, and the vehicle assistant's current display position is already within the safe display area, then there is no need to perform a migration action again. The current display position's coordinates can be compared with the coordinate range of the safe display area to determine if it is already within the safe area. If it is, the current display position remains unchanged, avoiding unnecessary resource waste caused by migration actions.

[0114] (2) When the real-time vehicle speed is less than or equal to the second vehicle speed threshold and greater than the first vehicle speed threshold, the driver’s line of sight information to the target object is collected at the first collection frequency, and the first duration is used as the preset duration threshold.

[0115] When the real-time vehicle speed is less than or equal to the second speed threshold and greater than the first speed threshold, it indicates that the vehicle is in a medium-speed driving state. In this state, the vehicle still has a certain speed, and the driver's attention still needs to be mainly focused on the road ahead, but the risk of driver distraction is reduced compared to high-speed driving. Therefore, the gaze interaction function is not completely turned off, but rather the response sensitivity of gaze information is reduced.

[0116] Specifically, the line-of-sight information is collected at a first collection frequency, and a first duration is used as a preset duration threshold. The first collection frequency is lower than the normal collection frequency (i.e., the second collection frequency in low-speed mode), and the first duration is greater than the default value of the preset duration threshold (i.e., the second duration in low-speed mode).

[0117] Reducing the sensitivity of gaze information response refers to making the system "less sensitive" to driver gaze input by lowering the frequency of gaze information acquisition and extending the gaze duration threshold required to trigger a shift. In one implementation, during medium-speed driving, driver gaze information can be acquired at a lower frequency (e.g., 10 frames / second) and the gaze duration threshold can be set to a longer duration (e.g., 1200 milliseconds). Position shifting is only initiated in response to the driver's gaze information when the driver gazes at a specific location on the screen for an extended period (more than 1200 milliseconds). This prevents occasional glances from being misinterpreted as interaction intentions, effectively reducing driver interference caused by frequent gaze input responses.

[0118] For example, when a vehicle is traveling at 40 km / h on an auxiliary road of an urban expressway, the driver may occasionally glance at the navigation map to confirm the route. If a high sensitivity is applied to every glance, the virtual avatar may move frequently, interfering with the driver's normal information acquisition. By reducing the acquisition frequency to 10 frames per second and extending the gaze duration threshold to 1200 milliseconds, brief glances by the driver will not trigger a migration action. Migration is only performed when the driver intentionally gazes at a location for more than 1.2 seconds, thus reducing unnecessary interference while maintaining a certain level of interactivity.

[0119] (3) When the real-time vehicle speed is less than or equal to the first vehicle speed threshold, the driver’s line of sight to the target object is collected at the second collection frequency, and the second duration is used as the preset duration threshold.

[0120] When the real-time vehicle speed is less than or equal to the first vehicle speed threshold, it indicates that the vehicle is in a low-speed driving state or a stopped state. In this state, the vehicle speed is low or completely stationary, and the driver's attention can be relatively more focused on the vehicle's infotainment screen, resulting in lower safety risks associated with eye-tracking interaction. Therefore, it responds to eye-tracking information with high sensitivity, providing users with a smooth eye-tracking interaction experience.

[0121] Specifically, the gaze information is collected at a second collection frequency, and a second duration is used as the preset duration threshold. The second collection frequency is higher than the first collection frequency, and the second duration is shorter than the first duration; that is, the second collection frequency is the normal collection frequency, and the second duration is the default valid determination duration.

[0122] In one implementation, at low speeds or when stationary, the driver's gaze information is acquired at a normal frequency of 30 frames per second, and the gaze duration threshold is set to 800 milliseconds. The driver only needs to gaze at the target position on the screen for about 0.8 seconds to respond and complete the virtual avatar's migration; the second acquisition frequency can also be set to 60 frames per second to provide a smoother gaze tracking effect; the second duration can be set to 500 milliseconds to achieve a more sensitive response.

[0123] For example, when the driver is preparing to stop the vehicle and the speed drops to 10 km / h, the driver wants to move the virtual avatar from the lower right corner of the screen to the center for voice interaction. At this time, the driver looks at the central area of ​​the screen for about 0.8 seconds, and the effective gaze is detected and the virtual avatar is moved to the center of the screen. The interaction process is smooth and natural, without the driver having to manually drag or issue voice commands.

[0124] It should also be noted that the correspondence between the first collection frequency being lower than the second collection frequency and the first duration being longer than the second duration can prioritize safety when driving at high speeds, moderately suppress eye-tracking interaction when driving at medium speeds (reducing the collection frequency and extending the judgment duration), and provide a sensitive eye-tracking interaction experience when driving at low speeds or when parked (normal collection frequency and default judgment duration), thus achieving a balance between driving safety and convenient interaction.

[0125] This application embodiment divides the vehicle driving state into three intervals: high speed, medium speed, and low speed by setting a first vehicle speed threshold and a second vehicle speed threshold, and configures different virtual assistant display control strategies for each interval. Compared with a single threshold control scheme, it can more precisely adapt to the safety and interaction needs at different vehicle speeds, and retain the convenience and usability of eye-to-eye interaction to the greatest extent while ensuring driving safety.

[0126] In some embodiments, S12 (moving the vehicle assistant from the current display position to the safe display area) in the above embodiments further includes: controlling the vehicle assistant to move from the current display position to the safe display area with a first animation effect. S13 (moving the vehicle assistant from the current display position to the target display position) further includes: controlling the vehicle assistant to move from the current display position to the target display position with a second animation effect.

[0127] The migration speed of the first animation effect is greater than that of the second animation effect.

[0128] Understandably, the first animation corresponds to a scenario where a virtual avatar is forcibly moved to a safe display area while traveling at high speed. In this scenario, the vehicle is traveling at a high speed, and the driver's attention needs to be focused on the road ahead as much as possible. The shorter the movement of the virtual avatar on the screen, the less visual interference it causes to the driver, and the faster the driver can quickly return their attention to the driving task. Therefore, the first animation uses a relatively fast migration speed, allowing the virtual avatar to quickly leave the center area of ​​the screen or its current location, completing the migration to the safe display area in the shortest possible time.

[0129] In the second animation, the virtual avatar is moved to the target display position based on line-of-sight information in a low-speed or stationary state. In this scenario, the vehicle is traveling at a low speed or is stationary, and the driver's attention to the in-vehicle screen is relatively high. Line-of-sight interaction is itself a proactive interaction behavior. At this time, using a slower migration speed can provide the driver with a smoother, more natural, and more comfortable visual transition experience, enhancing the immersion and enjoyment of the interaction.

[0130] Specifically, the migration speed of the first animation effect is 2 to 5 times that of the second animation effect. For example, the first animation effect can complete the migration from the current display position to the safe display position within 0.3 seconds, while the second animation effect can complete the migration within 0.8 to 1.2 seconds. In another embodiment, the migration speed of the first animation effect can also be set to 3 times that of the second animation effect to achieve rapid migration at high speeds while maintaining a smooth visual transition effect at low speeds.

[0131] Furthermore, the first and second animation effects can employ different animation curves to achieve differentiated migration speeds. For example, the first animation effect can use a fade-in / fade-out curve, making the virtual character slower at the start and end of the migration and faster in the middle phase, creating a crisp and clean migration effect. The second animation effect can use a fade-out curve, making the virtual character faster at the start of the migration and gradually decelerating as it approaches the target location, creating a gentle and smooth landing effect.

[0132] It should be noted that the first and second animation effects can also differ in their transparency changes. For example, in the first animation effect, the transparency of the virtual avatar during migration can decrease rapidly (e.g., from 100% opaque to 30% opaque) to further reduce the interference of the virtual avatar on the driver's vision at high speeds. In the second animation effect, the transparency of the virtual avatar during migration can remain constant or change slowly to maintain the visibility and recognizability of the virtual avatar during interaction.

[0133] It should be added that the first animation effect can also be combined with the shrinking effect of the virtual avatar. For example, during high-speed migration, as the virtual avatar moves from its current display position to the safe display area, its size gradually shrinks (e.g., from its original size to 60% of its original size) to fit the limited display space of the safe display area (such as a corner of the screen). During low-speed migration, the second animation effect can be combined with the enlargement or restoration effect of the virtual avatar. For example, after the virtual avatar migrates to the target display position, its size gradually returns to its original size to enhance the sense of interactive feedback.

[0134] During the execution of the first and second animation effects, the interactive state of the vehicle assistant instance remains continuously synchronized during the migration process. That is, regardless of the animation effect used for the virtual avatar's migration, the virtual avatar's voice interaction state, facial expression state, and dialogue context will not be interrupted or lost due to position migration. For example, during the migration of the virtual avatar from the center of the screen to a corner, if the virtual avatar is performing a voice broadcast, the voice broadcast continues without interruption due to position movement; the virtual avatar's facial expressions and movement states also remain continuous, without any jumps or resets. Through the above methods, the continuity and consistency of the virtual assistant's interactive experience can be guaranteed, avoiding negative impacts on the user experience caused by interaction interruptions due to position migration.

[0135] In this embodiment, by configuring different migration animations for high-speed migration scenarios and low-speed line-of-sight interaction scenarios, the migration process of the virtual avatar can be matched with the current driving state: rapid migration is completed when driving at high speeds to ensure safety, while smooth migration is performed when driving at low speeds or when stopped to improve the interactive experience. This differentiated animation control method takes into account both driving safety and interactive comfort, further enhancing the user experience of the in-vehicle virtual assistant in different driving scenarios.

[0136] After the above migration is completed, the display form of the virtual image needs to be adapted and adjusted according to the display area characteristics of the target location. This includes the following steps: Obtain the preset display parameters of the vehicle assistant corresponding to the safe display area or the target display position; adjust the display mode of the vehicle assistant based on the preset display parameters.

[0137] In some embodiments, different display areas on the vehicle infotainment screen have different spatial characteristics and display constraints. For example, the safety display area is usually located in the corner or status bar area of ​​the screen, where the available space is relatively limited. If the virtual image is displayed in this area at its original size, it may exceed the boundary, obscure system information, or overlap with other display elements. The central area of ​​the screen usually has ample space, but if the virtual image is too small when displayed in the central area, it will be detrimental to user observation and interaction. Therefore, after the virtual image is moved to the target location, the display form of the virtual image needs to be adapted and adjusted according to the preset display parameters corresponding to the target location, so that the virtual image can be presented with an appropriate size, range of motion, and display style in the new display location.

[0138] It should be noted that the preset display parameters refer to a set of virtual avatar display parameters pre-configured for each host area or display position. Different display positions can correspond to different preset display parameters, so that the display form of the virtual avatar can adapt to the display characteristics of different positions.

[0139] Specifically, the preset display parameters include at least size parameters; these size parameters define the width and height of the virtual avatar when displayed at the target display location. For example, the safety display area is located in the lower right corner of the screen, where space is limited. Therefore, the virtual avatar needs to be displayed at a smaller size (e.g., reduced to 50% or 60% of its original size) to fully display the virtual avatar within the limited corner space, while avoiding obscuring other important system or navigation information. However, after migrating the virtual avatar to the center of the screen or the driver's desired target display location based on line-of-sight information, since the central area has ample space and is used for active interaction with the driver, the virtual avatar can be displayed at its original size or a larger size (e.g., enlarged to 120% of its original size) to enhance its recognizability and interactivity.

[0140] The preset display parameters may also include motion amplitude parameters; these parameters define the magnitude of the virtual avatar's movements when displayed at the target display location. In the safe display area, the virtual avatar is located in the background at the edge of the screen, its primary function being to maintain its presence rather than actively attract the user's attention. Therefore, the virtual avatar can be controlled to display small movements in the safe display area, such as slight breathing or simple blinking, to reduce its visual activity in the corner areas and avoid attracting the driver's attention and distracting them from driving due to excessively large movements. In the target display location (e.g., the center of the screen), the virtual avatar is in the foreground interactive position and needs to engage in voice dialogue or facial expression interaction with the driver. Therefore, the virtual avatar can be controlled to display larger movements, such as waving or nodding, to enhance the expressiveness of the interaction and the emotional companionship experience.

[0141] The preset display parameters may also include a transparency parameter. When driving at high speed, after the virtual avatar moves to a safe display area, its transparency can be adjusted from completely opaque (100%) to semi-transparent (e.g., 50% or 30%) to further reduce its visual salience in edge areas, ensuring that the driver's peripheral vision is not easily distracted by the virtual avatar while focusing on the road ahead. Conversely, at low speeds or when stationary, after the virtual avatar moves to the target display position, its transparency can be restored to completely opaque to ensure clarity and recognizability during interaction.

[0142] The preset display parameters may also include display style parameters. For example, in the safety display area, the virtual avatar can be controlled to be displayed in a static icon mode or a simplified mode (e.g., only the avatar's head, silhouette, or thumbnail is displayed) to minimize its obstruction of screen content and interference with the driver's field of vision. In the target display location (i.e., the interaction location), the virtual avatar can be controlled to be displayed in its full form (including full-body or half-body image, dynamic background lighting effects, particle effects, etc.) to create an immersive interactive experience.

[0143] It should be noted that the preset display parameters corresponding to the safety display area can be pre-configured in the vehicle system configuration file, and different safety display area positions can be configured with their own preset display parameters.

[0144] For example, as a vehicle travels at 80 km / h on a highway, a virtual avatar rapidly migrates from the center of the screen to the safety display area in the lower right corner with a first animation effect. After the migration, preset display parameters corresponding to the lower right safety display area are obtained, the virtual avatar is shrunk to 50% of its original size, and its transparency is adjusted to 60%. Simultaneously, the virtual avatar displays gentle breathing movements to maintain low visual activity at the edge of the display area. When the vehicle leaves the highway and enters a parking lot, the speed decreases to 10 km / h, and the driver focuses on the center of the screen. The virtual avatar then smoothly migrates from the lower right corner to the center of the screen with a second animation effect. After the migration, preset display parameters corresponding to the center of the screen are obtained, the virtual avatar is restored to its original size, its transparency is adjusted back to 100%, and the virtual avatar displays vivid expressions and larger movements to actively engage with the driver.

[0145] This application embodiment adjusts the display form of the virtual image according to the preset display parameters corresponding to the target location after the virtual image migrates to the target location. This allows the size, range of motion, transparency, and display style of the virtual image to match the display characteristics of the current location. This not only ensures the low-interference characteristics of the safe display area but also enhances the visual expressiveness and interactive experience of the interactive location, further optimizing the adaptability of the in-vehicle virtual assistant in different display locations.

[0146] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods according to the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method.

[0147] Embodiments of this application also provide a control device for a vehicle-mounted assistant, which corresponds one-to-one with the method claims. Figure 4 This is a schematic diagram of the control device 500 for the vehicle assistant provided in this disclosure, as shown below. Figure 5As shown, the device 500 of this embodiment includes: The monitoring unit 51 is used to monitor the real-time speed of the target vehicle and the current display position of the vehicle assistant on the vehicle screen; Control unit 52 is configured to, when the real-time vehicle speed is greater than a preset threshold and the current display position is in a non-safe display area, Move the vehicle assistant from its current display position to a secure display area on the vehicle screen. The control unit 52 is further configured to collect the driver's line-of-sight information on the target object when the real-time vehicle speed is less than or equal to the preset threshold, and determine the target display position based on the line-of-sight information, so as to migrate the vehicle assistant from the current display position to the target display position.

[0148] As an optional implementation of this application, the control unit 52 is specifically used to acquire the driver's gaze duration and gaze direction on the target object; and when the gaze duration is greater than or equal to a preset duration threshold, to determine the target display position based on the gaze direction.

[0149] As an optional implementation of this application, the control unit 52 is specifically used to acquire the driver's facial image through an image acquisition device; extract the driver's eye feature information and / or head posture information based on the facial image; determine the driver's gaze direction on the vehicle screen based on the eye feature information and / or the head posture information; determine the driver's gaze point on the vehicle screen based on the gaze direction, and record the duration of the gaze point as the gaze duration.

[0150] As an optional implementation of this application, the preset threshold includes a first vehicle speed threshold and a second vehicle speed threshold, wherein the first vehicle speed threshold is less than the second vehicle speed threshold; the control unit 52 is specifically used to: when the preset threshold includes a first vehicle speed threshold and a second vehicle speed threshold, wherein the first vehicle speed threshold is less than the second vehicle speed threshold; when the real-time vehicle speed is greater than the second vehicle speed threshold and the current display position is in a non-safe display area, move the vehicle assistant from the current display position to a safe display area in the vehicle screen; when the real-time vehicle speed is less than or equal to the second vehicle speed threshold and greater than the first vehicle speed threshold, collect the driver's gaze information on the target object at a first collection frequency and use a first duration as the preset duration threshold; when the real-time vehicle speed is less than or equal to the first speed threshold, collect the driver's gaze information on the target object at a second collection frequency and use a second duration as the preset duration threshold; wherein the first collection frequency is less than the second collection frequency, and the first duration is greater than the second duration.

[0151] As an optional implementation of this application, the control unit 52 is further configured to control the vehicle assistant to migrate from the current display position to the safe display area with a first animation effect; and control the vehicle assistant to migrate from the current display position to the target display position with a second animation effect; wherein the migration speed of the first animation effect is greater than the migration speed of the second animation effect.

[0152] As an optional implementation of this application, the control unit 52 is further configured to stop collecting the driver's line-of-sight information about the target object when the real-time vehicle speed is greater than the preset threshold.

[0153] As an optional embodiment of this application, the control unit 52 is specifically used to acquire driver seat side information and determine a target safety display area from multiple safety display areas based on the driver seat side information; and to move the vehicle assistant from the current display position to the target safety display area.

[0154] As an optional implementation of this application, the control unit 52 is further configured to obtain preset display parameters of the vehicle assistant corresponding to the safe display area or the target display position; and adjust the display form of the vehicle assistant based on the preset display parameters.

[0155] Embodiments of this application also provide an electronic device, including a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to perform the steps in any of the above-described vehicle assistant control method embodiments.

[0156] Embodiments of this application also provide a computer-readable storage medium storing a computer program, wherein the computer program is configured to execute the steps in any of the above-described vehicle-mounted assistant control method embodiments when running.

[0157] In one exemplary embodiment, the aforementioned computer-readable storage medium may include, but is not limited to, various media capable of storing computer programs, such as a USB flash drive, read-only memory (ROM), random access memory (RAM), portable hard disk, magnetic disk, or optical disk.

[0158] The embodiments of this application also provide a computer program product, which includes a computer program that, when executed by a processor, implements the steps in any of the above-described vehicle assistant control method embodiments.

[0159] Embodiments of this application also provide another computer program product, including a non-volatile computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps in any of the above-described vehicle assistant control method embodiments.

[0160] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0161] The control method, electronic device, and vehicle for a vehicle-mounted assistant provided in this application have been described in detail above. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the embodiments above are only intended to help understand the method and core ideas of this application. It should be noted that those skilled in the art can make various improvements and modifications to this application without departing from its principles, and these improvements and modifications also fall within the protection scope of the claims of this application.

Claims

1. A control method for a vehicle infotainment assistant, characterized in that, include: Monitor the real-time speed of the target vehicle and the current display position of the vehicle assistant on the vehicle screen; When the real-time vehicle speed is greater than a preset threshold and the current display position is in a non-safe display area, Move the vehicle assistant from its current display position to a secure display area on the vehicle screen. When the real-time vehicle speed is less than or equal to the preset threshold, the driver's line of sight to the target object is collected, and the target display position is determined based on the line of sight information, so as to move the vehicle assistant from the current display position to the target display position.

2. The method according to claim 1, characterized in that, The step of collecting the driver's line-of-sight information about the target object and determining the target display position based on the line-of-sight information includes: The driver's gaze duration and gaze direction towards the target object are obtained; If the gaze duration is greater than or equal to a preset duration threshold, the target display position is determined based on the gaze direction.

3. The method according to claim 1, characterized in that, The step of obtaining the driver's gaze duration and gaze direction towards the target object includes: The driver's facial image is captured by an image acquisition device; Extract the driver's eye feature information and / or head posture information from the facial image; Based on the eye feature information and / or the head posture information, determine the driver's gaze direction on the vehicle screen; The driver's gaze point on the vehicle's infotainment screen is determined based on the gaze direction, and the duration of the gaze at the gaze point is recorded as the gaze duration.

4. The method according to claim 2, characterized in that, The preset threshold includes a first vehicle speed threshold and a second vehicle speed threshold, wherein the first vehicle speed threshold is less than the second vehicle speed threshold. If the real-time vehicle speed is greater than the second vehicle speed threshold and the current display position is in a non-safe display area, the vehicle assistant will be moved from the current display position to a safe display area on the vehicle screen. When the real-time vehicle speed is less than or equal to the second vehicle speed threshold and greater than the first vehicle speed threshold, the step of collecting the driver's gaze information on the target object includes: collecting the driver's gaze information on the target object at a first collection frequency, and using a first duration as the preset duration threshold. When the real-time vehicle speed is less than or equal to the first vehicle speed threshold, the step of collecting the driver's gaze information on the target object includes: collecting the driver's gaze information on the target object at a second collection frequency, and using a second duration as the preset duration threshold; Wherein, the first acquisition frequency is less than the second acquisition frequency, and the first duration is greater than the second duration.

5. The method according to claim 1, characterized in that, Moving the vehicle assistant from its current display location to the safe display area includes: Control the vehicle assistant to move from the current display position to the safety display area with a first animation effect; The step of migrating the vehicle assistant from the current display position to the target display position includes: Control the vehicle assistant to migrate from the current display position to the target display position using a second animation effect; The migration speed of the first animation effect is greater than that of the second animation effect.

6. The method according to claim 1, characterized in that, The method further includes: When the real-time vehicle speed exceeds the preset threshold, the collection of the driver's line-of-sight information regarding the target object is stopped.

7. The method according to claim 1, characterized in that, Moving the vehicle assistant from its current display position to a secure display area on the vehicle screen includes: Obtain driver seat side information, and determine a target safety display area from multiple safety display areas based on the driver seat side information; Move the vehicle assistant from its current display location to the target safety display area.

8. The method according to claim 1, characterized in that, After moving the vehicle assistant from its current display position to a safe display area on the vehicle screen, or moving the vehicle assistant from its current display position to the target display position, the method further includes: Obtain the preset display parameters of the vehicle assistant corresponding to the safety display area or the target display position; Based on the preset display parameters, the display mode of the vehicle assistant is adjusted.

9. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor for executing the computer program to implement the control method of the vehicle assistant as described in any one of claims 1-8.

10. A vehicle, characterized in that, It includes a memory and a processor, wherein the memory stores executable program code, and the processor is used to call and execute the executable program code to implement the vehicle assistant control method as described in any one of claims 1-8.

Images