Interaction method and device based on head-up display equipment

Abstract

The embodiment of the invention provides an interaction method and device based on head-up display equipment, and the method comprises the steps: determining a first target position of an eye box icon in a head-up display image according to the first eye position data of a target user, and determining the size of the eye box icon according to the head-up display image, and the eye box icon is displayed in the head-up display image according to the first target position and size of the eye box icon, so that the target user can realize the movement of the eye box icon in the head-up display image according to the sight change, and the target user can realize interaction with the head-up display image by controlling the eye box icon. The driving experience of the user is improved, and the safety of the driving process is improved.

Description

Technical Field

[0001] The present application relates to the technical field of head-up display, and in particular to an interaction method and device based on a head-up display device. Background Art

[0002] At present, the Augmented Reality-Head Up Display (AR-HUD) is basically a one-way projection display of the optical machine. The driver sees the virtual image projected by the optical machine through the windshield. There is a natural barrier to human-computer interaction and lacks interactivity with people. People cannot feedback what they see to the optical machine, so there is a lack of suitable AR-HUD human-computer interaction methods. Traditional human-computer interaction requires lowering or tilting the head to operate the multimedia host to input signals, which has certain safety risks, and cross-screen operation, and poor interactive experience. Summary of the invention

[0003] An embodiment of the present application provides an interaction method and apparatus based on a head-up display device, which determines the position and size of an eye box icon in a head-up display image through the eye position data of a target user and a head-up display image, and displays it in the head-up display image according to the position and size, so that the target user can interact with the head-up display image by controlling the eye box icon.

[0004] In a first aspect, an embodiment of the present application provides an interaction method based on a head-up display device, the method comprising:

[0005] Get the first-view data of target users;

[0006] A first target position of the eye box icon is determined according to the first eye position data, and a size of the eye box icon is determined according to the head-up display image, wherein the first target position is used to represent the position of the eye box icon in the head-up display image.

[0007] The eye box icon is displayed in the head-up display image according to the size of the eye box icon and the first target position.

[0008] It can be seen that in the embodiment of the present application, the first target position of the eye box icon in the head-up display image is determined according to the first eye position data of the target user, the size of the eye box icon is determined according to the head-up display image, and the eye box icon is displayed in the head-up display image according to the first target position and size of the eye box icon. In this way, the target user can move the eye box icon in the head-up display image according to the change of line of sight, and then the target user can interact with the head-up display image by controlling the eye box icon, thereby improving the user's driving experience and improving the safety of the driving process.

[0009] In a feasible embodiment, the first eye position data includes the pupil center position and the gaze direction, and the first target position and target state of the eye box icon are determined according to the first eye position data, including: determining the eye coordinates of the target user in the vehicle coordinate system according to the pupil center position, and determining the eye box coordinates according to the gaze direction, the virtual image plane coordinates in the vehicle coordinate system and the eye coordinates; determining the first target position according to the eye box coordinates and the virtual image plane coordinates.

[0010] In an embodiment of the present application, the first target position of the eye box icon is determined in the vehicle coordinate system according to the pupil center position and the gaze direction in the first eye position data, which can improve the accuracy of determining the first target position.

[0011] In a feasible embodiment, the method further includes: determining a target state of an eye box icon according to the first eye position data; and adjusting the head-up display image according to the eye box icon and the target state.

[0012] In an embodiment of the present application, the head-up display image is adjusted through the eye box icon and the target attribute of the eye box icon, thereby realizing the interaction between the target user and the head-up display device, improving the user's driving experience, and improving the safety of adjusting the head-up display image.

[0013] Specifically, in a feasible embodiment, the first eye position data includes eye status information, and determining the target state of the eye box icon according to the first eye position data includes: determining the target state according to the eye status information.

[0014] In an embodiment of the present application, the target state is determined based on the state information of the eyes, so as to facilitate the user to interact with the head-up display device.

[0015] Specifically, in a feasible embodiment, the head-up display image is adjusted according to the eye box icon and the target state, including: when it is detected that the eye box icon intersects with the target button icon in the head-up display image, and the target state is a selected state, obtaining the second eye position data of the target user; and adjusting the head-up display image according to the function of the target button and the second eye position data.

[0016] In an embodiment of the present application, the target button is selected by the eye box icon, and the head-up display image is adjusted according to the second eye position data and the function of the target button, which facilitates the user to adjust the head-up display image, while improving the user's driving experience and the safety of adjusting the head-up display image.

[0017] Specifically, in a feasible embodiment, when the function of the target button is height adjustment, the head-up display image is adjusted according to the function of the target button and the second eye position data, including: determining the second target position of the eye box icon according to the second eye position data; and adjusting the height of the head-up display image according to the second target position.

[0018] In the embodiment of the present application, the height of the head-up display image is adjusted according to the height of the target user's line of sight, which improves the flexibility of adjusting the height of the head-up display image and improves the user's driving experience.

[0019] In addition, in a feasible embodiment, when the function of the target button is brightness adjustment, the head-up display image is adjusted according to the function of the target button and the second eye position data, including: determining the second target position of the eye box icon according to the second eye position data; if the target coordinate corresponding to the second target position is greater than the target coordinate corresponding to the first target position, then increasing the brightness of the head-up display image, the target coordinate being either the horizontal coordinate or the vertical coordinate; if the target coordinate corresponding to the second target position is smaller than the target coordinate corresponding to the first target position, then reducing the brightness of the head-up display image.

[0020] In the embodiment of the present application, the brightness of the head-up display image is adjusted according to the first target position and the second target position, which improves the flexibility of adjusting the height of the head-up display image and improves the driving experience of the user.

[0021] In addition, in a feasible embodiment, when the function of the target button is to adjust the tilt angle, the head-up display image is adjusted according to the function of the target button and the second eye position data, including: determining the second target position of the eye box icon according to the second eye position data; if the target coordinate corresponding to the second target position is greater than the target coordinate corresponding to the first target position, then increasing the tilt angle of the head-up display image, the target coordinate being any one of the horizontal coordinate and the vertical coordinate; if the target coordinate corresponding to the second target position is smaller than the target coordinate corresponding to the first target position, then reducing the tilt angle of the head-up display image.

[0022] In the embodiment of the present application, the tilt angle of the head-up display image is adjusted according to the first target position and the second target position, which improves the flexibility of adjusting the tilt angle of the head-up display image and improves the driving experience of the user.

[0023] In addition, in a feasible embodiment, if the head-up display image is distorted during display, the head-up display image is adjusted according to the eye box icon and the target state, including: adjusting the coordinates of at least one of the four vertices of the head-up display image according to the first target position and the target state of the eye box icon; for vertex A among the at least one vertex, when it is detected that the first target position is located at the target vertex position in the head-up display image and the target state is a selected state, obtaining second eye position data of the target user; and adjusting the coordinates of vertex A according to the second eye position data.

[0024] In the embodiment of the present application, the coordinates of the vertices of the head-up display image are adjusted after being selected, thereby achieving the picture distortion adjustment of the head-up display image, improving the flexibility of the picture distortion adjustment of the head-up display image, and improving the driving experience of the user.

[0025] In a second aspect, an embodiment of the present application provides an interactive device based on a head-up display device, the device comprising:

[0026] An acquisition unit, used for acquiring first glance position data of a target user;

[0027] a determining unit, configured to determine a first target position of the eye box icon according to the first eye position data, and to determine a size of the eye box icon according to the head-up display image, wherein the first target position is used to represent a position of the eye box icon in the head-up display image,

[0028] The display unit is used to display the eye box icon in the head-up display image according to the size of the eye box icon and the first target position.

[0029] In a third aspect, an embodiment of the present application provides an electronic device, which includes a processor and a memory, wherein the processor is coupled to the memory, and the processor calls executable program code stored in the memory to execute, for example, some or all of the steps described in any method of the first aspect.

[0030] In a fourth aspect, an embodiment of the present application provides a vehicle, which includes the interaction device of the second aspect or the electronic device shown in the third aspect.

[0031] In a fifth aspect, an embodiment of the present application provides a computer-readable storage medium, in which electronic data is stored. When the electronic data is executed by a processor, it is used to execute the electronic data to implement some or all of the steps described in the first aspect of the embodiment of the present application.

[0032] In a sixth aspect, an embodiment of the present application provides a computer program product, wherein the computer program product includes a non-transitory computer-readable storage medium storing a computer program, and the computer program is operable to cause a computer to execute some or all of the steps described in the first aspect of the embodiment of the present application. The computer program product may be a software installation package. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0034] Figure 1 A schematic diagram of the structure of an interactive system provided in an embodiment of the present application;

[0035] Figure 2 A schematic diagram of a flow chart of an interaction method based on a head-up display device provided in an embodiment of the present application;

[0036] Figure 3 A schematic diagram of the structure of a vehicle coordinate system provided in an embodiment of the present application;

[0037] Figure 4 A schematic diagram of the structure of a head-up display image provided in an embodiment of the present application;

[0038] Figure 5 An example diagram of the tilt angle of a head-up display image provided in an embodiment of the present application;

[0039] Figure 6 A schematic diagram of the structure of another head-up display image provided in an embodiment of the present application;

[0040] Figure 7a It is a block diagram of the functional units of an interactive device based on a head-up display device provided in an embodiment of the present application;

[0041] Figure 7b It is a block diagram of the functional units of another interactive device based on a head-up display device provided in an embodiment of the present application;

[0042] Figure 8 It is a structural block diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0043] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0044] The terms "first", "second", etc. in the specification and claims of this application and the above-mentioned drawings are used to distinguish different objects, rather than to describe a specific order. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps is not limited to the listed steps, but optionally includes steps that are not listed, or optionally includes other steps inherent to these processes, methods, products or devices.

[0045] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0046] See also Figure 1 , Figure 1 A schematic diagram of the structure of an interactive system provided in an embodiment of the present application is shown in FIG. Figure 1 As shown, the interactive system 100 includes an eye box tracking device 101 and a head-up display device 102 .

[0047] The eye box tracking device 101 is used to obtain the eye position data of the driver. The eye box tracking device 101 can be a driver monitoring system (DMS) camera. The driver monitoring system is a real-time system based on the driver's facial image processing to study the driver's physical and mental conditions. The driver's state can be detected from eyelid closure, blinking, gaze direction, yawning and head movement. Obtaining eye position data through the DMS camera is to capture the driver's image through the DMS camera, and according to the human eye tracking algorithm, the driver's face dynamic recognition, eye area interception, expression state monitoring, dynamic pupil center positioning, and human eye vector gaze direction. The eye position data is mainly the data obtained by identifying the eye area in the face, including the state of the eyes, the position of the pupil center, the eye gaze direction, etc.

[0048] The head-up display device 102 is used to project important driving information such as speed and navigation onto the windshield in front of the driver. After the eye box tracking device 101 obtains the driver's eye position data, the driver's eye position data is sent to the head-up display device 102. The head-up display device 102 is used to generate a corresponding eye box icon according to the eye position data, and display the eye box icon on the head-up display image projected onto the windshield. The eye box icon can move with the driver's line of sight, and the driver can adjust the head-up display image according to the eye box icon. This improves the driver's driving experience and improves the safety of operation.

[0049] Based on this, an embodiment of the present application provides an interaction method based on a head-up display device, and the embodiment of the present application is described in detail below in conjunction with the accompanying drawings.

[0050] See also Figure 2 , Figure 2 A flowchart of an interactive method based on a head-up display device provided in an embodiment of the present application is shown in FIG. Figure 2As shown, the method comprises the following steps:

[0051] Step 201, obtaining the first eye position data of the target user.

[0052] Among them, the first-eye position data of the target user is obtained by shooting with a DMS camera.

[0053] Step 202: determine a first target position of an eye box icon according to the first eye position data, and determine a size of the eye box icon according to the head-up display image.

[0054] The first target position is used to represent the position of the eye box icon in the head-up display image. The size of the eye box icon is determined according to the size of the head-up display image, and can also be determined according to the size of the target button in the head-up display image, for example, the size of the eye box icon is a first preset ratio of the size of the head-up display image, or the size of the eye box icon is a second preset ratio of the size of the target button.

[0055] In a feasible embodiment, the first eye position data includes the pupil center position and the gaze direction, and the first target position and target state of the eye box icon are determined according to the first eye position data, including: determining the eye coordinates of the target user in the vehicle coordinate system according to the pupil center position, and determining the eye box coordinates according to the gaze direction, the virtual image plane coordinates in the vehicle coordinate system and the eye coordinates; determining the first target position according to the eye box coordinates and the virtual image plane coordinates.

[0056] After the DMS camera captures the pupil center position in the first eye position data, it will generate corresponding coordinates in the coordinate system corresponding to the DMS camera. At this time, the coordinates need to be converted into coordinates in the vehicle coordinate system. After the eye coordinates in the vehicle coordinate system are determined, a vector image can be determined according to the eye gaze direction of the target user. The starting point of the vector image is the eye coordinates in the vehicle coordinate system. The eye box coordinates can be determined according to the vector image and the virtual image plane coordinates in the vehicle coordinate system. The virtual image plane is the image formed after the head-up display device is projected on the windshield. The virtual image plane coordinates are the coordinates of the image, and the eye box coordinates are in the virtual image plane coordinates. Therefore, the first target position of the eye box icon can be determined according to the eye box coordinates and the virtual image plane coordinates.

[0057] For example, see Figure 3 , Figure 3 A schematic diagram of a vehicle coordinate structure provided in an embodiment of the present application, such as Figure 3 As shown, Figure 3Including eye coordinates E (x1, y1, z1), and a virtual image plane 301, coordinates P (x6, y6, z6) are the coordinates of a virtual image point formed by the eye coordinates E (x1, y1, z1) according to the virtual image plane 301, and the eye box coordinates determined on the virtual image plane 301 by a vector image formed by the eye coordinates E (x1, y1, z1) according to the gaze direction are H (x7, y7, z7).

[0058] In an embodiment of the present application, the first target position of the eye box icon is determined in the vehicle coordinate system according to the pupil center position and the gaze direction in the first eye position data, which can improve the accuracy of determining the first target position.

[0059] Step 203: display the eye box icon in the head-up display image according to the size of the eye box icon and the first target position.

[0060] Among them, to display the eye box icon in the head-up display image, it is necessary to determine the position of the corresponding eye box icon in the source image plane according to the position of the eye box icon in the virtual image plane, and generate the corresponding eye box icon in the source image plane according to the image generation unit, and finally project the source image plane to obtain the head-up display image. The source image plane corresponds to the virtual image plane, and the source image plane specifically refers to the plane image generated by the image generation unit in the head-up display device before projection.

[0061] In a feasible embodiment, the method further includes: determining a target state of an eye box icon according to the first eye position data; and adjusting the head-up display image according to the eye box icon and the target state.

[0062] Among them, in the head-up display image, the attributes of the eye box icon are associated with the target state, that is, when the target state changes, the attributes of the eye box icon will also change, and the attributes of the eye box icon include color, size, shape, animation effect, etc. The target state includes a selected state and an unselected state. According to the eye box icon and the target state, the interaction between the target user and the head-up display device is realized in the head-up display image, so as to adjust the head-up display image.

[0063] In an embodiment of the present application, the head-up display image is adjusted through the eye box icon and the target attribute of the eye box icon, thereby realizing the interaction between the target user and the head-up display device, improving the user's driving experience, and improving the safety of adjusting the head-up display image.

[0064] Specifically, in a feasible embodiment, the first eye position data includes eye status information, and determining the target state of the eye box icon according to the first eye position data includes: determining the target state according to the eye status information.

[0065] Among them, the state information of the eyes refers to the relative state of the eyelid and the eyeball in the eye position, including the number of blinks, the degree of eye opening and closing, etc. Exemplarily, if the target user blinks twice in succession, the target state changes from the unselected state to the selected state, and when the target user blinks twice again, the target state changes from the selected state to the unselected state; or, if the target user's eyes are in a fully open state, the target state is the unselected state, and if the target user's eyes are in a half-open state or other incompletely open state, the target state is the selected state; in addition, it can also be based on a combination of the two, for example, if the user's eyes are in a half-open state or other incompletely open state after blinking twice in succession, the target state is the selected state, and if the user's eyes are in a fully open state after blinking twice in succession again, the target state is the unselected state.

[0066] In an embodiment of the present application, the target state is determined based on the state information of the eyes, so as to facilitate the user to interact with the head-up display device.

[0067] Specifically, in a feasible embodiment, the head-up display image is adjusted according to the eye box icon and the target state, including: when it is detected that the eye box icon intersects with the target button icon in the head-up display image, and the target state is a selected state, obtaining the second eye position data of the target user; and adjusting the head-up display image according to the function of the target button and the second eye position data.

[0068] Among them, the intersection between the eye box icon and the target button needs to be larger than the preset threshold of the size of the eye box icon or the target button icon itself. When the eye box icon and the target button icon in the head-up display image have an intersection, the target button icon is selected by adjusting the target state, and the function corresponding to the target button is adjusted by the subsequent line of sight movement of the target user. In other words, the eye box icon is similar to the mouse in the computer display interface, the head-up display image is similar to the computer display interface, the target button icon is similar to the function icon in the computer display interface, and the selection of the target button icon by the eye box icon is similar to the selection of the function icon in the computer display interface by the mouse.

[0069] In an embodiment of the present application, the target button is selected by the eye box icon, and the head-up display image is adjusted according to the second eye position data and the function of the target button, which facilitates the user to adjust the head-up display image, while improving the user's driving experience and the safety of adjusting the head-up display image.

[0070] Specifically, in a feasible embodiment, when the function of the target button is height adjustment, the head-up display image is adjusted according to the function of the target button and the second eye position data, including: determining the second target position of the eye box icon according to the second eye position data; and adjusting the height of the head-up display image according to the second target position.

[0071] Among them, when the function of the target button is height adjustment, after the target user selects the target button, the display height of the head-up display image will change with the change of the height of the target user's line of sight. Specifically, after the second target position of the eye box icon corresponding to the target user's line of sight is determined according to the second eye position data, the height of the head-up display image will be determined according to the second target position.

[0072] In the embodiment of the present application, the height of the head-up display image is adjusted according to the height of the target user's line of sight, which improves the flexibility of adjusting the height of the head-up display image and improves the user's driving experience.

[0073] In addition, in a feasible embodiment, when the function of the target button is brightness adjustment, the head-up display image is adjusted according to the function of the target button and the second eye position data, including: determining the second target position of the eye box icon according to the second eye position data; if the target coordinate corresponding to the second target position is greater than the target coordinate corresponding to the first target position, then increasing the brightness of the head-up display image, the target coordinate being either the horizontal coordinate or the vertical coordinate; if the target coordinate corresponding to the second target position is smaller than the target coordinate corresponding to the first target position, then reducing the brightness of the head-up display image.

[0074] Wherein, unlike the height adjustment of the head-up display image, when the function of the target button is brightness adjustment, it is also necessary to adjust the brightness according to the relative position between the first target position and the second target position. For example, refer to Figure 4 , Figure 4 A schematic diagram of a head-up display image provided in an embodiment of the present application is shown in FIG. Figure 4 As shown, Figure 4 The head-up display image 410 is included, and the head-up display image 410 includes a first eye box icon 401 determined according to the first eye position data, and the first eye box icon 401 is in a selected state. At this time, the second eye position data is obtained, as shown in the head-up display image 420, and the horizontal coordinate of the second eye box icon 402 corresponding to the second eye position data is greater than the horizontal coordinate of the first eye box icon 401, then the brightness of the head-up display image is increased according to the brightness adjustment ratio corresponding to the relative position between the second eye box icon 402 and the first eye box icon 401. Among them, the coordinates corresponding to the first eye box icon 401 are the coordinates corresponding to the first target position determined according to the first eye position data, and the coordinates corresponding to the second eye box icon 402 are the coordinates corresponding to the second target position determined according to the second eye position data.

[0075] In the embodiment of the present application, the brightness of the head-up display image is adjusted according to the first target position and the second target position, which improves the flexibility of adjusting the height of the head-up display image and improves the driving experience of the user.

[0076] In addition, in a feasible embodiment, when the function of the target button is to adjust the tilt angle, the head-up display image is adjusted according to the function of the target button and the second eye position data, including: determining the second target position of the eye box icon according to the second eye position data; if the target coordinate corresponding to the second target position is greater than the target coordinate corresponding to the first target position, then increasing the tilt angle of the head-up display image, the target coordinate being any one of the horizontal coordinate and the vertical coordinate; if the target coordinate corresponding to the second target position is smaller than the target coordinate corresponding to the first target position, then reducing the tilt angle of the head-up display image.

[0077] The adjustment of the tilt angle of the head-up display image is similar to the adjustment of the brightness of the head-up display image, that is, both are adjusted according to the relative position between the first target position and the second target position, and no further details are given here. Figure 5 , Figure 5 An example diagram of the tilt angle of a head-up display image provided in an embodiment of the present application is shown in FIG. Figure 5 As shown, Figure 5 The head-up display image 510 is included in the side view, wherein the head-up display image 510 is tilted at an angle as shown in 501 .

[0078] In the embodiment of the present application, the tilt angle of the head-up display image is adjusted according to the first target position and the second target position, which improves the flexibility of adjusting the tilt angle of the head-up display image and improves the driving experience of the user.

[0079] In addition, in a feasible embodiment, if the head-up display image is distorted during display, the head-up display image is adjusted according to the eye box icon and the target state, including: adjusting the coordinates of at least one of the four vertices of the head-up display image according to the first target position and the target state of the eye box icon; for vertex A among the at least one vertex, when it is detected that the first target position is located at the target vertex position in the head-up display image and the target state is a selected state, obtaining second eye position data of the target user; and adjusting the coordinates of vertex A according to the second eye position data.

[0080] When the head-up display image is distorted, the user can control the movement of the eye box icon according to the change of sight to drag the four vertices of the head-up display image to change the coordinates of the vertices of the head-up display image, thereby achieving image distortion adjustment. For example, refer to Figure 6 , Figure 6 A schematic diagram of another head-up display image provided in an embodiment of the present application is shown in FIG. Figure 6 As shown, Figure 6Including a head-up display image 610, wherein the head-up display image 610 is distorted. At this time, the eye box icon 601 is controlled to move to the upper left vertex position of the head-up display image 610, and the target state is changed to a selected state. The eye box icon 601 is controlled to move to the left until it moves to the position of the eye box icon 602. At this time, the head-up display image 610 is adjusted according to the position of the eye box icon 602. The adjusted image is shown in the head-up display image 620.

[0081] In the embodiment of the present application, the coordinates of the vertices of the head-up display image are adjusted after being selected, thereby achieving the picture distortion adjustment of the head-up display image, improving the flexibility of the picture distortion adjustment of the head-up display image, and improving the driving experience of the user.

[0082] It can be seen that in the embodiment of the present application, the first target position of the eye box icon in the head-up display image is determined according to the first eye position data of the target user, the size of the eye box icon is determined according to the head-up display image, and the eye box icon is displayed in the head-up display image according to the first target position and size of the eye box icon. In this way, the target user can move the eye box icon in the head-up display image according to the change of line of sight, and then the target user can interact with the head-up display image by controlling the eye box icon, thereby improving the user's driving experience and improving the safety of the driving process.

[0083] In accordance with the above-mentioned embodiments, please refer to Figure 7a , Figure 7a is a block diagram of the functional units of an interactive device based on a head-up display device provided in an embodiment of the present application, such as Figure 7a As shown, the interactive device 70 based on the head-up display device includes:

[0084] An acquisition unit 701 is used to acquire the first eye position data of a target user;

[0085] The determining unit 702 is used to determine a first target position of the eye box icon according to the first eye position data, and to determine a size of the eye box icon according to the head-up display image, wherein the first target position is used to represent a position of the eye box icon in the head-up display image.

[0086] The display unit 703 is configured to display the eye box icon in the head-up display image according to the size of the eye box icon and the first target position.

[0087] In a feasible embodiment, the first eye position data includes a pupil center position and a gaze direction, and a determination unit 702 is used to determine a first target position and a target state of an eye box icon according to the first eye position data, including: determining the eye coordinates of the target user in the vehicle coordinate system according to the pupil center position, and determining the eye box coordinates according to the gaze direction, the virtual image plane coordinates in the vehicle coordinate system, and the eye coordinates; determining the first target position according to the eye box coordinates and the virtual image plane coordinates.

[0088] In a feasible embodiment, the device further includes: a determination unit 702, configured to determine a target state of an eye box icon according to the first eye position data; and an adjustment unit 707, configured to adjust the head-up display image according to the eye box icon and the target state.

[0089] In a feasible embodiment, the first eye position data includes eye status information, and the determination unit 702, for determining the target state of the eye box icon according to the first eye position data, includes: determining the target state according to the eye status information.

[0090] In a feasible embodiment, the adjustment unit 707 is used to adjust the head-up display image according to the eye box icon and the target state, including: when it is detected that the eye box icon and the target button icon in the head-up display image have an intersection, and the target state is a selected state, obtaining the second eye position data of the target user; adjusting the head-up display image according to the function of the target button and the second eye position data.

[0091] In a feasible embodiment, when the function of the target button is height adjustment, the adjustment unit 707 is used to adjust the head-up display image according to the function of the target button and the second eye position data, including: determining the second target position of the eye box icon according to the second eye position data; adjusting the height of the head-up display image according to the second target position.

[0092] In a feasible embodiment, when the function of the target button is brightness adjustment, the adjustment unit 707 is used to adjust the head-up display image according to the function of the target button and the second eye position data, including: determining the second target position of the eye box icon according to the second eye position data; if the target coordinate corresponding to the second target position is greater than the target coordinate corresponding to the first target position, then increasing the brightness of the head-up display image, the target coordinate is any one of the horizontal coordinate and the vertical coordinate; if the target coordinate corresponding to the second target position is less than the target coordinate corresponding to the first target position, then reducing the brightness of the head-up display image.

[0093] In a feasible embodiment, when the function of the target button is tilt angle adjustment, the adjustment unit 707 is used to adjust the head-up display image according to the function of the target button and the second eye position data, including: determining the second target position of the eye box icon according to the second eye position data; if the target coordinate corresponding to the second target position is greater than the target coordinate corresponding to the first target position, then increasing the tilt angle of the head-up display image, the target coordinate is any one of the horizontal coordinate and the vertical coordinate; if the target coordinate corresponding to the second target position is less than the target coordinate corresponding to the first target position, then reducing the tilt angle of the head-up display image.

[0094] In a feasible embodiment, if the head-up display image is distorted when being displayed, the adjustment unit 707 is used to adjust the head-up display image according to the eye box icon and the target state, including: adjusting the coordinates of at least one vertex among the four vertices of the head-up display image according to the first target position and the target state of the eye box icon; for vertex A among the at least one vertex, when it is detected that the first target position is located at the target vertex position in the head-up display image and the target state is a selected state, obtaining second eye position data of the target user; and adjusting the coordinates of vertex A according to the second eye position data.

[0095] It can be understood that since the method embodiment and the device embodiment are different presentation forms of the same technical concept, the content of the method embodiment part in this application should be synchronously adapted to the device embodiment part and will not be repeated here.

[0096] In the case of integrated units, such as Figure 7b As shown, Figure 7b This is a block diagram of the functional units of another interactive device based on a head-up display device provided in an embodiment of the present application. Figure 7b In the embodiment, the interactive device 71 based on the head-up display device includes: a processing module 712 and a communication module 711. The processing module 712 is used to control and manage the actions of the interactive device based on the head-up display device, for example, the steps of the acquisition unit 701, the determination unit 702, the display unit 703 and the adjustment unit 707, and / or other processes for executing the technology described herein. The communication module 711 is used to support the interaction between the interactive device based on the head-up display device and other devices. Figure 7b As shown, the interactive device 71 based on the head-up display device may further include a storage module 713, and the storage module 713 is used to store program codes and data of the interactive device based on the head-up display device.

[0097] Among them, the processing module 712 can be a processor or a controller, for example, a central processing unit (CPU), a general processor, a digital signal processor (DSP), an ASIC, an FPGA or other programmable logic device, a transistor logic device, a hardware component or any combination thereof. It can implement or execute various exemplary logic blocks, modules and circuits described in conjunction with the disclosure of this application. The processor can also be a combination that implements a computing function, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, and the like. The communication module 711 can be a transceiver, an RF circuit or a communication interface, and the like. The storage module 713 can be a memory.

[0098] Among them, all relevant contents of each scene involved in the above method embodiment can be referred to the functional description of the corresponding functional module, and will not be repeated here. The above-mentioned head-up display device-based interactive device 71 can execute the above-mentioned Figure 2 The interactive method based on the head-up display device is shown.

[0099] The above embodiments can be implemented in whole or in part by software, hardware, firmware or any other combination. When implemented using software, the above embodiments can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions or computer programs. When a computer instruction or computer program is loaded or executed on a computer, a process or function according to an embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. Computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, computer instructions can be transmitted from one website site, computer, server or data center to another website site, computer, server or data center by wired or wireless means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center containing one or more available media sets. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium. The semiconductor medium can be a solid-state hard disk.

[0100] Figure 8 is a structural block diagram of an electronic device provided in an embodiment of the present application. Figure 8As shown, the electronic device 800 may include one or more of the following components: a processor 801, and a memory 802 coupled to the processor 801, wherein the memory 802 may store one or more computer programs, and the one or more computer programs may be configured to implement the methods described in the above embodiments when executed by one or more processors 801.

[0101] The processor 801 may include one or more processing cores. The processor 801 uses various interfaces and lines to connect the various parts of the entire electronic device 800, and executes various functions and processes data of the electronic device 800 by running or executing instructions, programs, code sets or instruction sets stored in the memory 802, and calling data stored in the memory 802. Optionally, the processor 801 can be implemented in at least one hardware form of digital signal processing (Digital Signal Processing, DSP), field programmable gate array (Field-Programmable Gate Array, FPGA), and programmable logic array (Programmable Logic Array, PLA). The processor 801 can integrate one or a combination of a central processing unit (Central Processing Unit, CPU), a graphics processor (Graphics Processing Unit, GPU) and a modem. It is understandable that the above-mentioned modem may not be integrated into the processor 801, and may be implemented separately through a communication chip.

[0102] The memory 802 may include a random access memory (RAM) or a read-only memory (ROM). The memory 802 may be used to store instructions, programs, codes, code sets or instruction sets. The memory 802 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for implementing at least one function (such as a touch function, a sound playback function, an image playback function, etc.), instructions for implementing the above-mentioned various method embodiments, etc. The data storage area may also store data created by the electronic device 800 during use, etc.

[0103] It is understandable that the electronic device 800 may include more or fewer structural elements than those in the above structural block diagram, for example, including a power module, physical buttons, a WiFi (Wireless Fidelity) module, a speaker, a Bluetooth module, a sensor, etc., which are not limited here.

[0104] An embodiment of the present application provides a vehicle, which includes the electronic device shown in the above embodiment or the interactive system of the above embodiment.

[0105] An embodiment of the present application provides a computer-readable storage medium, wherein program data is stored in the computer-readable storage medium. When the program data is executed by a processor, the program data is used to execute part or all of the steps of any one of the interactive methods based on a head-up display device recorded in the above method embodiments.

[0106] The present application also provides a computer program product, which includes a non-transitory computer-readable storage medium storing a computer program, and the computer program is operable to cause a computer to execute some or all of the steps of any of the interactive methods based on a head-up display device as described in the above method embodiments. The computer program product may be a software installation package.

[0107] It should be noted that, for any of the aforementioned method embodiments of the interactive method based on the head-up display device, for the sake of simplicity, they are all expressed as a series of action combinations, but those skilled in the art should be aware that this application is not limited by the order of the actions described, because according to this application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions involved are not necessarily required by this application.

[0108] Although the present application is described herein in conjunction with various embodiments, in the process of implementing the claimed application, those skilled in the art may understand and implement other variations of the disclosed embodiments by viewing the drawings, the disclosure, and the appended claims. In the claims, the word "comprising" does not exclude other components or steps, and "a" or "one" does not exclude multiple situations. Certain measures are recorded in different dependent claims, but this does not mean that these measures cannot be combined to produce good results.

[0109] A person of ordinary skill in the art can understand that all or part of the steps in the various methods of any of the above-mentioned method embodiments of the interactive method based on the head-up display device can be completed by instructing related hardware through a program, and the program can be stored in a computer-readable memory, and the memory can include: a flash drive, a read-only memory (English: Read-Only Memory, abbreviated as: ROM), a random access memory (English: Random Access Memory, abbreviated as: RAM), a disk or an optical disk, etc.

[0110] The embodiments of the present application are introduced in detail above. Specific examples are used herein to illustrate the principles and implementation methods of an interactive method and device based on a head-up display device of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea; at the same time, for a general technician in this field, according to the idea of ​​an interactive method and device based on a head-up display device of the present application, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

[0111] The present application is described with reference to the flowcharts and / or block diagrams of the methods, hardware products, and computer program products of the embodiments of the present application. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of the processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0112] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0113] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0114] It can be understood that any product that is controlled or configured to execute the processing method of the flowchart described in the method embodiment of the interactive method based on a head-up display device in the present application, such as the terminal and computer program product in the above flowchart, belongs to the category of related products described in the present application.

[0115] Obviously, those skilled in the art can make various changes and modifications to the interactive method and device based on the head-up display device provided by the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalent technologies, the present application is also intended to include these modifications and variations.

Claims

1. An interactive method based on a head-up display device, characterized in that: The method comprises: Get the first-view data of target users; determining a first target position of an eye box icon according to the first eye position data, and determining a size of the eye box icon according to a head-up display image, wherein the first target position is used to represent a position of the eye box icon in the head-up display image, The eye box icon is displayed in the heads-up display image according to a size of the eye box icon and a first target position.

2. The method according to claim 1, characterized in that: The first eye position data includes a pupil center position and a gaze direction, and determining a first target position of an eye box icon according to the first eye position data includes: Determining eye coordinates of the target user in a vehicle coordinate system according to the pupil center position, and determining eye box coordinates according to the gaze direction, virtual image plane coordinates in the vehicle coordinate system, and the eye coordinates; The first target position is determined according to the eye box coordinates and the virtual image plane coordinates.

3. The method according to claim 1 or 2, characterized in that: The method further comprises: determining a target state of the eye box icon according to the first eye position data; The head-up display image is adjusted according to the eye box icon and the target state.

4. The method according to claim 3, characterized in that The first eye position data includes eye status information, and determining the target status of the eye box icon according to the first eye position data includes: The target state is determined according to the state information of the eyes.

5. The method according to claim 3, characterized in that: The adjusting the head-up display image according to the eye box icon and the target state includes: When it is detected that the eye box icon intersects with the target button icon in the head-up display image, and the target state is a selected state, obtaining second eye position data of the target user; The head-up display image is adjusted according to the function of the target button and the second eye position data.

6. The method according to claim 5, characterized in that When the function of the target button is height adjustment, adjusting the head-up display image according to the function of the target button and the second eye position data includes: determining a second target position of the eye box icon according to the second eye position data; The height of the heads-up display image is adjusted according to the second target position.

7. The method according to claim 5, characterized in that When the function of the target button is brightness adjustment, adjusting the head-up display image according to the function of the target button and the second eye position data includes: determining a second target position of the eye box icon according to the second eye position data; If the target coordinate corresponding to the second target position is greater than the target coordinate corresponding to the first target position, then increasing the brightness of the head-up display image, the target coordinate being any one of a horizontal coordinate and a vertical coordinate; If the target coordinates corresponding to the second target position are smaller than the target coordinates corresponding to the first target position, the brightness of the head-up display image is reduced.

8. The method according to claim 5, characterized in that When the function of the target button is tilt angle adjustment, the adjusting the head-up display image according to the function of the target button and the second eye position data includes: determining a second target position of the eye box icon according to the second eye position data; If the target coordinate corresponding to the second target position is greater than the target coordinate corresponding to the first target position, increasing the tilt angle of the head-up display image, the target coordinate being any one of a horizontal coordinate and a vertical coordinate; If the target coordinates corresponding to the second target position are smaller than the target coordinates corresponding to the first target position, the tilt angle of the head-up display image is reduced.

9. The method according to claim 3, characterized in that: The adjusting the head-up display image according to the eye box icon and the target state includes: adjusting the coordinates of at least one vertex among the four vertices of the head-up display image according to the first target position and the target state of the eye box icon; For vertex A in the at least one vertex, when it is detected that the first target position is located at the target vertex position in the head-up display image and the target state is a selected state, obtaining second eye position data of the target user; The coordinates of the vertex A are adjusted according to the second eye position data.

10. An electronic device, characterized in that: The method comprises a processor and a memory, wherein the processor is coupled to the memory, and the processor calls executable program code stored in the memory to execute the method according to any one of claims 1 to 9.

11. A vehicle, characterized in that: The electronic device comprising claim 10.

12. A computer-readable storage medium, characterized in that: A computer program for electronic data exchange is stored, wherein the computer program enables a computer to execute the method according to any one of claims 1 to 9.

Images