Method and device for adjusting interface elements and electronic equipment

Abstract

The invention discloses an interface element adjusting method and device and electronic equipment. The method comprises the steps of obtaining current user state information of a user and display parameters of interface elements on a target screen; determining a visual parameter of the user relative to the target screen according to the user state information, and generating an adjustment parameter of each interface element according to the visual parameter and the display parameter; and adjusting the display parameters of the interface elements based on the adjustment parameters. According to the embodiment of the invention, the purposes of dynamically adjusting the display parameters of the interface elements based on the visual parameters of the user relative to the screen and optimizing the information readability and the interaction comfort can be achieved.

Description

TECHNICAL FIELD

[0001] Embodiments of the present application relate to the technical field of interface element adjustment, and in particular to an interface element adjustment method and device and electronic equipment. BACKGROUND

[0002] Vehicle-mounted human-machine interaction (HMI) is a bridge for information transmission, instruction operation and function feedback between a driver or passenger and a vehicle system through hardware devices, software logic and interaction design.

[0003] At present, most HMI interfaces adopt a fixed layout and cannot be dynamically adjusted according to the real-time state (such as visual angle, seat position, attention concentration) of a driver, resulting in fixed and rigid information presentation, poor information readability and interaction comfort, and limited operation experience. SUMMARY

[0004] Embodiments of the present application provide an interface element adjustment method, device and electronic equipment, which can be used to dynamically adjust the display parameters of interface elements based on the visual parameters of a user relative to a screen, and optimize information readability and interaction comfort. The technical solution is as follows: In one aspect, the present application provides an interface element adjustment method, which comprises: obtaining user state information of a user at present and display parameters of each interface element on a target screen; determining a visual parameter of the user relative to the target screen according to the user state information, and generating adjustment parameters of each interface element according to the visual parameter and the display parameters; adjusting the display parameters of each interface element based on the adjustment parameters.

[0005] In another aspect, the present application provides an interface element adjustment device, which comprises: an information obtaining module, configured to obtain user state information of a user at present and display parameters of each interface element on a target screen; a parameter generating module, configured to determine a visual parameter of the user relative to the target screen according to the user state information, and generate adjustment parameters of each interface element according to the visual parameter and the display parameters; a parameter adjusting module, configured to adjust the display parameters of each interface element based on the adjustment parameters.

[0006] In yet another aspect, the present application provides an electronic equipment, which comprises a driver monitoring system, a three-dimensional scene camera device, a data analysis module and an interface management module, wherein, The driver monitoring system is configured to acquire a user image of a user driving a vehicle when the user is watching a target screen, and analyze the user image to obtain user state information of the user; The three-dimensional scene camera is configured to acquire display parameters of each interface element on a vehicle screen of the vehicle; The data analysis module is configured to determine visual parameters of the user relative to the vehicle screen according to the user state information, and generate adjustment parameters of each interface element according to the visual parameters and the display parameters; The interface management module is configured to adjust the display parameters of each interface element on the vehicle screen based on the adjustment parameters.

[0007] In another aspect, the embodiments of the present application provide a computer readable storage medium, wherein the storage medium stores at least one instruction, at least one program, a code set or an instruction set, and the at least one instruction, the at least one program, the code set or the instruction set are loaded and executed by a processor to implement the method in the above aspect.

[0008] In another aspect, the embodiments of the present application provide a computer program product, when the computer program product is executed, the computer program product is used to execute the method in the above aspect.

[0009] In the technical scheme provided by the embodiments of the present application, by acquiring the current user state information of the user and the visual parameters of the user relative to the screen, and generating the adjustment parameters of the interface element according to the visual parameters and the current display parameters of the interface element to adjust the display parameters of the interface element, the dynamic adjustment of the display parameters of the interface element can be realized, and the information readability of the interface element in the screen and the comfort of human-computer interaction can be optimized. BRIEF DESCRIPTION OF DRAWINGS

[0010] Figure 1 A step flowchart of an interface element adjustment method provided by the embodiments of the present application; Figure 2 A step flowchart of another interface element adjustment method provided by the embodiments of the present application; Figure 3 A step flowchart of a user state information acquisition method provided by the embodiments of the present application; Figure 4 A step flowchart of a tilt angle adjustment method provided by the embodiments of the present application; Figure 5 A step flowchart of a proportional position adjustment method provided by the embodiments of the present application; Figure 6 A step flowchart of a parallax parameter adjustment method provided by the embodiments of the present application; Figure 7 A step flow chart of an information output method provided by an embodiment of the present application is shown in FIG. 1. Figure 8 A step flow chart of a safety prompt output method provided by an embodiment of the present application is shown in FIG. 2. Figure 9 A schematic diagram of a gaze perception angle provided by an embodiment of the present application is shown in FIG. 3. Figure 10 A schematic diagram of another gaze perception angle provided by an embodiment of the present application is shown in FIG. 4. Figure 11 A structural schematic diagram of an interface element adjustment device provided by an embodiment of the present application is shown in FIG. 5. Figure 12 A structural schematic diagram of an electronic device provided by an embodiment of the present application is shown in FIG. 6. DETAILED DESCRIPTION

[0011] The embodiments of the present application will be further described in detail below with reference to the accompanying drawings.

[0012] Reference Figure 1 is a step flow chart of an interface element adjustment method provided by an embodiment of the present application. As shown in Figure 1 , the interface element adjustment method can include steps 101, 102 and 103.

[0013] Step 101: Obtain the current user state information of the user, and the display parameters of each interface element on the target screen.

[0014] In the present embodiment, the user state information refers to various data capable of reflecting the current physiological state of the user, for judging the interaction state of the user and the screen. In the present example, the user state information can include eye features (pupil size, blink frequency, gaze direction), head posture (tilt angle, rotation direction), fatigue degree (obtained through facial expression or biological sensor), etc.

[0015] The target screen is a physical carrier for displaying interface elements, such as a mobile phone screen, a computer display, a tablet screen, a smart TV, a vehicle-mounted screen, etc. The present embodiment does not limit the specific type of the target screen.

[0016] The interface element refers to various components on the screen that can be perceived or interacted by the user, including but not limited to: interactive components (such as buttons, input boxes, drop-down menus, etc.), information display components (such as icons, pop-up windows, etc.), layout components (such as panels, dividing lines, list items, etc.

[0017] The display parameter refers to a quantitative index describing the current display state of the interface element, and determines the visual presentation effect of the element. In the example, the display parameter of the interface element can include, but is not limited to, a space parameter, an inclination angle, a zoom ratio, and the like.

[0018] In a specific implementation, the user state information of the user at present and the display parameters of the interface elements on the target screen can be acquired.

[0019] In an example, taking driving a vehicle by the user as an example, the target screen can be a vehicle-mounted screen for driving the vehicle by the user. Then, the image of the user can be captured by a vehicle-mounted camera device (such as a DMS (Driver Monitoring System)) to analyze the state information of the user, and the like. The display parameters of the interface elements on the vehicle-mounted screen can be acquired by a 3D scene camera (such as a Kanzi 3D scene camera).

[0020] In an example, taking watching a home theater by the user as an example, the target screen can be a viewing screen of the home theater. Then, the image of the user can be captured by a camera device installed on the viewing screen or other cinema hardware devices to analyze the state information of the user, and the like. The display parameters of the interface elements on the viewing screen can be acquired by a camera device pre-installed on the viewing screen, and the like.

[0021] It can be understood that the above two examples are only example scenarios listed for better understanding the technical solutions of the embodiments of the present application, and the technical solutions provided by the present application can also be applied to other scenarios, such as a smart television scenario, in actual application.

[0022] After the user state information of the user at present and the display parameters of the interface elements on the target screen are acquired, step 102 is performed.

[0023] Step 102: determining a visual parameter of the user relative to the target screen according to the user state information, and generating an adjustment parameter of each interface element according to the visual parameter and the display parameter.

[0024] The visual parameter refers to an index reflecting the visual ability of the user relative to the interface elements on the screen, which is derived based on the user state information and used to determine the perception difficulty of the user to the interface elements. In the example, the visual parameter can include a viewing angle, a viewing distance, a left-eye image parallax, and the like.

[0025] The adjustment parameter refers to a specific numerical value or rule used to modify the display parameter of the interface element, which is the result of the comparison between the "visual parameter" and the "current display parameter". For example, taking the size adjustment parameter as an example, the specific adjustment parameter can be "increase the width by 20%", "enlarge the font from 14px to 18px", etc. Taking the position adjustment parameter as an example, the specific adjustment parameter can be "move from coordinates (100, 200) to (300, 200)", etc.

[0026] After obtaining the current user state information of the user and the display parameters of each interface element on the target screen, the visual parameter of the user relative to the target screen can be determined according to the user state information, and the adjustment parameter of each interface element can be generated according to the visual parameter and the display parameter of each interface element.

[0027] In an implementation manner, the viewing angle of the user can be determined according to the gaze direction information in the user state information, and the tilt angle adjustment parameter of each interface element can be generated according to the viewing angle and the display parameter of each interface element. Through this parameter, the tilt angle of the interface element can be adjusted.

[0028] In another implementation manner, the visual distance parameter of the user relative to the target screen can be determined according to the user head posture information in the user state information. Then, the scale adjustment parameter and / or the position adjustment parameter of each interface element can be generated according to the visual distance parameter and the display parameter of each interface element. Through this parameter, the scaling ratio of the interface element and / or the spatial position of the interface element in the screen can be adjusted.

[0029] In another implementation manner, the viewing angle information of the user relative to the target screen can be determined according to the gaze direction information in the user state information, and the position difference information of the left eye and the right eye of the user relative to the target screen can be determined according to the viewing angle information. Then, the left-eye and right-eye image parallax parameter of each interface element can be generated according to the position difference information, and the relative displacement parameter of each interface element can be generated according to the left-eye and right-eye image parallax parameter and the display parameter of each interface element. Through this parameter, the left-eye and right-eye image parallax of each interface element can be adjusted to present a 3D effect.

[0030] In another implementation manner, the gaze point of the user on the target screen and the gaze duration corresponding to the gaze point can also be determined according to the gaze direction information and the head posture information in the user state information. And in the case that the gaze point is in the specified interaction region of the target screen, and the gaze duration is greater than or equal to the first duration threshold and less than the second duration threshold, the interaction information of the user is output. Such as the interaction information of the function start, the interaction information of the voice greeting, etc.

[0031] In the vehicle interaction scenario, whether the user has been gazing at the driving direction of the vehicle for a long time can be identified according to the user state information, and if so, a safety prompt information can be output to prompt the user to drive safely.

[0032] Step 103: adjusting the display parameter of each interface element based on the adjustment parameter.

[0033] After the adjustment parameter of each interface element is generated, the display parameter of each interface element can be adjusted according to the generated adjustment parameter of each interface element, so as to realize dynamic adjustment of the display parameter of the interface element, optimize the information readability of the interface element in the screen, and improve the comfort of human-computer interaction.

[0034] The adjustment method of the interface element provided in the embodiment of the present application can obtain the current user state information of the user, obtain the visual parameter of the user relative to the screen, and generate the adjustment parameter of the interface element according to the visual parameter and the current display parameter of the interface element, so as to adjust the display parameter of the interface element. Thus, the dynamic adjustment of the display parameter of the interface element can be realized, and the information readability of the interface element in the screen and the comfort of human-computer interaction can be optimized.

[0035] Next, the adjustment method of the interface element provided in the embodiment of the present application will be described in detail. Figure 2 The flow of the adjustment of the interface element in the vehicle scene will be described in detail.

[0036] Referring to Figure 2 , a step flowchart of another adjustment method of an interface element provided in the embodiment of the present application is shown. As Figure 2 indicated, the adjustment method of the interface element can include steps 201, 202 and 203.

[0037] Step 201: obtaining the current user state information of the user driving the vehicle and the display parameter of each interface element on the vehicle screen of the vehicle.

[0038] In the embodiment, the vehicle can be a tool such as a car, a truck, a ship, an airplane, etc.

[0039] The user state information refers to real-time data reflecting the physiological state of the driver. In the example, the user state information can include physiological data such as the eye position, the line of sight direction, the pupil size, the blink frequency, the head posture, etc. of the driver.

[0040] The vehicle screen refers to a display device for displaying information in the vehicle, which needs to be adapted to the ergonomics design of the driving scene. In the example, the vehicle screen can be a center screen (entertainment, navigation), an instrument panel (vehicle speed, fuel quantity, etc. core data), a head-up display (projected to the front windshield glass to reduce the line of sight shift), a rear entertainment screen, etc.

[0041] Interface elements refer to driving-related information components on the vehicle screen. In this example, interface elements may include: speedometer, tachometer, fuel / battery level indicator, warning icons (such as malfunction indicator lights and collision warnings), navigation key commands, etc.

[0042] During the operation of a vehicle, the current user status information of the user driving the vehicle, as well as the display parameters of various interface elements on the vehicle's screen, can be obtained.

[0043] In practical implementation, a driver monitoring system and 3D scene cameras (such as Kanzi 3D cameras) are installed inside the vehicle. The driver monitoring system's camera can be installed inside the vehicle near the driver (such as on the A-pillar, center console, or dashboard). The 3D scene camera can be integrated into the vehicle's central control display hardware system for rendering and displaying 3D scenes. This driver monitoring system can obtain the current user status information of the driver and, simultaneously, obtain the display parameters of various interface elements on the vehicle's screen. Specifically, it uses real-time rendering technology to obtain the spatial coordinates and tilt angles of each UI (User Interface Designer) element in the 3D scene on the screen.

[0044] The methods for obtaining user status information can be combined with... Figure 3 The following is a detailed description.

[0045] Reference Figure 3 The diagram illustrates a flowchart of a user status information acquisition method provided in an embodiment of this application. Figure 3 As shown, the method for obtaining user status information may include steps 301 and 302.

[0046] Step 301: Acquire the user's current user image.

[0047] In this embodiment, when acquiring user status information, the current user image can be captured. For example, the current user image of a vehicle can be captured through a driver monitoring system. In a specific implementation, the hardware device of the DMS (mainly a camera) can be used to continuously and stably acquire images of the driver's face and upper body to provide raw data for subsequent status analysis, which must meet the requirements of "no blind spots, low interference, and high adaptability" (adapting to different lighting conditions and postures).

[0048] Step 302: Recognize the user image to obtain the user's status information.

[0049] After obtaining the user image, the user state information of the user can be identified from the user image. That is, by integrating an image processing algorithm in the driver monitoring system, the user image is identified and analyzed to obtain the user state information of the user and the like.

[0050] Embodiments of the present application implement a process of “image acquisition→state identification”, and the core is “hardware adaptation to driving scene + algorithm focusing on safety-related features”: the camera installation needs to consider the field of view and concealment, and the algorithm needs to stably extract key states such as line of sight, fatigue, and behavior in a complex environment (light, obstruction), and finally provides reliable real-time data support for driving safety and dynamic adjustment of the interface.

[0051] After obtaining the user state information and the display parameters of each interface element on the vehicle screen, step 202 is performed.

[0052] Step 202: determining the visual parameter of the user relative to the vehicle screen according to the user state information, and generating the adjustment parameter of each interface element according to the visual parameter and the display parameter.

[0053] After obtaining the user state information, the visual parameter of the user relative to the vehicle screen can be determined according to the user state information, and the adjustment parameter of each interface element can be generated according to the visual parameter and the display parameter of each interface element.

[0054] In the present example, the adjustment parameter can be a tilt angle adjustment parameter, a scale adjustment parameter, a position adjustment parameter, a relative displacement parameter, and the like. The process of obtaining the adjustment parameter will be described in detail in the following steps, and the present embodiment will not be described again.

[0055] Step 203: adjusting the display parameter of each interface element on the vehicle screen based on the adjustment parameter.

[0056] After obtaining the adjustment parameter of each interface element, the display parameter of each interface element on the vehicle screen can be adjusted based on the adjustment parameter, so as to dynamically adapt the HMI (Human-Machine Interface, human-machine interaction interface) interface element (tilt angle, zoom scale, spatial positioning) based on the current viewing angle and seat position of the driver, greatly optimizing the information readability and interaction comfort, and providing a tailor-made interaction experience for each driver.

[0057] In a specific implementation of the present application, the adjustment parameter can be a tilt angle adjustment parameter of the interface element, and the implementation process of generating the tilt angle adjustment parameter of each interface element can be combined with the implementation process of generating the display parameter of each interface element. Figure 4 The following detailed description is made.

[0058] Reference is made to Figure 4FIG. 1 shows a flow chart of a method for generating a tilt angle adjustment parameter according to an embodiment of the present application. As shown in FIG. 1, the method for generating a tilt angle adjustment parameter can include the following steps: step 101, step 102, step 103, and step 104. Figure 4

[0059] Step 101: determining viewing angle information of the user relative to the screen of the vehicle according to line-of-sight direction information in the user state information.

[0060] In the present embodiment, the line-of-sight direction information refers to quantitative data reflecting the line-of-sight direction of the driver, which is collected by a driver monitoring system (DMS), and includes horizontal deviation angle (left-right direction) and vertical deviation angle (up-down direction) of the line-of-sight in three-dimensional space. For example, “horizontal deviation angle -10° (deviated 10° to the left), vertical deviation angle 5° (deviated 5° upward)”, indicates that the line-of-sight of the driver deviates from the straight-ahead direction to the upper left.

[0061] The viewing angle information refers to the spatial viewing angle of the driver relative to the screen of the vehicle, i.e., the included angle between the line-of-sight of the driver and the normal line of the screen plane (a virtual line perpendicular to the screen), which includes horizontal viewing angle (angle of deviation from the normal line in the left-right direction) and vertical viewing angle (angle of deviation from the normal line in the up-down direction). For example, “horizontal viewing angle 15°, vertical viewing angle 8°”, indicates that the driver views the screen from the right side of the normal line of the screen by 15° and from the upper side by 8°.

[0062] In a specific implementation, the DMS outputs a three-dimensional direction vector of the line-of-sight of the driver (e.g., in the vehicle coordinate system, with the center of the screen as the origin, the line-of-sight vector is (x, y, z)) through an eye tracking algorithm (based on the eye image collected by an infrared camera).

[0063] The reference plane of the screen of the vehicle (e.g., the center control screen) has a fixed orientation in the vehicle coordinate system, and the normal vector (perpendicular to the screen plane) thereof is pre-calibrated (e.g., the normal vector of the center control screen is (0, 0, 1), indicating the straight-ahead direction of the screen).

[0064] Through spatial geometric calculation, the included angle between the line-of-sight direction vector and the normal vector of the screen is calculated, which is decomposed into horizontal viewing angle (included angle around the vertical axis) and vertical viewing angle (included angle around the horizontal axis), i.e., the viewing angle information is obtained. For example, the horizontal included angle between the line-of-sight and the normal is 15°, and the vertical included angle is 8°, i.e., the viewing angle information is “horizontal 15°, vertical 8°”, etc.

[0065] After obtaining the viewing angle information, step 102 is performed.

[0066] ​Step 402: determining display angle parameters of each interface element in the carrier screen according to the viewing angle information, the display angle parameters representing that the display angle of each interface element is directly opposite the viewing angle of the user.

[0067] The display angle parameter refers to the display inclination angle of the interface element on the screen, which is used to describe the included angle between the element plane and the screen reference plane (the plane of the screen itself). The display angle parameter can be used to represent that the display angle of each interface element is directly opposite the viewing angle of the user.

[0068] After obtaining the viewing angle information, the display angle parameters of each interface element in the carrier screen can be determined according to the viewing angle information. Specifically, based on the principle of geometric optics, the display angle of the interface element needs to be "mirror matched" with the viewing angle - if the driver views from the right side of the screen normal at 15° (horizontal viewing angle 15°), the element needs to be inclined to the left at 15° (horizontal display angle 15°) to make the element normal line coincide with the viewing direction; similarly, the vertical viewing angle is 8° (viewed from above), the element needs to be inclined upward by 8° (vertical display angle 8°). Further, for each interface element, target parameters including horizontal inclination angle and vertical inclination angle can be generated. For example: the display angle parameter of the vehicle speed number is "horizontal 15°, vertical 8°", and the display angle parameter of the navigation icon is the same (maintaining overall consistency).

[0069] Step 403: generating inclination angle adjustment parameters of each interface element according to the display angle parameters and initial display angle parameters in the display parameters.

[0070] The inclination angle adjustment parameter refers to a specific value (difference) for adjusting the current display angle of the interface element to the target display angle, that is, "target display angle parameter - current display angle parameter". For example: the current element is inclined horizontally by 5°, and the target needs to be 15°, so the horizontal inclination adjustment parameter is "+10°" and so on.

[0071] After obtaining the display angle parameters of each interface element, the inclination angle adjustment parameters of each interface element can be generated according to the determined display angle parameters and the display angle parameters of each interface element in the display parameters. That is, the inclination angle adjustment parameter = display angle parameter - initial display angle parameter. For example: horizontal angle 15° - initial 5° = horizontal adjustment parameter +10°; vertical angle 8° - initial 3° = vertical adjustment parameter +5° and so on.

[0072] In actual application, in order to avoid interface confusion caused by excessive inclination of elements, a maximum inclination threshold (such as horizontal / vertical direction maximum ±30°) is set, and if the calculation result exceeds the threshold, the threshold is taken as the adjustment parameter (such as target angle 40°, the adjustment parameter is limited to 30° and so on).

[0073] Step 404: adjusting the tilt angle of each interface element on the vehicle screen according to the tilt angle adjustment parameter.

[0074] After obtaining the tilt angle adjustment parameter, the tilt angle of each interface element on the vehicle screen can be adjusted according to the tilt angle adjustment parameter.

[0075] In the embodiments of the present application, the elements are adapted to the viewing angle of the driver through the tilt angle, reducing the distortion of graphics / text (such as trapezoidal distortion) caused by "oblique viewing", so that the driver can quickly identify information (such as vehicle speed numbers and navigation instructions). Moreover, the elements are directly opposite to the line of sight, reducing the adjustment burden of the driver's eye muscles (without additional focusing for "oblique viewing of distorted elements"), which is especially suitable for long-time driving scenarios.

[0076] In the embodiments of the present application, the adjustment parameter can be a proportional position adjustment parameter, and the acquisition method of the proportional position adjustment parameter can be combined with Figure 5 which will be described in detail.

[0077] Referring to Figure 5 , a step flowchart of a proportional position adjustment parameter generation method provided by an embodiment of the present application is shown. As Figure 5 shown, the proportional position adjustment parameter generation method can include steps 501, 502, 503, and 504.

[0078] Step 501: determining a viewing distance parameter of the user relative to the vehicle screen according to user head posture information of the user state information.

[0079] In the embodiments of the present application, the user head posture information refers to quantitative data reflecting the spatial position and angle of the driver's head collected by a driver monitoring system (DMS). It can include position parameters (i.e. coordinates of the head in the vehicle three-dimensional coordinate system (such as horizontal distance and vertical height from the screen)), angle parameters (i.e. pitch angle (tilt up and down, such as lowering / raising the head), yaw angle (turning left and right, such as turning the head to look at the rearview mirror), roll angle (tilt, such as body tilt), etc.

[0080] The viewing distance parameter refers to the straight-line distance between the driver's head and the vehicle screen (usually the vertical distance from the head center point to the screen plane), which is a core indicator for measuring "visual clarity requirements". For example, "viewing distance parameter = 70 cm" indicates that the vertical distance from the head to the screen is 70 cm; when the viewing distance increases (such as the driver leaning back in the seat), the element needs to be larger to be clearly identified.

[0081] After obtaining the user state information, the viewing distance parameter of the user relative to the vehicle screen can be determined according to the user head posture information of the user state information. Specifically, the Z-axis coordinate of the head center point (usually the average coordinate of the midpoint of the line connecting the two eyes and the nose tip) can be taken as the viewing distance parameter (because the Z-axis is perpendicular to the screen plane, it directly reflects the vertical distance). For example: head center point Z coordinate = 70 cm, then viewing distance parameter = 70 cm, etc.

[0082] Step 502: generating the scale adjustment parameter and / or the position adjustment parameter of each interface element in the vehicle screen according to the viewing distance parameter and the display parameter.

[0083] The scale adjustment parameter refers to a quantitative instruction for changing the size of the interface element, which is usually a scaling factor (such as "1.2" indicating an enlargement to 120% of the original size) or an absolute size difference (such as "+50px" indicating an increase of 50px in width), and needs to be positively correlated with the viewing distance parameter (the larger the viewing distance, the larger the adjustment parameter).

[0084] The position adjustment parameter refers to a quantitative instruction for changing the position of the interface element on the screen, which is usually a coordinate offset (such as "x+30px, y-20px"), and the goal is to keep the element within the core visual range of the driver at all times (such as when the head is left-biased, the element is left-biased to remain in the center of the line of sight).

[0085] After obtaining the viewing distance parameter, the scale adjustment parameter and / or the position adjustment parameter of each interface element in the vehicle screen can be generated according to the viewing distance parameter and the display parameter. That is, the specific scaling and offset are calculated in combination with the display state of the current element and the viewing distance requirement, to ensure that the element is clearly visible and does not deviate from the line of sight at different distances and / or head positions.

[0086] The generation method of the scale adjustment parameter can be to establish a "viewing distance-scale" mapping model: based on the visual characteristics of the human eye (the size of the element that can be clearly identified is proportional to the viewing distance), the preset formula can be: target scale = baseline scale * (current viewing distance / standard viewing distance). For example: the standard viewing distance is 80 cm (the distance when most drivers are driving normally), and the baseline scale of a certain button is 1.0 (the size is 100px x 50px); when the viewing distance becomes 100 cm (25% farther than the standard), the target scale = 1.0 x (100 / 80) = 1.25, that is, the button needs to be enlarged to 125px x 62.5px. And calculate the adjustment amount in combination with the current display parameter: scale adjustment parameter = target scale - current scale. If the current scale is 1.0, then the adjustment parameter = +0.25.

[0087] The generation manner of the position adjustment parameter can be: calculating element offset based on X / Y axis offset (horizontal / vertical position) of the head posture, horizontal direction (head X coordinate is +5cm (offset right 5cm), and the element needs to offset right 2% of the screen width (preset mapping relationship: head offset 1cm, and the element offsets 0.4% of the screen width), that is, the position adjustment parameter is "x+2% screen width"), vertical direction (head Y coordinate is +10cm (offset up 10cm), and the element needs to offset up 3% of the screen height, and the adjustment parameter is "y+3% screen height"). Then, the final position adjustment parameter is generated in combination with the current position parameter: if the current x coordinate of the element is 10% on the left side of the screen center, and the target needs to be right by 2%, the final adjustment parameter is "x+12% screen width", and the like.

[0088] Step 503: adjusting the display proportion of each interface element on the vehicle screen according to the proportion adjustment parameter.

[0089] After obtaining the proportion adjustment parameter, the display proportion of each interface element on the vehicle screen can be adjusted according to the proportion adjustment parameter, that is, the size of the interface element on the vehicle screen is adjusted.

[0090] Step 504: adjusting the display position of each interface element on the vehicle screen according to the position adjustment parameter.

[0091] After obtaining the position adjustment parameter, the display position of each interface element on the vehicle screen can be adjusted according to the position adjustment parameter.

[0092] The embodiment of the application makes the size and position of the interface element adaptive to the head distance and position change of the driver through the distance perception and dynamic adjustment based on the head posture, which can not only ensure the clarity through magnifying the element at a long distance, but also keep the visual center aligned through position correction when the head is offset, thereby reducing the visual search time and eye focusing burden of the driver, and improving the driving safety and interface interaction efficiency.

[0093] In a specific implementation of the application, the adjustment parameter can further include a relative displacement parameter, and the generation manner of the relative displacement parameter can be combined with the generation manner of the position adjustment parameter. Figure 6 The following will be described in detail.

[0094] Referring to Figure 6 , a step flowchart of a displacement parameter generation method provided by the embodiment of the application is shown. As Figure 6 shown, the displacement parameter generation method can include: step 601, step 602, step 603, step 604 and step 605.

[0095] Step 601: Determine the viewing angle information of the user relative to the vehicle screen according to the line-of-sight direction information in the user state information.

[0096] In this embodiment, after obtaining the user state information, the viewing angle information of the user relative to the vehicle screen can be determined according to the line-of-sight direction information in the user state information.

[0097] Step 602: Determine the position difference information of the left eye and the right eye of the user relative to the vehicle screen according to the viewing angle information.

[0098] The position difference information refers to the position difference of the left eye and the right eye relative to the vehicle screen in three-dimensional space, which is usually based on the screen plane and includes horizontal distance difference (left-right direction), vertical distance difference (up-down direction), and depth distance difference (front-back direction). For example, "position difference information: horizontal difference 6.5 cm (left eye on the right, right eye on the left), vertical difference 0 cm, depth difference 1 cm", which is determined by the human eye pupil distance (about 6-7 cm) and the viewing angle.

[0099] After obtaining the viewing angle information, the position difference information of the left eye and the right eye of the user relative to the vehicle screen can be determined according to the viewing angle information. Specifically, using the triangular geometric relationship, the spatial coordinates of each eye to the screen are calculated according to the monocular viewing angle (horizontal / vertical) and the viewing distance (depth from head to screen): Left eye coordinates: (viewing distance x tan (left eye horizontal viewing angle), viewing distance x tan (left eye vertical viewing angle), viewing distance).

[0100] Right eye coordinates: (viewing distance x tan (right eye horizontal viewing angle), viewing distance x tan (right eye vertical viewing angle), viewing distance).

[0101] Position difference calculation: i.e. position difference = left eye coordinates - right eye coordinates.

[0102] Step 603: Generate left and right eye image disparity parameters corresponding to each interface element based on the position difference information.

[0103] The left and right eye image disparity parameters refer to the position offset (mainly in the horizontal direction, as human stereoscopic perception mainly relies on horizontal disparity) of the same interface element in the left eye view and the right eye view to simulate human stereoscopic vision, which is usually measured in pixels or degrees. For example, "disparity parameter = 8 pixels" indicates that the element in the left eye image is 8 pixels to the right of the right eye image, producing a stereoscopic effect.

[0104] After obtaining the position difference information, the left and right eye image disparity parameters corresponding to each interface element can be generated based on the position difference information. That is, the binocular position difference is converted into image disparity to make the element present a "depth perception" consistent with human stereoscopic perception.

[0105] In a specific implementation, a mapping model of parallax and position difference can be established in advance, that is, based on the principle of stereoscopic vision, the parallax parameter is proportional to the horizontal position difference of the two eyes and inversely proportional to the viewing distance (formula: parallax = k x (horizontal position difference / viewing distance), k is a conversion coefficient of screen pixel-physical size, such as 10 pixels corresponding to 1 cm). According to the obtained position difference information in advance, the corresponding left and right eye image parallax parameters can be mapped.

[0106] Step 604: According to the left and right eye image parallax parameters and the display parameters, the relative displacement parameters of the left and right eye images corresponding to each interface element in the screen of the vehicle are generated.

[0107] The relative displacement parameter refers to the specific value of the original image of the interface element being offset to the left and right eye views respectively (i.e., the difference between the left eye image displacement and the right eye image displacement), which is used to realize the preset parallax parameter. For example, "relative displacement parameter = left eye + 5 pixels, right eye - 3 pixels", the total offset difference of the left and right eye images is 8 pixels, matching the parallax parameter.

[0108] After obtaining the left and right eye image parallax parameters, the relative displacement parameters of the left and right eye images corresponding to each interface element in the screen of the vehicle can be generated according to the left and right eye image parallax parameters and the display parameters of each interface element. Specifically, the original coordinates of the elements can be read from the display parameters, such as the display parameters of a certain warning icon (x = 300px, y = 200px), and the left and right eye displacement amounts are assigned, that is, the parallax parameter = 4 pixels (the left eye image needs to be right shifted by 4 pixels than the right eye image), and the relative displacement parameter is assigned as follows: left eye image displacement: +2 pixels (x = 300+2 = 302px); right eye image displacement: -2 pixels (x = 300-2 = 298px).

[0109] Step 605: Adjust the parallax parameters of the left and right eye images according to the relative displacement parameters.

[0110] After obtaining the relative displacement parameters, the parallax parameters of the left and right eye images can be adjusted according to the relative displacement parameters. Specifically, the difference between the current relative displacement corresponding to the current parallax parameter and the target relative displacement required by the relative displacement parameter can be calculated as the correction amount of the parallax parameter, so as to adjust the parallax parameters of the left and right eye images according to the correction amount.

[0111] The embodiment of the present application adjusts the parallax by simulating the human eye stereoscopic vision, so that the interface elements of the vehicle screen dynamically adapt to the left and right eye image displacement according to the driver's binocular position and viewing angle, which not only enhances the depth perception and stereoscopic effect of the elements (improves the recognition efficiency), but also avoids ghosting or blurring caused by viewing angle deviation, reduces the visual fatigue of the driver, and finally improves the readability and interaction safety of the interface information in complex driving scenarios.

[0112] In a specific implementation of the present application, when the driver's gaze briefly scans the virtual personal assistant (VPA) icon on the right side of the center screen and stays for about 1.5 seconds, the intelligent gaze interaction submodule recognizes this gaze behavior and immediately triggers the VPA to light up and issue a voice greeting of "Hello, what can I help you with?" without the driver manually clicking or voice awakening. Of course, if the user's gaze time is too long, a safety prompt message can be output to remind the driver to pay attention to safe driving. For this implementation process, the following Figure 7 will be described in detail.

[0113] Referring to Figure 7 , a step flowchart of an information output method provided by an embodiment of the present application is shown. As Figure 7 shown, the information output method can include steps 701, 702, and 703.

[0114] Step 701: determining, according to the line-of-sight direction information and the head pose information in the user state information, a gaze point of the user on the vehicle screen and a gaze duration corresponding to the gaze point.

[0115] In the present embodiment, the gaze point refers to the intersection of the user's line-of-sight direction and the plane of the vehicle screen, i.e., the specific coordinate position on the screen where the user's current vision is focused.

[0116] The gaze duration refers to the length of time that the user's gaze point remains in a certain fixed region (or coordinate range) on the screen.

[0117] In a specific implementation, after obtaining the user state information, the line-of-sight direction information and the head pose information in the user state information can be used to determine the gaze point of the user on the vehicle screen and the gaze duration corresponding to the gaze point. Specifically, the system predefines the physical size and installation position of the vehicle screen (such as the coordinates (X0, Y0, Z0) of the upper left corner of the screen in the vehicle coordinate system, the mapping relationship between the pixel coordinates and the physical coordinates corresponding to the screen resolution), and performs geometric operations on the "head space position" corresponding to the head pose information and the "line-of-sight vector" corresponding to the line-of-sight direction information to solve the intersection of the line-of-sight vector and the screen plane, i.e., the gaze point (specific coordinates in the output screen pixel coordinate system, such as (320, 480)).

[0118] At the same time, the system can monitor the coordinate changes of the gaze point in real time through a timer. If the gaze point always falls within the same fixed region (such as the coordinate range of a specified interaction region) in consecutive frames (such as 30 frames of images per second), the timer is continuously accumulated; if the gaze point deviates from the region, the timer is reset, and finally the gaze duration is obtained.

[0119] Step 702: In the case that the gaze point is in the designated interaction area of the vehicle screen, and the gaze duration is greater than or equal to the first duration threshold and less than the second duration threshold, output interaction information of the user.

[0120] The designated interaction area refers to a pre-set function area on the vehicle screen that supports interaction triggered by "gaze" (such as a navigation destination search box, a screen area corresponding to an air conditioner adjustment button, a media play control area), and the area boundary coordinates need to be defined in the system in advance. In this example, the designated interaction area can be the VPN area.

[0121] The first duration threshold refers to the minimum gaze time standard (such as 800ms) preset by the system for "triggering interaction", and only when the gaze duration reaches or exceeds this threshold, does it have the basic condition for triggering interaction.

[0122] The second duration threshold refers to the critical gaze time standard (such as 3s) preset by the system for "triggering safety prompts", and when the gaze duration exceeds this threshold, it is determined that the user is excessively focused on the screen, which may affect driving safety.

[0123] The interaction information refers to the feedback content output by the system when the user meets the "gaze at the designated area + duration meets the standard" condition. In this example, the interaction information can be greeting information, function start information, etc.

[0124] In the case that the gaze point is in the designated interaction area of the vehicle screen, and the gaze duration is greater than or equal to the first duration threshold and less than the second duration threshold, output interaction information of the user.

[0125] Step 703: In the case that the gaze duration is greater than the second duration threshold, output safety prompt information.

[0126] The safety prompt information refers to the warning content (such as the voice prompt "Please focus on driving and reduce screen gaze time") output by the system when the user's gaze duration on the screen exceeds the second threshold.

[0127] In the case that the gaze duration is greater than the second duration threshold, safety prompt information can be output, such as playing a short warning through the vehicle audio (such as "Please pay attention to the road conditions ahead and reduce screen attention"), to avoid the user being distracted again by viewing the screen prompt. Or, a short yellow / red warning icon (such as "Safety driving reminder") is popped up at the edge of the screen, or the instrument panel indicator light flashes to assist in prompting, without occupying the core display area of the screen.

[0128] The embodiment of the application realizes the dual goals of contactless intelligent interaction of the vehicle HMI and driving safety protection through double monitoring of the gaze point + gaze duration: without manual operation (such as clicking, sliding the screen) of the user, the interaction can be triggered only through the gaze, which improves the convenience and safety of operation during driving; and the duration threshold can be used for control to avoid the user from being distracted by excessive focus on the screen, and the driving concentration is enhanced in time through the safety prompt, thereby balancing the HMI interaction experience and driving safety.

[0129] In another specific implementation of the application, when it is detected that the driver's line of sight is not looking at the front road for a long time (for example, more than 3 seconds), but is continuously looking at the outside of the window or the information on the co-driver side screen, the pre-warning and reminding module will issue a reminder action and a soft voice prompt through the transition color change of the instrument panel and the VPA virtual assistant: “Please pay attention to the road conditions ahead, drive safely.” and the like, to help the driver timely pull back his attention. For this implementation process, the method for determining the gaze direction information of the user can be combined with the method for determining the head posture information of the user. Figure 8 The following will be described in detail.

[0130] Referring to Figure 8 , a step flowchart of a safety prompt output method provided by an embodiment of the application is shown. As Figure 8 indicated, the safety prompt output method includes steps 801 and 802.

[0131] Step 801: determining the gaze direction information of the user according to the line-of-sight direction information and the head posture information in the user state information.

[0132] In this embodiment, the gaze direction information refers to the spatial direction data pointed to by the user's visual focus, which is calculated based on the user's line-of-sight direction and head posture, and can accurately reflect where the user is currently looking, for example, pointing to the vehicle screen, the road outside the window, other areas in the vehicle, etc.

[0133] The third duration threshold refers to a time standard pre-set for judging whether the user has a “distraction risk”, which is usually determined in combination with the requirements of traffic regulations and driving safety experimental data (for example, commonly set to 2-3 seconds, that is, the user triggers the prompt if he does not look at the driving direction for more than 2 seconds in a row), which is a core determination parameter in the algorithm.

[0134] After obtaining the user state information, the gaze direction information of the user can be determined according to the line-of-sight direction information and the head posture information in the user state information. Specifically, the coupling interference of head movement and line-of-sight movement can be eliminated by an algorithm. For example, when the user turns the head to the left but the line of sight is still looking forward, it is necessary to distinguish between the deflection of the head posture and the actual pointing direction of the line of sight. In actual application, a “head coordinate system” can be established with the head posture as the reference (taking the center of the head as the origin, the forward direction as the Z axis, the right direction as the X axis, and the upward direction as the Y axis). Then, the line-of-sight direction information is converted from the “absolute coordinate system” (taking the vehicle-mounted screen as the origin) to the “head coordinate system”, and the influence of head movement on line-of-sight judgment is corrected.

[0135] Step 802: In the case where the gaze direction information indicates that the user has not gazed at the driving direction of the vehicle within a third time length threshold, output safety prompt information.

[0136] The driving direction of the vehicle refers to the spatial direction in which the vehicle (such as a car or a high-speed train) is currently driving. Generally, the driving direction is determined based on the forward axis of the vehicle in combination with the driving trajectory data of devices such as GPS and gyroscope, to determine the safety view range (such as the road in front of the car, the intersection, etc.) that needs to be focused on by the user.

[0137] The safety prompt information refers to the warning content output by the system when it is determined that the user has a safety risk. The form includes but is not limited to: vehicle-mounted screen pop-up prompt, steering wheel vibration, voice broadcast (such as “please focus on the road ahead”), instrument panel light flashing, etc. The purpose is to quickly wake up the user's attention.

[0138] In the case where the gaze direction information indicates that the user has not gazed at the driving direction of the vehicle within a third time length threshold, the safety prompt information is output. Specifically, the gaze direction information obtained in the above step is compared with the driving direction of the vehicle, that is, the safety view range of the driving direction is first determined through the vehicle-mounted GPS and gyroscope, and then it is judged whether the current gaze direction of the user falls within the safety view range. If it does not fall within the range, the “time length counting” is started. If the accumulated time length of the user's “not gazing at the driving direction” is continuously accumulated and compared with the preset third time length threshold (such as 2 seconds), if the accumulated time length < the third time length threshold, it is determined as “short-term distraction”, and the prompt is not triggered (such as the user glances at the center control screen quickly and then looks back at the front). If the accumulated time length ≥ the third time length threshold, it is determined as “high-risk distraction”, and the safety prompt process is triggered.

[0139] When giving safety prompts, the optimal prompt mode can be selected to output safety prompt information according to the hardware configuration of the vehicle, for example, in a driving scene, “steering wheel vibration + voice prompt” is preferentially triggered (screen prompts are avoided to distract more attention), and in a passenger scene, a screen pop-up prompt can be triggered to ensure that the prompt is effective and does not increase safety risks.

[0140] The embodiment of the application accurately determines whether the user is focused on the driving direction of the vehicle through multi-dimensional data fusion of “line of sight + head posture”, can identify the risk of the user being distracted for a long time (such as looking down at the mobile phone while driving, the passenger misblocking the driver's line of sight, etc.), effectively reduces safety accidents caused by “visual deviation from the driving direction” by timely outputting safety prompt information, ensures the safety of traffic travel while avoiding misjudgment caused by single reliance on line of sight or head posture data (such as not mis-triggering the prompt when the user turns his head but the line of sight is still looking forward), and balances the determination accuracy and user experience.

[0141] In the field of vehicle technology, a dynamic human-computer interaction system based on gaze perception is also provided, and the main constituent modules and working principles are as follows: The system architecture can include: a driver monitoring system (DMS), a 3D scene camera module, a data fusion and analysis module, an HMI interface management module (including a screen control dynamic self-adaptation sub-module, an intelligent gaze interaction sub-module, and a high-precision parallax simulation sub-module), a display output module, and a warning and reminding module.

[0142] 1. Driver monitoring system (DMS): Function: Real-time monitoring of physiological data such as eye position, line of sight, pupil size, blink frequency, and head posture of the driver.

[0143] Implementation: An infrared camera combined with image processing algorithms or other visual sensors integrated in the cockpit can be used.

[0144] 2. 3D scene camera module: Function: Obtain the relative positions of visual elements in the 3D space on the screen.

[0145] Implementation: Real-time rendering technology is used to obtain the spatial coordinates and inclination angles of UI elements in the 3D scene on the screen.

[0146] 3. Data fusion and analysis module: Function: Real-time reception and fusion of raw data from the DMS and 3D scene camera module. Through multi-modal data fusion algorithms and machine learning models, the real-time visual angle of the driver, the seat position, the coordinates of the gaze point on the screen, the gaze duration, and the attention state (for example, whether tired or distracted) are accurately calculated and inferred.

[0147] Implementation: Running on the vehicle's central processor, using deep learning, computer vision, and other technologies for data processing and state estimation.

[0148] 4. HMI interface management module: Function: Based on the output of the data fusion and analysis module, dynamically generate and manage the HMI interface.

[0149] Screen control dynamic adaptive sub-module: Function: According to the real-time view of the driver and the seat position provided by the data fusion and analysis module, intelligently adjust the tilt angle, zoom ratio and spatial positioning of each element in the display interface (such as speedometer, VPA virtual assistant, navigation control, etc.).

[0150] Implementation: When the driver's head or seat position changes, the system can automatically calculate the new perspective projection parameters, adjust the rotation, size and relative position of the interface elements on the screen, to ensure that the information has the best readability and operation convenience under different observation angles.

[0151] Intelligent gaze interaction sub-module: Function: Monitor the driver's gaze behavior. When the system detects that the driver's gaze briefly scans a specific interaction area (such as the virtual assistant VPA) and stays for a certain period of time (configurable threshold, such as 0.5-2 seconds), it triggers the intelligent response of the virtual assistant.

[0152] Implementation: Based on gaze point detection and gaze duration judgment, through the preset interaction logic, trigger the voice greeting, safety information prompt or function start of the VPA.

[0153] High-precision gaze simulation sub-module: Function: According to the driver's eye position provided by the data fusion and analysis module, change the tilt angle, zoom ratio and spatial positioning of 3D elements, while supporting real-time generation and adjustment of the parallax of left and right eye images, so as to present 3D dynamic effects with strong depth of field on general 2D display screens.

[0154] Implementation: Using real-time rendering technology, spatial rotation and displacement, multi-layer image superposition and displacement algorithm, simulate the parallax effect when the human eye watches real 3D objects, without the need to wear 3D glasses.

[0155] 5. Display output module: Function: Receive the dynamic interface data generated by the HMI interface management module, and efficiently render and present it to the driver.

[0156] Implementation: It can be any vehicle display device such as the vehicle's central control screen, instrument panel, head-up display (HUD), etc.

[0157] 6. Warning and reminder module: Function: Real-time monitoring of the driver's attention state. When the data fusion and analysis module determines that the driver's line of sight deviates from the front road conditions for a long time (for example, beyond the preset safe gaze area or for too long), it actively issues a reminder, and the instrument area and VPA have feedback actions to remind the driver to drive safely, improving driving safety.

[0158] Implementation: Through appropriate and non-interfering ways such as voice prompts (for example, VPA soft voice reminders), visual prompts (for example, instrument and screen edge flickering, icon color change), or tactile feedback (for example, slight steering wheel vibration), help the driver return to a safe driving posture in time.

[0159] The embodiment of the application realizes dynamic self-adaptation of screen controls based on the driver's perspective and seat position through real-time linkage of eye tracking and 3D scene cameras in the driver monitoring system, supports intelligent gaze interaction and immersive 3D visual effects, and thus builds a highly intelligent, personalized, and safety-enhanced dynamic HMI interface. Figure 9 and Figure 10 As shown in

[0160] The VPA can perceive the driver's gaze, realize implicit and active interaction, make human-computer dialogue more natural and emotionally connected, realize high-precision parallax simulation of 3D dynamic effects on a general 2D screen, provide a future-oriented visual experience, and enhance the immersion of the cockpit.

[0161] Through real-time eye tracking and attention state monitoring, the system can timely and gently remind the driver when distracted, effectively prevent potential risks, change passive warning to active intervention, and ensure driving safety.

[0162] The application innovatively integrates DMS and 3D scene camera data to enhance the feedback of the driver monitoring state, and provides a solid foundation for highly competitive HMI products.

[0163] The following is an apparatus embodiment of the application, which can be used to execute the method embodiments of the application. For details not disclosed in the apparatus embodiment of the application, please refer to the method embodiments of the application.

[0164] Referring to Figure 11 , a structural schematic diagram of an interface element adjustment device is shown. As shown in Figure 11 , the interface element adjustment device 1100 can include the following modules: The information obtaining module 1110 is configured to obtain current user state information of a user and display parameters of interface elements on a target screen. The parameter generating module 1120 is configured to determine visual parameters of the user relative to the target screen according to the user state information, and generate adjustment parameters of the interface elements according to the visual parameters and the display parameters. The parameter adjustment module 1130 is configured to adjust the display parameters of the interface elements on the target screen based on the adjustment parameters.

[0165] Optionally, the user includes a user driving a vehicle, the target screen is a vehicle screen of the vehicle driven by the user, and the visual parameters of the user relative to the target screen include visual parameters of the user relative to the vehicle screen. The parameter adjustment module includes: The parameter adjustment unit is configured to adjust the display parameters of the interface elements on the vehicle screen based on the adjustment parameters.

[0166] Optionally, the information obtaining module includes: The image collecting unit is configured to collect a user image of the user. The information obtaining unit is configured to identify the user image to obtain user state information of the user.

[0167] Optionally, the parameter adjustment unit includes: The first visual angle determining sub-unit is configured to determine viewing visual angle information of the user relative to the vehicle screen according to line-of-sight direction information in the user state information. The angle parameter determining sub-unit is configured to determine display angle parameters of the interface elements in the vehicle screen according to the viewing visual angle information, the display angle parameters indicating that display angles of the interface elements are directly opposite the viewing visual angle of the user. The angle adjustment parameter generating sub-unit is configured to generate tilt angle adjustment parameters of the interface elements according to the display angle parameters and initial display angle parameters in the display parameters. The tilt angle adjusting sub-unit is configured to adjust tilt angles of the interface elements on the vehicle screen according to the tilt angle adjustment parameters.

[0168] Optionally, the parameter adjustment unit includes: The visual distance parameter determining sub-unit is configured to determine visual distance parameters of the user relative to the vehicle screen according to user head posture information in the user state information. The adjustment parameter generation sub-unit is configured to generate a proportion adjustment parameter and / or a position adjustment parameter of each interface element in the vehicle screen according to the visual distance parameter and the display parameter. The proportion adjustment sub-unit is configured to adjust a display proportion of each interface element in the vehicle screen according to the proportion adjustment parameter. The position adjustment sub-unit is configured to adjust a display position of each interface element in the vehicle screen according to the position adjustment parameter.

[0169] Optionally, the parameter adjustment unit comprises: The second visual angle determination sub-unit is configured to determine viewing visual angle information of the user relative to the vehicle screen according to the line-of-sight direction information in the user state information. The position difference determination sub-unit is configured to determine position difference information of the left eye and the right eye of the user relative to the vehicle screen according to the viewing visual angle information. The parallax parameter generation sub-unit is configured to generate left-eye and right-eye image parallax parameters corresponding to each interface element based on the position difference information. The position parameter generation sub-unit is configured to generate relative displacement parameters of left-eye and right-eye images corresponding to each interface element in the vehicle screen according to the left-eye and right-eye image parallax parameters and the display parameter. The parallax parameter adjustment sub-unit is configured to adjust the parallax parameters of the left-eye and right-eye images according to the relative displacement parameters.

[0170] Optionally, the device further comprises: The gaze information determination module is configured to determine a gaze point of the user on the vehicle screen and a gaze duration corresponding to the gaze point according to the line-of-sight direction information and the head posture information in the user state information. The interaction information output module is configured to output interaction information of the user in a case where the gaze point is in a designated interaction area of the vehicle screen, and the gaze duration is greater than or equal to a first duration threshold and less than a second duration threshold. The prompt information output module is configured to output safety prompt information in a case where the gaze duration is greater than the second duration threshold.

[0171] Optionally, the device further comprises: The gaze direction determination module is configured to determine gaze direction information of the user according to the line-of-sight direction information and the head posture information in the user state information. The safety prompt output module is configured to output safety prompt information in a case where the gaze direction information indicates that the user does not gaze at a driving direction of the vehicle within a third duration threshold.

[0172] The adjusting device of the interface element provided in the embodiments of the present application can obtain the current user state information of the user and the visual parameter of the user relative to the screen, and generate the adjusting parameter of the interface element according to the visual parameter and the current display parameter of the interface element, so as to adjust the display parameter of the interface element, thereby realizing the dynamic adjustment of the display parameter of the interface element, and optimizing the information readability of the interface element in the screen and the comfort of human-computer interaction.

[0173] It should be noted that the device provided in the above embodiments is only exemplified by the division of the above functional modules in realizing its functions, and in actual application, the above functions can be completed by different functional modules according to needs, that is, the internal structure of the device is divided into different functional modules to complete all or part of the above described functions. In addition, the device and method embodiments provided in the above embodiments belong to the same concept, and the specific implementation process is detailed in the method embodiments, which will not be described here.

[0174] Referring to Figure 12 , a structural schematic diagram of an electronic device provided in an embodiment of the present application is shown. As shown in Figure 12 , the electronic device 1200 can include a driver monitoring system 1210, a three-dimensional scene camera device 1220, a data analysis module 1230, and an interface management module 1240, wherein, The driver monitoring system 1210 is configured to collect a user image of a user watching a target screen of a driving vehicle and analyze the user image to obtain user state information of the user. The three-dimensional scene camera 1220 is configured to obtain display parameters of each interface element on a vehicle screen of the vehicle in real time. The data analysis module 1230 is configured to determine a visual parameter of the user relative to the vehicle screen according to the user state information, and generate adjusting parameters of each interface element according to the visual parameter and the display parameter. The interface management module 1240 is configured to adjust the display parameters of each interface element on the vehicle screen based on the adjusting parameters.

[0175] In the present embodiment, the electronic device can be a vehicle controller. It can also be a vehicle-mounted terminal which can be in communication connection with the vehicle controller.

[0176] The electronic device provided by the embodiments of the present application can obtain the current user state information of a user and the visual parameter of the user relative to the screen, and generate the adjustment parameter of the interface element according to the visual parameter and the current display parameter of the interface element, so as to adjust the display parameter of the interface element, thereby realizing the dynamic adjustment of the display parameter of the interface element, optimizing the information readability of the interface element in the screen, and improving the comfort of human-computer interaction.

[0177] In the example embodiments, a computer readable storage medium is also provided, in which at least one instruction, at least one program, a code set or an instruction set is stored, and the at least one instruction, the at least one program, the code set or the instruction set is used to implement the above-mentioned interface element adjustment method when executed by a processor of an electronic device.

[0178] Optionally, the above-mentioned computer readable storage medium can be a ROM, a RAM, a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.

[0179] In the example embodiments, a computer program product is also provided, which is used to implement the above-mentioned interface element adjustment method when executed.

[0180] It should be understood that "multiple" mentioned herein refers to two or more. The "and / or" describes the association relationship of the associated objects, which means that there can be three relationships, for example, A and / or B can represent the following three cases: A exists alone, A and B exist together, and B exists alone. The character " / " generally represents that the associated objects before and after it are in an "or" relationship.

[0181] The above only describes the example embodiments of the present application, and does not limit the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principles of the present application shall be included in the protection scope of the present application.

Claims

1. A method for adjusting interface elements, characterized in that, The method includes: Obtain the user's current user status information and the display parameters of each interface element on the target screen; The user's visual parameters relative to the target screen are determined based on the user status information, and adjustment parameters for each interface element are generated based on the visual parameters and the display parameters. The display parameters of each interface element are adjusted based on the aforementioned adjustment parameters.

2. The method according to claim 1, characterized in that, The user includes a user driving a vehicle, and the target screen is the vehicle screen of the vehicle driven by the user; The user's visual parameters relative to the target screen include: the user's visual parameters relative to the vehicle screen. The adjustment of the display parameters of each interface element based on the adjustment parameters includes: The display parameters of each interface element on the vehicle screen are adjusted based on the adjustment parameters.

3. The method according to claim 1 or 2, characterized in that, The step of obtaining the user's current user status information includes: Collect the user's current image; The user's image is identified to obtain the user's status information.

4. The method according to claim 2, characterized in that, The adjustment of the display parameters of each interface element on the vehicle screen based on the adjustment parameters includes: Based on the gaze direction information in the user status information, determine the user's viewing angle information relative to the vehicle screen; Based on the viewing angle information, the display angle parameters of each interface element in the vehicle screen are determined, wherein the display angle parameters indicate that the display angle of each interface element is facing the user's viewing angle. Based on the display angle parameters and the initial display angle parameters in the display parameters, generate tilt angle adjustment parameters for each of the interface elements; The tilt angle of each interface element on the vehicle screen is adjusted according to the tilt angle adjustment parameters.

5. The method according to claim 2, characterized in that, The adjustment of the display parameters of each interface element on the vehicle screen based on the adjustment parameters includes: Based on the user's head posture information in the user status information, determine the user's viewing distance parameter relative to the vehicle screen; Based on the viewing distance parameter and the display parameter, generate the proportion adjustment parameter and / or position adjustment parameter for each interface element in the vehicle screen; Adjust the display ratio of each interface element on the vehicle screen according to the ratio adjustment parameters; and / or Adjust the display position of each interface element on the vehicle screen according to the position adjustment parameters.

6. The method according to claim 2, characterized in that, The adjustment of the display parameters of each interface element on the vehicle screen based on the adjustment parameters includes: Based on the gaze direction information in the user status information, determine the user's viewing angle information relative to the vehicle screen; Based on the viewing angle information, determine the positional difference information of the user's left and right eyes relative to the vehicle screen; Based on the position difference information, generate left and right eye image disparity parameters for each of the interface elements; Based on the left and right eye image disparity parameters and the display parameters, the relative displacement parameters of the left and right eye images corresponding to each interface element in the vehicle screen are generated; The disparity parameters of the left and right eye images are adjusted based on the relative displacement parameters.

7. The method according to claim 2, characterized in that, After obtaining the user's current user status information, the process also includes: Based on the gaze direction information and head posture information in the user status information, determine the user's gaze point on the vehicle screen and the gaze duration corresponding to the gaze point; When the gaze point is located in a designated interactive area of ​​the vehicle screen, and the gaze duration is greater than or equal to a first duration threshold and less than a second duration threshold, the interaction information with the user is output. If the gaze duration exceeds the second duration threshold, a safety warning message is output.

8. The method according to claim 2, characterized in that, After obtaining the user's current user status information, the process also includes: The user's gaze direction information is determined based on the gaze direction information and head posture information in the user status information; If the gaze direction information indicates that the user has not gazed at the direction of travel of the vehicle within a third time duration threshold, a safety warning message is output.

9. A device for adjusting interface elements, characterized in that, The device includes: The information acquisition module is used to acquire the user's current user status information and the display parameters of each interface element on the target screen; The parameter generation module is used to determine the visual parameters of the user relative to the target screen based on the user status information, and to generate adjustment parameters for each interface element based on the visual parameters and the display parameters. The parameter adjustment module is used to adjust the display parameters of each of the interface elements based on the adjustment parameters.

10. An electronic device, characterized in that, include: The system includes a driver monitoring system, a 3D scene camera device, a data analysis module, and an interface management module. The driver monitoring system is used to collect user images when a user of a vehicle is viewing a target screen, and to analyze the user images to obtain the user's status information. The three-dimensional scene camera is used to acquire the display parameters of each interface element on the vehicle screen of the vehicle. The data analysis module is used to determine the user's visual parameters relative to the vehicle screen based on the user status information, and to generate adjustment parameters for each interface element based on the visual parameters and the display parameters. The interface management module is used to adjust the display parameters of each interface element on the vehicle screen based on the adjustment parameters.

Images