Human-machine interface system for a motor vehicle and associated control method

Abstract

A human-machine interface system (10) for a motor vehicle includes: a display (12) configured to display a main menu (22) comprising a plurality of icons (23), two adjacent icons being separated by a predetermined distance; a gaze detection system (14) including a predetermined detection angle error and configured to detect the gaze position (15) of a user (11) on the display (12); a physical element (16) operable by the user (11); and a control unit (20) configured to receive: at least one detection signal from the gaze detection system (14) to determine a selected icon based on the gaze position (15) and to highlight the selected icon, and to receive at least one command signal (19) from the physical element (16) to verify the selected icon; wherein the predetermined distance is a function of the predetermined detection angle error.

Description

Technical Field

[0001] This invention relates to a human-machine interface (HMI) system for motor vehicles.

[0002] The present invention also relates to an associated method for controlling such HMI systems.

[0003] The present invention also relates to a motor vehicle including such an HMI system controlled by such a method.

[0004] This invention relates to the field of human-computer interaction systems, and more specifically to human-computer interfaces. Background Technology

[0005] In ways known to themselves, machines need to interact with humans to function properly. This interaction is crucial to harnessing the full potential offered by computer science across all fields.

[0006] Generally speaking, humans control machines by providing multiple mechanical inputs, whether these inputs are via controllers, keyboards, or any other physical element configured to convert the mechanical inputs into electrical signals that can be processed by the machine.

[0007] Research in the field of human-computer interaction systems has then explored other solutions, leading to the development of systems that enable humans to control machines using their gaze.

[0008] Such systems use image sensors configured to track the gaze in order to determine the position of the gaze on a display. These systems are integrated into motor vehicles to allow, for example, a driver to interact with the car's dashboard without moving or being distracted from the road for extended periods.

[0009] Image sensors used in such systems typically employ one of two categories of methods to perform gaze tracking. The first category, well-known in the art, is appearance-based: in this type of method, the image sensor is configured to identify the direction of the gaze by recording the user's eyes and performs a correlation analysis between the recorded image and a pre-recorded set of images included in the image sensor. The second category is model-based, where the image sensor first records an image of the user's eyes and then performs feature extraction on those images. The extracted features are then compared with multiple near-eye images to model the geometry of the user's eyes, allowing the creation of a 3D eye model of the user's eyes and the estimation of the gaze direction via that model.

[0010] Known systems using eye tracking are configured, for example, to display a predefined menu on a car's dashboard, which includes different icons associated with different functions of the car; and to detect the driver's gaze on the dashboard via an image sensor using one of the methods described above, and to perform functions associated with the icons where the gaze has already been detected on the dashboard.

[0011] However, there is a need to improve known systems and design eye-detection-based HMI systems that provide greater display flexibility and operability based on the accuracy of the eye-detection. Summary of the Invention

[0012] Therefore, the present invention relates to an HMI system for motor vehicles, the HMI system comprising:

[0013] - A display configured to show a main menu, which includes multiple icons associated with main functions, with adjacent icons separated by a predetermined distance.

[0014] - A gaze detection system configured to detect the gaze position of a user of the HMI system on the display, the gaze detection system including a predetermined detection angle error.

[0015] - A physical component that can be operated by the user.

[0016] - Control unit, configured to receive

[0017] ○ At least one detection signal from the gaze detection system to determine the selected icon based on the gaze position on the display, and to highlight the selected icon on the display, and

[0018] ○ At least one command signal from the physical element to verify the selected icon when the selected icon is highlighted;

[0019] The predetermined distance is a function of the predetermined detection angle error.

[0020] Advantageously, by adjusting the distance between two adjacent icons displayed based on the detection angle error, eye tracking becomes more accurate and intuitive for the user. More specifically, the human-machine interface system of the present invention achieves effective monitoring of the display by preventing the control unit from determining the wrong selected icon.

[0021] According to other advantageous aspects of the invention, the HMI system includes one or more of the following features, either individually or in any technically possible combination:

[0022] The control unit is also configured to, when verifying the selected icon, command the display to show a sub-icon list including the selected icon and a sub-menu with a return icon, and to receive at least one command signal from the physical element to select and verify a sub-icon from the sub-icon list.

[0023] The control unit is also configured to command the display to resume showing the main menu when a line of sight is detected on the return icon.

[0024] - The physical element is a controller, which includes at least one of a press button, a rotary button, a scroll wheel, and an optical sensor.

[0025] - Each of two adjacent icons in a pair has a side length, and the given predetermined distance is at least equal to a minimum distance calculated based on the side length and a detection error, which is a function of the predetermined detection angle error and the distance between the detected gaze position on the display and the user's eye position.

[0026] - A device for starting or stopping the line-of-sight detection system.

[0027] The present invention also relates to a method for controlling an HMI system for a motor vehicle, the method being implemented by an HMI system for a motor vehicle as briefly described above, the method comprising:

[0028] - The main menu is displayed on the display, the main menu including the plurality of icons associated with main functions, with two adjacent icons separated by the predetermined distance.

[0029] - Detect the user's gaze position on the display of the HMI system based on the predetermined detection angle error.

[0030] - Determine the selected icon based on the viewing position on the display.

[0031] - Highlight the selected icon on the display.

[0032] - When the selected main icon is highlighted, the selected icon is verified based on at least one command signal from the physical element;

[0033] The predetermined distance is a function of the predetermined detection angle error.

[0034] According to other advantageous aspects of the invention, the method includes one or more of the following features, considered individually or in any technically possible combination:

[0035] - After verifying the selected icon, a sub-menu including a sub-icon list of the selected icon and a return icon is displayed, and a sub-icon is selected from the sub-icon list according to at least one command signal from the physical element, and the sub-icon is verified according to a command signal from the physical element;

[0036] - When a line of sight is detected on the return icon, the main menu is restored.

[0037] The present invention also relates to a motor vehicle including such an HMI system, which is adapted to implement the method briefly described above. Attached Figure Description

[0038] These features and advantages of the invention will become apparent from the following description, which is provided only by way of non-limiting example and with reference to the accompanying drawings, wherein:

[0039] Figure 1 This is a schematic diagram of an HMI system according to the present invention, wherein the HMI system is controlled by a user;

[0040] Figure 2 This is a schematic diagram illustrating the operation of the HMI system according to the present invention;

[0041] Figure 3 This is a schematic diagram illustrating the relationship between detection angle error and detection error;

[0042] Figure 4 This is a schematic diagram showing the main menu and its related parameters;

[0043] Figure 5 This is a diagram illustrating the display of a submenu that includes a list of sub-icons and a back icon.

[0044] Figure 6 This is a flowchart of a method for controlling an HMI system according to the present invention, the method comprising: Figure 2 The HMI system implementation shown. Detailed Implementation

[0045] The following text refers to Figure 1 Describes an HMI system 10 controlled by the eyes of user 11.

[0046] HMI system 10 is integrated, for example, into a transportation vehicle (e.g., a car) and can be used by users, drivers, or passengers.

[0047] The HMI system described is not limited to applications in transportation vehicles, but can also be, for example, part of a home multimedia system.

[0048] like Figure 2As shown, the HMI system 10 includes a display 12, a gaze detection system 14, physical elements 16, and a control unit 20.

[0049] User 11 controls the HMI system by looking at the display 12 according to the direction of his gaze 13.

[0050] The display 12 is configured to display a main menu 22 comprising multiple icons 23, each icon being associated with a main function implemented by the HMI system 10.

[0051] For example, display 12 is a car dashboard, and the main menu 22 displayed on display 12 includes icons 23 related to navigation, music, radio, and parameters.

[0052] The display 12 is also configured to display the icon 23 horizontally or vertically, or both horizontally and vertically, wherein the icon 23 is, for example, a square with the same side length.

[0053] according to Figure 1 The example shown includes a main menu 22 displayed on monitor 12 comprising four square icons A, B, C, and D with equal side lengths, displayed horizontally and vertically. .

[0054] Alternatively, the icon can have other geometric shapes.

[0055] The display 12 is also configured to display at a distance d greater than the minimum distance. Two adjacent icons.

[0056] The distance d between two adjacent icons is the distance between the centers of two vertically or horizontally adjacent icons in the display plane, such as... Figure 4 The distance shown , , .

[0057] In other words, in Figure 4 In the example, the icons in any pair of adjacent icons (horizontally adjacent (A,B), (C,D) and vertically adjacent (A,C), (B,D)) are respectively separated by a distance of... , , All of these distances are greater than or equal to the minimum distance. .

[0058] exist Figure 4 In the example shown, each distance between adjacent icons is displayed. , , equal to minimum distance .

[0059] The gaze detection system 14 is configured to detect the gaze position 15 of the gaze 13 on the display 12 by recording at least one image of the user 11's eyes. The gaze detection method applied to the gaze detection system 14 of the present invention can be a feature-based method, an appearance-based method, or a 3D model-based method.

[0060] Feature-based methods involve detecting and tracking specific features of the eye, such as the pupil, iris, and corneal reflection. Algorithms analyze these features to determine the direction of the gaze.

[0061] Appearance-based methods use machine learning techniques to analyze the appearance of the eye region in an image. Convolutional neural networks (CNNs) are typically used to learn patterns and predict gaze direction based on visual input. This approach is more robust to changes in lighting and head position.

[0062] Methods based on 3D models create 3D models of the eye and its surrounding structures. By tracking eye movement in three dimensions, these methods can provide highly accurate gaze estimation.

[0063] For example, the gaze detection system 14 includes an image sensor or built-in camera oriented towards the user, and an associated image processing unit that processes the images captured by the image sensor. For example, the gaze detection system 14 is integrated into a DMS (“Driver Monitoring System”) or OMS (“Occupant Monitoring System”).

[0064] The gaze detection system 14 includes a predetermined detection angle error of the gaze position 15 on the display 12. and the scheduled detection speed .

[0065] Pre-determined detection angle error It is the angle between the direction of gaze 13 and the direction of the detected gaze. The direction of the detected gaze is the direction between the user 11's eye and the gaze position 15 detected by the gaze detection system 14 on the display 12.

[0066] In other words, the error of the predetermined detection angle This corresponds to the angle of detection error between the gaze position 15 detected by the gaze detection system 14 and the actual gaze position 30 on the display 12, such as... Figure 3 As shown.

[0067] Predetermined detection angle error of line of sight detection system 14 For example, it includes the range between 0.1° and 2°, and is preferably less than 2°.

[0068] In addition, the minimum distance between two adjacent icons displayed on the display 12 Based on the predetermined detection angle error To determine.

[0069] Detection error on the display It can be based on the distance between the user's eyes and the display 12. and the predetermined detection angle error This can be represented as:

[0070] [Mathematical Expression 1]

[0071]

[0072] For example, for the predetermined detection angle error =0.5° and distance =1 m, the detection error on the display is =8.7mm.

[0073] In addition, for the predetermined detection angle error =1° and distance =1 m, the detection error on the display is =17.5mm.

[0074] Then, used for calculation The given formula depends on the parameters and The value.

[0075] On the one hand, such as Figure 4 As shown, if , scheduled distance It can be directly expressed as follows:

[0076] [Mathematical Expression 2]

[0077]

[0078] For example, for detection error =17.5mm and icon side length =20mm, the minimum distance between two horizontally or vertically adjacent icons is =27.5mm.

[0079] On the other hand, if Using its formula will cause two adjacent icons to overlap. ).

[0080] Since the surface of the display 12 has limited resolution, the minimum distance between two adjacent icons 23 is increased. This will reduce the number of icons 23 that are displayed on monitor 12 at the same time.

[0081] Therefore, the minimum distance between two adjacent icons 23 It will be finely tuned to achieve the optimal predetermined detection speed. The balance between displaying a sufficient number of icons 23 on monitor 12 and displaying a sufficient number of icons 23.

[0082] Advantageously, the HMI system 10 also includes means for activating or deactivating the line-of-sight detection system 14.

[0083] Such devices may be buttons, dedicated icons on display 12, or voice commands.

[0084] Therefore, this feature allows user 11 to customize the use of the gaze detection system 14 according to his preferences, and specifically, to prevent the gaze detection system 14 from being activated when it is not needed.

[0085] According to one example, physical element 16 includes a press-down button 17 and a rotary button 18. According to another example, physical element 16 includes a press-down button 17.

[0086] Alternatively, physical element 16 may include a roller or an optical sensor.

[0087] Physical element 16 is configured to be operable by user 11 and to issue command signal 19 whenever it is actuated by user 11.

[0088] Physical element 16 is, for example, a command set built into the vehicle, preferably located near the steering wheel for easy driver operation.

[0089] Command signal 19 includes a verification command (if user 11 actuates the press button 17) or a rotation command (if user 11 actuates the rotation button 18).

[0090] If physical element 16 is as follows Figure 1 The remote control element shown transmits command signals 19 via an integrated wireless network.

[0091] Alternatively, if physical element 16 is, for example, a built-in command set in a vehicle, command signals 19 are transmitted via an integrated wired network.

[0092] For example, if user 11 actuates the press button 17, the command signal 19 will include a verification command, and if user 11 actuates the left button of the rotation button 18, the command signal 19 will include a rotation command, such as "left".

[0093] The control unit 20 is configured to determine the icon 23 selected by the user 11 by processing the detection signal 21 received from the gaze detection system 14.

[0094] Once the selected icon is determined, the control unit 20 is configured to highlight the selected icon on the display 12.

[0095] The control unit 20 is also configured to receive at least one command signal 19 from the physical element 16. If the control unit 20 receives a command signal 19 including a verification command from the physical element 16 while a selected icon is highlighted on the display 12, the control unit 20 is adapted to verify the selected icon.

[0096] "Verify the selected icon" means to execute the main function associated with the selected icon.

[0097] For example, if the control unit 20 verifies an icon related to music, then the control unit 20 performs a music-related function.

[0098] According to an alternative embodiment of the invention, the control unit 20 is configured to automatically verify the selected icon.

[0099] "Automatic verification of the selected icon" means that after the selected icon is highlighted on the display, the selected icon is verified after a predetermined time.

[0100] Advantageously, the detection signal 21 includes a distance D between the user 11's eye and the gaze position 15 detected by the gaze detection system 14, and a predetermined detection angle error. Relevant information.

[0101] exist Figure 1 and Figure 3 In the example shown, as indicated by the direction of gaze 13, user 11 is looking at the position of icon A on display 12. Gaze detection system 14 detects gaze position 15 on display 12 and sends detection signal 21 to control unit 20. Gaze position 15 intersects with icon surface A.

[0102] After processing the detection signal 21, the control unit 20 determines icon A as the selected icon and highlights icon A on the display 12. The control unit 20 then receives a command signal 19 including a verification command from the press button 17 of the physical element 16 and verifies icon A.

[0103] according to Figure 5 In the embodiment of the present invention shown, after the control unit 20 verifies the icon, the control unit 20 is further configured to command the display 12 to display a submenu 24 including a sub-icon list 25 and a return icon 26.

[0104] Return icon 26 is different from all other icons or sub-icons.

[0105] The sub-icon list 25 includes multiple sub-icons associated with sub-functions of the main function, which is associated with verified icons.

[0106] For example, if the control unit 20 verifies an icon associated with the music function, the control unit 20 will command the display 12 to show a list 25 of sub-icons associated with the music function, such as a song list.

[0107] exist Figure 5 In the example shown, after the control unit 20 verifies icon A, the control unit 20 commands the display 12 to show a list of sub-icons A1, A2, A3, A4 and a return icon 26.

[0108] The control unit 20 is also configured to receive at least one command from the physical element 16 to select and verify sub-icons 25 in the sub-icon list 25.

[0109] Advantageously, when submenu 24 is displayed on display 12, the control unit is configured to receive multiple rotation commands included in multiple command signals 19 to navigate within submenu 24 and select sub-icon 25. Control unit 20 is then adapted to receive a verification command included in command signals 19 to verify the selected sub-icon 25.

[0110] The control unit 20 is also configured to command the display 12 to resume displaying the main menu 22 when the control unit 20 receives a detection signal 21 from the eye detection system 14 indicating that a gaze position 15 has been detected on the return icon 26 on the display 12.

[0111] Advantageously, the return icon 26 is used to exit the submenu 24 and display back the main menu 22.

[0112] Now refer to Figure 6 To explain the operation of HMI system 10, the figure shows a flowchart of the steps of a method for controlling HMI system 10, which is implemented by HMI system 10.

[0113] During the initial display step 100, a main menu 22 comprising multiple main icons 23 is displayed on the display of the HMI system 10. The control unit 20 is configured to command the display 12 to display two adjacent icons with a predetermined distance based on a predetermined detection angle error.

[0114] During the subsequent detection step 110, the gaze detection system 14 detects the gaze position 15 on the display 12. The gaze detection system 14 records the distance between the user 11 and the gaze position 15. The system then transmits a detection signal 21, which includes the distance, to the control unit 20, along with an image of the user 11's eyes. And the line-of-sight position 15 on the display 12.

[0115] During the subsequent determination step 120, the control unit 20 receives the detection signal 21 and determines the selected icon based on the information contained in the detection signal 21.

[0116] During the subsequent highlighting step 130, the control unit 20 commands the display 12 to highlight the selected icon.

[0117] During the subsequent verification step 140, the control unit 20 receives a verification command contained in the command signal 19 from the physical element 16 when the selected icon is highlighted, and verifies the selected icon.

[0118] According to an alternative embodiment of the invention, during verification step 140, the control unit verifies the selected icon after a predetermined time following the highlighting of step 130.

[0119] Then, during the subsequent submenu display step 150, the control unit 20 commands the display 12 to display a submenu 24, which includes a list 25 of sub-icons associated with the selected icon.

[0120] During the recovery step 160, the gaze detection system 14 detects the gaze position 15 on the return icon 26, and as performed during step 100, the control unit 20 commands the display 12 to resume displaying the main menu 22.

[0121] During selection step 170, control unit 20 receives multiple rotation commands contained in multiple command signals 19 from rotation button 18 of physical element 16, and control unit 20 selects the sub-icon from sub-icon list 25.

[0122] Then, during the sub-icon verification step 180 after step 170, the control unit 20 receives a verification command contained in the command signal 19 from the press button 17 of the physical element 16, and the control unit 20 verifies the sub-icon 25 selected during step 170.

[0123] According to an alternative embodiment of the invention, recovery step 160 is performed simultaneously with step 170 or step 180.

[0124] Therefore, advantageously, by adapting the display of the main menu 22 to a function of the predetermined detection angle error based on the distance between the two icons 23, the detection step 110 becomes more efficient and accurate. The user 11 does not need to repeatedly look at the display 12 to select the correct icon 23, and such operation provides safety during use if the HMI system 10 is integrated into a vehicle and used by the driver while driving.

[0125] Furthermore, advantageously, achieving a predetermined distance between two adjacent icons based on the error range of the gaze detection system 14 improves the user experience. In fact, this means that the control unit 20 is less likely to determine the selected icon that the user 11 did not originally intend to choose due to blinking, slight shifts in gaze, or other "noise" that may interfere with the operation of the gaze detection system 14.

Claims

1. A human-machine interface (HMI) system (10) for motor vehicles, the HMI system comprising: - Display (12), the display being configured to display a main menu (22), the main menu including a plurality of icons (23) associated with main functions, two adjacent icons being separated by a predetermined distance. - A gaze detection system (14), configured to detect the gaze position (15) of the user (11) of the HMI system (10) on the display (12), the gaze detection system (14) including a predetermined detection angle error, -Physical element (16), which can be operated by the user (11), - Control unit (20), the control unit is configured to receive At least one detection signal (21) from the gaze detection system (14) is used to determine the selected icon based on the gaze position (15) on the display (12), and to highlight the selected icon on the display (12). ○ At least one command signal (19) from the physical element (16) to verify the selected icon when the selected icon is highlighted; The predetermined distance is a function of the predetermined detection angle error.

2. The HMI system according to claim 1, wherein the control unit (20) is further configured to, when verifying the selected icon, command the display (12) to display a sub-menu (24) including a sub-icon list (25) of the selected icon and a return icon (26), and to receive at least one command signal (19) from the physical element (16) to select and verify a sub-icon from the sub-icon list (25).

3. The HMI system according to claim 2, wherein the control unit (20) is further configured to command the display (12) to resume displaying the main menu (22) when a gaze position (15) is detected on the return icon (26).

4. The HMI system according to any one of the preceding claims, wherein the physical element (16) is a controller, the controller comprising at least one of a push button (17), a rotary button (18), a scroll wheel, and an optical sensor.

5. The HMI system according to claims 1 to 4, wherein each of a pair of two adjacent icons has a side length, and wherein the given predetermined distance is at least equal to a minimum distance calculated based on the side length and a detection error, the detection error being a function of the predetermined detection angle error and the distance between the gaze position (15) detected on the display (12) and the eye position of the user (11).

6. The HMI system according to any one of the preceding claims, the HMI system further comprising means for activating or deactivating the gaze detection system (14).

7. A method for controlling a human-machine interface (HMI), said method being implemented by an HMI system (10) according to any one of the preceding claims, said method comprising: - The main menu (22) is displayed (100) on the display (12), the main menu including the plurality of icons (23) associated with the main functions, two adjacent icons being separated by the predetermined distance. - Detect (110) the line-of-sight position (15) of the user (11) of the HMI system (10) on the display (12) according to the predetermined detection angle error. -The icon selected (120) is determined (120) based on the viewing position (15) on the display (12). - The selected icon (130) is highlighted (130) on the display (12), and - When the selected main icon is highlighted, the selected icon is verified (140) according to at least one command signal (19) from the physical element (16); The predetermined distance is a function of the predetermined detection angle error.

8. The method according to claim 7, further comprising: After verifying the selected icon, a sub-menu (24) is displayed (150) including a sub-icon list (25) of the selected icon and a return icon (26), and a sub-icon is selected (170) from the sub-icon list (25) according to at least one command signal (19) from the physical element (16), and the sub-icon is verified (180) according to a command signal (19) from the physical element (16).

9. The method according to claim 8, further comprising: When a line of sight (15) is detected on the return icon (26), the main menu (22) is restored (160).

10. A motor vehicle comprising an HMI system (10) according to claims 1 to 6, the HMI system being adapted to implement the method according to claims 7 to 9.

Images