Operating device

Abstract

An operating device 1 includes: an operating unit 2 that is disposed in a vehicle 8 and that accepts a first operation; a gaze detecting unit 4 that detects a gaze 40 of an occupant 9 of the vehicle 8; and a control unit 6 that, when the operating unit 2 has accepted the first operation and determined the presence of a gaze position display area 41 in which the gaze 40 of the occupant 9 is located, among a plurality of display areas for displaying images, enables only operations performed with respect to the gaze position display area 41 via the operating unit 2.

Description

operating device CROSS-REFERENCE TO RELATED APPLICATIONS

[0001] This application claims priority from Japanese Patent Application No. 2023-097272, filed on June 13, 2023, the entire contents of which are incorporated herein by reference.

[0002] The present invention relates to an operating device.

[0003] 2. Description of the Related Art There is known a display device that includes a display unit that performs display in a display area and a display control unit that causes the display unit to perform display in the display area (see, for example, Patent Document 1).

[0004] The display control unit of this display device can enlarge the display mode of the displayed symbol in accordance with the user's line of sight. This symbol is an operation icon for operating the display device.

[0005] Japanese Patent Application Laid-Open No. 2022-148024

[0006] When there are multiple display areas, the total of the multiple display areas is larger than each individual display area. In a conventional display device, for example, when the entire multiple display areas are to be operated, even if the icon that the user is gazing at is enlarged, the total of the multiple display areas is large, so the operability of the icon does not necessarily improve.

[0007] An object of the present invention is to provide an operating device that can improve operability.

[0008] An operating device according to one embodiment of the present invention is arranged in a vehicle and includes an operating unit that accepts a first operation, a gaze detection unit that detects the gaze of a vehicle occupant, and a control unit that, when the operating unit accepts the first operation and determines, among multiple display areas that display images, the existence of a gaze position display area in which the occupant's gaze is located, only allows operation on the gaze position display area via the operating unit.

[0009] According to one embodiment of the present invention, it is possible to provide an operating device that improves operability.

[0010] FIG. 1A is an explanatory diagram showing a vehicle in which an operation device according to an embodiment is arranged. FIG. 1B is an explanatory diagram showing a steering wheel in which an operation device according to an embodiment is arranged. FIG. 2A is a block diagram of an operation device according to an embodiment. FIG. 2B is a block diagram of a vehicle control system including an operation device according to an embodiment. FIG. 3A is a diagram for explaining gaze detection by a gaze detection unit of an operation device according to an embodiment. FIG. 3B is an explanatory diagram showing a display device. FIG. 3C is an explanatory diagram showing a display device. FIG. 4A is a diagram for explaining a first operation in an operation device according to an embodiment. FIG. 4B is an explanatory diagram showing a display area in which a gaze is detected. FIG. 4C is an explanatory diagram showing a gaze position display area. FIG. 5A is an explanatory diagram showing an operation to change a layout. FIG. 5B is an explanatory diagram showing a display area in which a gaze is detected. FIG. 5C is an explanatory diagram showing a change of layout. FIG. 6A is an explanatory diagram showing an operation to swap display areas. FIG. 6B is an explanatory diagram showing a display area in which a gaze is detected. FIG. 6C is an explanatory diagram showing swapping of display areas. FIG. 7 is a flowchart showing an operation to make a desired display area operable. 8 and 9 are flowcharts showing the operation of changing the layout of the display area, and the operation of switching the display area, respectively.

[0011] (Summary of the embodiment) An operating device according to the embodiment is disposed in a vehicle and includes an operating unit that accepts a first operation, a gaze detection unit that detects the gaze of a vehicle occupant, and a control unit that, when the operating unit accepts the first operation and determines, among a plurality of display areas that display images, the existence of a gaze position display area where the occupant's gaze is located, allows only operation on the gaze position display area via the operating unit.

[0012] This operating device allows operation of only the gaze position display area where the occupant directs his or her gaze among the multiple display areas, so the area that can be operated is limited compared to when all of the multiple display areas are the target of operation, improving operability.

[0013] [Embodiment] (Outline of operation device 1) Fig. 1A is an explanatory diagram showing a vehicle in which an operation device according to an embodiment is arranged, and Fig. 1B is an explanatory diagram showing a steering wheel in which an operation device according to an embodiment is arranged. Fig. 2A is a block diagram of an operation device according to an embodiment, and Fig. 2B is a block diagram of a vehicle control system including the operation device. Fig. 3A is a diagram for explaining gaze detection by a gaze detection unit of an operation device according to an embodiment, and Figs. 3B and 3C are diagrams showing a display device.

[0014] In the drawings relating to the embodiments described below, the ratios and shapes of figures may differ from the actual ratios and shapes. Also, in Figures 2A and 2B, arrows indicate the flow of main signals and information.

[0015] As shown in Figures 1A, 2A, 2B, and 3A, the operating device 1 is arranged in a vehicle 8 and has an operating unit 2 that accepts a first operation, a gaze detection unit 4 that detects the gaze 40 of an occupant 9 of the vehicle 8, and a control unit 6 that, when the operating unit 2 accepts the first operation and determines the existence of a gaze position display area 41 in which the gaze 40 of the occupant 9 is located among multiple display areas that display images, allows only operation of the gaze position display area 41 via the operating unit 2.

[0016] The control unit 6 is configured to allow only operation via the operation unit 2 on the gaze position display area 41, and to display a pointer 410 that can be operated via the operation unit 2 in the gaze position display area 41.

[0017] The operation unit 2 also has a left operation unit and a right operation unit arranged on the left and right sides of the steering wheel 81 of the vehicle 8. The control unit 6 defines an operation performed on the left operation unit and the right operation unit, which overlaps for at least a partial period, as a first operation. This operation which overlaps for at least a partial period means that the period of the operation performed on the left operation unit and the period of the operation performed on the right operation unit overlap for at least a partial period.

[0018] The left and right operation sections of this operation section 2 are touchpads that accept touch operations. In the following, as an example, the left operation section is referred to as the left touchpad 20, and the right operation section is referred to as the right touchpad 21. Note that the left and right operation sections are not limited to touchpads, and may be joysticks or cross keys with a push button located in the center.

[0019] When the control unit 6 receives touch operations performed on the left touchpad 20 and the right touchpad 21, the touch operations overlapping at least partially, and determines the presence of the gaze position display area 41, the control unit 6 enables only operations on the gaze position display area 41. As an example, the first operation is a touch operation performed on the left touchpad 20 and the right touchpad 21, the touch operations overlapping at least partially.

[0020] 4A , which will be described later, this touch operation indicates a contact of a left operating finger 910 of a left hand 91 with the operating surface 200 of the left touchpad 20, and a contact of a right operating finger 920 of a right hand 92 with the operating surface 210 of the right touchpad 21. Note that the left operating finger 910 and the right operating finger 920 are thumbs as an example, but are not limited to this, and one may be a thumb and the other another finger, or both may be fingers other than thumbs.

[0021] The operation unit 2 further accepts a second operation. The control unit 6 is configured to move the gaze position display area 41 to change the layout of the multiple display areas when a second operation is performed on the operation unit 2 after allowing only operations on the gaze position display area 41, and then a third operation is performed.

[0022] Therefore, the operation unit 2 has a left touchpad 20 and a right touchpad 21 arranged on the left and right sides of the steering wheel 81 of the vehicle 8, and a specific operation unit arranged adjacent to at least one of the left touchpad 20 and the right touchpad 21 and which accepts a second operation.

[0023] The specific operation units in this embodiment are, for example, a left operation bar 22 arranged adjacent to the left touchpad 20 and a right operation bar 23 arranged adjacent to the right touchpad 21. The left operation bar 22 and the right operation bar 23 are push switches, but are not limited to these, and may be capacitive touch sensors.

[0024] As an example, the second operation is a long press operation performed on at least one of the left operation bar 22 and the right operation bar 23, but is not limited to this, and may also be a long press operation, touch operation, tap operation, etc. performed at a specific position on at least one of the left touchpad 20 and the right touchpad 21.

[0025] The third operation is, for example, a drag-and-drop operation performed on one of the left touchpad 20 and the right touchpad 21, or an operation including a selection operation of a destination display area by changing the line of sight of the occupant 9 and a release of the second operation, but is not limited to these, and may also be a flick operation or a tracing operation performed on one of the left touchpad 20 and the right touchpad 21. In the case of a flick operation or a tracing operation, the desired display area may be moved to a position corresponding to the end position of the operation, or the position of the desired display area may be changed by repeating a flick operation or a tracing operation multiple times until the second operation is released and then releasing the second operation. The third operation in this embodiment is, for example, an operation that enables a layout change by a drag-and-drop operation and a layout change by moving the line of sight.

[0026] The occupant 9 can activate the desired display area by performing touch operations on the left touchpad 20 and the right touchpad 21 and directing his / her gaze 40 toward the display area he / she wants to operate, that is, by selecting the desired display area and making it operable.

[0027] In this embodiment, the occupant 9 is, for example, a driver seated in the driver's seat 800, but is not limited to this and may be an occupant seated in the passenger seat 801. In this case, the occupant 9 seated in the passenger seat 801 can, for example, perform a first operation on the touchpad device 86 arranged on the floor console 802 and direct his / her line of sight 40 to the desired display area, thereby making only the desired display area operable. The occupant 9 seated in the passenger seat 801 can also change the layout of the above-mentioned multiple display areas or swap the display positions by combining the operation with a switch arranged near the touchpad device 86.

[0028] As shown in FIG. 1A , vehicle 8 is provided with a display device 84 having a large display screen 840 that extends from in front of driver's seat 800 to in front of passenger seat 801. As shown in FIG. 1A , this display screen 840 is large enough to cover most of the left-right direction of instrument panel 803. However, because they are disposed on steering wheel 81, operation surface 200 of left touchpad 20 and operation surface 210 of right touchpad 21 are much smaller than the large display screen 840. It is difficult for occupant 9 to operate the entire display screen 840 by operating operation surface 200 and operation surface 210. The operation device 1 of this embodiment is configured to reduce the difficulty caused by the difference in ratio by allowing operation on a display area basis rather than the entire display screen 840.

[0029] As an example, the display area is an area obtained by dividing the display screen 840 in the vertical direction, but is not limited to this, and the display area may be shaped so that at least a portion of the display area overlaps in the vertical direction.

[0030] 2B , the operation device 1, together with a display device 84, constitutes a vehicle control system 82. The vehicle control system 82 is configured to include, for example, a vehicle control device 83, a display device 84, a mirror device 85, a touchpad device 86, an in-vehicle device 87, and a vehicle LAN (Local Area Network) 88.

[0031] The vehicle control device 83 is a microcomputer that comprehensively controls the vehicle 8, and is configured to control the drive device, automatic driving control, driving assistance control, etc. The vehicle control device 83 controls the display device 84, for example.

[0032] Examples of the display device 84 include an organic electroluminescence (EL) display, a mini-light emitting diode (LED), and a micro-LED display. The display device 84 has, for example, a display screen 840 that is long in the left-right direction of the vehicle 8. The display screen 840 is divided into a plurality of display areas.

[0033] The multiple display areas may be configured with multiple displays. That is, the display device 84 may be configured with multiple displays. As a modified example, the display device 84 may be configured with a head-up display that projects an image onto the windshield 804, or may be configured to include a head-up display.

[0034] 3B shows an example in which a display screen 840 is divided into four display areas: a first display area 841 to a fourth display area 844. The first display area 841 to the fourth display area 844 may have the same shape or different shapes. The first display area 841 displays, as an example, a display image 841a relating to multiple functions of an air conditioner. The second display area 842 displays, as an example, a display image 842a relating to a map image of a navigation system. The third display area 843 displays, as an example, a display image 843a relating to speed, direction indication, etc. The fourth display area 844 displays, as an example, a display image 844a relating to a fuel gauge, a water temperature gauge, etc.

[0035] 3C shows an example in which the display screen 840 is divided into three display areas: a first display area 841 to a third display area 843. As an example, the first display area 841 displays a display image 841a relating to the adjustment function of the mirror device 85. As an example, the second display area 842 displays a display image 842a relating to the seat device. As an example, the third display area 843 displays a display image 843a relating to speed, direction indication, etc.

[0036] Here, the control unit 6 is configured to keep at least the display area that displays the speed of the vehicle 8, among the multiple display areas, in front of the occupant 9, and to restrict changes to the layout of this display area and changes to the display position of this display area. The third display area 843 in FIG. 3B and the third display area 843 in FIG. 3C are display areas that display the speed. As an example, this display area that displays the speed is required by law to be displayed in front of the driver so that it is easily visible. Therefore, the control unit 6 is configured to keep the display position of the display area that displays the speed, and not to change the layout including this display area or change its display position.

[0037] The mirror device 85 is used to check the area behind the vehicle 8 and may be a physical mirror or an electronic mirror. If the mirror device 85 is a physical mirror, it is configured to allow the angle of the mirror portion to be changed, as shown in the first display area 841 in FIG. 3C . If the mirror device 85 is an electronic mirror, it is configured to include a camera and display an image of the rear view in a display area that can be enlarged, reduced, or adjusted for brightness, and a warning of approaching vehicles. If the mirror device 85 is an electronic mirror, the display device 84 displays images of the left and right rear views in display areas located at the left and right ends. If the image on the passenger seat 801 side is difficult to see, the occupant 9 can change the layout of the display area to display it closer to themselves.

[0038] 1A, the touchpad device 86 is disposed on a floor console 802 between a driver's seat 800 and a passenger's seat 801. The touchpad device 86 is a capacitive touchpad, similar to the left touchpad 20 and the right touchpad 21, and is configured to detect touch operations and the like.

[0039] Examples of the in-vehicle device 87 include an air conditioning device that adjusts the temperature inside the vehicle, a navigation device that displays a map of the current location and provides guidance to the destination, a seat device that controls the position and inclination of the seat, a music and video playback device that plays music and videos, etc. The display device 84 is configured to display information related to these in-vehicle devices 87 and to display images to which adjustments, function on / off, etc. are assigned.

[0040] The vehicle control system 82 is connected to the operation device 1, the vehicle control device 83, the display device 84, and the like via a vehicle LAN 88. The vehicle LAN 88 is a vehicle network such as a CAN (Controller Area Network) or a LIN (Local Interconnect Network) that enables mutual exchange of signals, information, and the like by wire or wirelessly.

[0041] As described above, the operation unit 2 is configured to include the left touch pad 20, the right touch pad 21, the left operation bar 22, and the right operation bar 23.

[0042] The left touchpad 20 and the right touchpad 21 are, for example, capacitive touchpads, but are not limited to this and may be pressure-sensitive touchpads, etc. The left touchpad 20 and the right touchpad 21 are capable of, for example, touch operations, tap operations, double tap operations, tracing operations, drag-and-drop operations, flick operations, pinch-out (pinch open) operations, and pinch-in (pinch close) operations. These operations may be performed on the left touchpad 20 and the right touchpad 21 separately, or may be performed separately.

[0043] Here, to distinguish it from a tap operation, the touch operation refers to an operation of continuously touching the operation surface or an operation of pressing and holding the operation surface, rather than an operation of momentarily touching the operation surface.

[0044] 1A, the steering wheel 81 on which the operation unit 2 is arranged is an irregular-shaped steering wheel that has no top and makes it easy to see the display device 84. The left touchpad 20 is arranged on the tip side of the left grip portion 810 of the steering wheel 81. The right touchpad 21 is arranged on the tip side of the right grip portion 811. The operation surface 200 of the left touchpad 20 and the operation surface 210 of the right touchpad 21 each have a rectangular shape with one corner facing upward.

[0045] The left operation bar 22 is arranged along the upper side of the operation surface 200 of the left touchpad 20. The right operation bar 23 is arranged along the upper side of the operation surface 210 of the right touchpad 21.

[0046] The left touchpad 20 and the right touchpad 21 store left touchpad operation information S １ and right touchpad operation information S ２ to the control unit 6. The left operation bar 22 and the right operation bar 23 also output left operation information S based on the push operation or long press operation. ３ and right operation information S ４ is output to the control unit 6.

[0047] (Configuration of gaze detection unit 4) The gaze detection unit 4 is, for example, electrically connected to the camera 3. As shown in Figures 1A and 3A, the camera 3 is arranged so that the face 90 of the occupant 9 seated in the driver's seat 800 falls within the imaging range 30. The camera 3 is, for example, arranged on an instrument panel 803 in front of the occupant 9.

[0048] The camera 3 captures an image of the imaging range 30 using infrared rays, for example, and stores image information S ５ to the line-of-sight detection unit 4. The camera 3 may be disposed as an in-vehicle device 87 and connected to the line-of-sight detection unit 4 via a vehicle LAN 88.

[0049] The line-of-sight detection unit 4 detects the image information S acquired from the camera 3.５ The line of sight 40 of the occupant 9 is detected based on the above. One example of a method for detecting the line of sight 40 is to detect the line of sight 40 from the positional relationship between a stationary part (reference point) of the eye of the occupant 9 and a moving part (moving point), but the method is not limited to this.

[0050] The gaze detection unit 4 detects, for example, gaze information S ６ to the control unit 6. That is, the line-of-sight detection unit 4 calculates the coordinates where the line of sight 40 intersects with the display screen 840, and outputs line-of-sight information S ６ Output as

[0051] (Configuration of Communication Unit 5) The communication unit 5 is configured to exchange information and signals between the operation device 1, the vehicle control device 83, the display device 84, and the like via the vehicle LAN 88.

[0052] (Configuration of control unit 6) The control unit 6 is a microcomputer including a CPU (Central Processing Unit) that performs calculations and processing on acquired data according to a stored program, and semiconductor memories such as a RAM (Random Access Memory) and a ROM (Read Only Memory). The ROM stores, for example, programs for the operation of the control unit 6. The RAM is used as a storage area for temporarily storing calculation results, for example. The control unit 6 also has a means for generating a clock signal therein and operates based on this clock signal.

[0053] 2A , the control unit 6 has operation pattern information 60. This operation pattern information 60 includes information on an operation pattern that enables operation of a desired display area, information on an operation pattern for changing the layout of the gaze position display area 41, and information on an operation pattern for swapping the display areas. Note that the operation pattern can be set by the occupant 9, for example. That is, as will be described later, when changing the layout, the occupant 9 may be required to perform a long press on the right operation bar 23 but may also be required to perform a long press on the left operation bar 22.

[0054] As an example, as shown in FIG. 2B , the control unit 6 receives display area information S ９ The display area information S ９ is not limited to the vehicle control device 83, and if there is a control device other than the vehicle control device 83 that controls the display device 84, it may be obtained from this control device.

[0055] Display area information S ９ is, for example, information specifying the range of the display area on the display screen 840. The control unit 6 receives the line-of-sight information S ６ and the display area information S acquired from the vehicle control device 83 ９ Based on this, it is determined in which display area the line of sight 40 is located, that is, the line of sight position display area 41.

[0056] The control unit 6 generates display instruction information S regarding the gaze position display area 41. ７ to the vehicle control device 83 via the vehicle LAN 88. The control unit 6 also outputs the operation unit operation information S ８ and outputs the display instruction information S to the vehicle control device 83 via the vehicle LAN 88. ７ The control unit 6 also receives the operation unit operation information S ８ may be configured to transmit the information to the in-vehicle device 87 corresponding to the gaze position display area 41.

[0057] The vehicle control device 83 receives the display instruction information S ７ Display control information S that controls the display of the display device 84 based on １０ This display control information S １０ The image information includes at least information about an image related to the in-vehicle device 87 to be displayed on the display device 84 and information about the display area in which the image is to be displayed.

[0058] 3B, when the display device 84 divides the display screen 840 into a first display area 841 to a fourth display area 844 and displays display images 841a to 844a, the vehicle control device 83 acquires information about the display images 841a to 844a from the related in-vehicle device 87 or the like and displays the information on the display screen 840. Then, the vehicle control device 83 receives the operation unit operation information S ８ When the user acquires the command, the user can move the pointer 410 displayed in the operable display area, scroll the menu, etc.

[0059] The vehicle control device 83 also displays the display instruction information S ７ When the above information is acquired, the eye gaze position display area 41 is activated, the layout is changed, or the display position is swapped.

[0060] Below, an example of an operation for making a desired display area operable, an operation for changing the layout of the display areas, and an operation for switching the display areas will be described.

[0061] Regarding the operation for enabling a desired display area to be operated: FIG. 4A is a diagram for explaining an example of a first operation of the operating device according to the embodiment; FIG. 4B is a diagram showing an example of a display area where a gaze is detected; and FIG. 4C is a diagram showing an example of an operable gaze position display area. In FIG. 4B, a plurality of display areas are displayed on a display screen 840: a first display area 841 displaying an "A" display image 841a, a second display area 842 displaying a "B" display image 842a, a third display area 843 displaying a "C" display image 843a, and a fourth display area 844 displaying a "D" display image 844a. In FIG. 4C, the operable gaze position display area 41 is indicated by diagonal lines. FIGS. 5A to 5C and 6A to 6C, which will be described later, illustrate display areas and display images based on FIGS. 4A to 4C.

[0062] In order to operate the desired display area, the occupant 9 performs a first operation on the operation unit 2, and then directs his / her line of sight 40 to the desired display area. Figure 4B shows the case where the line of sight 40 is directed to the second display area 842. In other words, the desired display area of ​​the occupant 9 is the second display area 842.

[0063] As an example, as shown in Figure 4A, this first operation is an operation in which the left operating finger 910 of the left hand 91 presses and holds the operating surface 200 of the left touchpad 20, and the right operating finger 920 of the right hand 92 presses and holds the operating surface 210 of the right touchpad 21, with the periods of pressing and holding overlapping at least partially.

[0064] The control unit 6 receives the left touchpad operation information S output from the left touchpad 20. １ , and right touchpad operation information S output from the right touchpad 21 ２ and the operation pattern information 60, the control unit 6 controls the gaze detection unit 4 to check the gaze 40. Note that the control unit 6 may be configured to check the operation after determining that the gaze 40 is located on the display screen 840.

[0065] When the line of sight 40 is detected in one of the display areas while the long press operation on the left touchpad 20 and the right touchpad 21 is continuing, the control unit 6 outputs display instruction information S ７ In FIG. 4B , the display area where the line of sight 40 is detected is the second display area 842, so the control unit 6 determines that the second display area 842 is the line of sight position display area 41 and is the desired display area of ​​the occupant 9, and outputs the display instruction information S ７ Output.

[0066] If the occupant 9 wishes to operate a display area other than the second display area 842, he / she can perform a long press on the left touchpad 20 and the right touchpad 21, and move the pointer 410 by directing his / her gaze 40 to the desired display area, thereby enabling operation.

[0067] - Regarding operations for changing the layout of the display area Figure 5A is a diagram showing an example of an operation for changing the layout using an operating device according to an embodiment, Figure 5B is a diagram showing an example of a display area in which a line of sight is detected, and Figure 5C is a diagram showing an example of a change in layout.

[0068] When changing the layout of a desired display area, the occupant 9 first performs a first operation to make the display area to be changed operable, and then directs the line of sight 40 to the desired display area. Figure 5B shows the case where the line of sight 40 is directed to the first display area 841. Note that if the display area to be changed is already active, it is not necessary to perform the first operation.

[0069] Next, after the pointer 410 moves to the first display area 841, the occupant 9 performs a long press operation on the right operation bar 23 as a second operation, and then performs a drag-and-drop operation on the operation surface 200 of the left touchpad 20 as a third operation, as shown in Fig. 5A . Note that the occupant 9 may also perform a drag-and-drop operation on the operation surface 210 of the right touchpad 21 while performing a long press operation on the left operation bar 22.

[0070] When the occupant 9 wants to display the first display area 841 at the position of the third display area 843, he / she performs a drag operation from left to right on the operation surface 200 as shown in Figure 5C, and when the first display area 841 reaches the desired position, he / she performs a drop operation, i.e., releases the left operation finger 910 from the operation surface 200.

[0071] The control unit 6 receives the right operation information S output from the right operation bar 23 in response to the second operation. ４ and left touchpad operation information S output from the left touchpad 20 when the third operation is performed. １ and the operation pattern information 60, the control unit 6 determines that a drag-and-drop operation is being performed on the first display area 841. The control unit 6 determines that a drag-and-drop operation is being performed on the first display area 841 based on the display instruction information S ７ Output.

[0072] - Regarding the operation of swapping the display area Fig. 6A is a diagram showing an example of an operation of swapping the display area performed on the operation device according to the embodiment, Fig. 6B is a diagram showing an example of a display area in which the line of sight is detected, and Fig. 6C is a diagram showing an example of swapping the display area.

[0073] When switching the display position of a desired display area, the occupant 9 first performs a first operation to set the display area to be switched to the gaze position display area 41, and then directs the gaze 40 to the desired display area. Fig. 6B shows a case where the first display area 841 is set to the gaze position display area 41. Note that if the desired display area is already active, the first operation does not need to be performed.

[0074] Next, as shown in Fig. 6A, the occupant 9 performs a long press on the right operation bar 23 as a second operation, and directs the line of sight 40 to a display area that the occupant 9 wishes to replace with the line of sight position display area 41. Fig. 6B shows the case where the line of sight 40 is directed to a fourth display area 844.

[0075] 6A to 6C , the occupant 9 can then switch the display areas by performing a long press on the right operation bar 23 as a second operation, sending the line of sight 40 to the fourth display area 844, and then releasing the right operation finger 920 from the right operation bar 23. As an example, the third operation is an operation that includes an operation of sending the line of sight 40 to the fourth display area 844 and an operation of releasing the right operation finger 920 from the right operation bar 23. Note that the occupant 9 may also perform a long press on the left operation bar 22 and send the line of sight 40 while performing the long press operation.

[0076] The control unit 6 receives the line-of-sight information S output from the line-of-sight detection unit 4. ６ and right operation information S output from the right operation bar 23 ４ and the operation pattern information 60, the control unit 6 determines that the long press operation on the right operation bar 23 has been released in order to switch the display positions of the first display area 841 and the fourth display area 844 while the first display area 841 is active. ７ Output.

[0077] (Operation) An example of the operation of the operating device 1 of this embodiment will be described below with reference to the flowcharts of FIGS.

[0078] Operation for Making a Desired Display Area Operable The control unit 6 of the operation device 1 monitors whether a first operation is performed on the operation unit 2. If "Yes" is determined in Step 1, that is, if touch operations on the left touchpad 20 and the right touchpad 21 that overlap at least partially in time, that is, the first operation, are detected (Step 1: Yes), the control unit 6 controls the gaze detection unit 4 to start detecting the gaze 40 of the occupant 9 (Step 2).

[0079] When the touch operation on the left touchpad 20 and the right touchpad 21, i.e., the first operation, continues and the line of sight 40 is detected in any of the display areas among the plurality of display areas (Step 3: Yes), the control unit 6 activates the display area where the detected line of sight 40 is located, i.e., makes the display area operable. ７ is output (Step 4).

[0080] Here, in step 3, if the first operation continues and the line of sight 40 is not detected in any of the multiple display areas (Step 3: No), the control unit 6 proceeds to step 1.

[0081] Operation of Changing the Layout of the Display Area When “Yes” in Step 10 is established, that is, when there is an active display area (Step 10: Yes), a long press operation on the operation bar, that is, a second operation, is performed (Step 11: Yes), and a drag-and-drop operation is performed as a third operation during the second operation (Step 12: Yes), the control unit 6 changes the layout of the display area based on the performed drag-and-drop operation (Step 13), and outputs display instruction information S ７ Output.

[0082] Here, in step 11, if the second operation has not been performed (Step 11: No), the control unit 6 proceeds to step 10.

[0083] Furthermore, in step 12, if the third operation is not performed during the second operation (Step 12: No), the control unit 6 proceeds to step 10.

[0084] - Operation of switching display areas When "Yes" is determined in step 20, i.e., there is an active display area (Step 20: Yes), and a long press operation, i.e., the second operation, is performed on the operation bar (Step 21: Yes), the control unit 6 controls the gaze detection unit 4 to start detecting the gaze 40 of the occupant 9 (Step 22).

[0085] When the second operation is released, if the control unit 6 detects a third operation in which the line of sight 40 is positioned in any of the display areas (Step 23: Yes), the control unit 6 outputs display instruction information S ７ Output.

[0086] Here, in step 21, if the second operation has not been performed (Step 21: No), the control unit 6 proceeds to step 20.

[0087] If the third operation is not detected in step 23 (step 23: No), the control unit 6 proceeds to step 20.

[0088] Effect of the embodiment The controller device 1 according to the present embodiment can improve operability. Specifically, the controller device 1 can operate only a desired display area based on a first operation on the operation unit 2 and the position of the line of sight 40. Therefore, compared to when multiple display areas are to be operated, the controller device 1 can reduce the difficulty caused by differences in ratios and improve operability by allowing operation on a display area basis.

[0089] The operating device 1 can display the pointer 410 so that only the desired display area can be operated by combining the first operation on the operating unit 2 and the position of the line of sight 40, so that the starting point of the pointer 410 can be easily changed compared to when this configuration is not adopted.

[0090] The operating device 1 can operate a pointer 410 displayed on a large display screen 840 using a left touchpad 20 and a right touchpad 21, which have small operating surfaces 200 and 210 arranged on the steering wheel 81, and therefore the time required for operation can be reduced compared to when the entire display screen is the target of operation.

[0091] The operating device 1 can easily layout and swap the display area based on the operation on the operating unit 2 and the position of the line of sight 40, so that the display area to be operated can be moved to a position closer to the occupant 9 compared to when this configuration is not adopted, thereby further improving operability.

[0092] In the operating device 1, the eye gaze position display area 41 has a difference in color and brightness between the display of the pointer 410 and other display areas, so it is easier to see which display area has become active than when this configuration is not adopted.

[0093] By limiting the operable display area, the operating device 1 can be operated using the left touchpad 20 and the right touchpad 21 in the same manner as a multi-function mobile phone or tablet terminal, making it easy to operate even if the display screen 840 is large.

[0094] The controller device 1 according to the above-described embodiment and modified examples may be partially realized by a program executed by a computer, an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or the like, depending on the application.

[0095] Although several embodiments and modifications of the present invention have been described above, these embodiments and modifications are merely examples and do not limit the scope of the claimed invention. These novel embodiments and modifications can be embodied in various other forms, and various omissions, substitutions, modifications, etc. can be made without departing from the spirit of the present invention. Furthermore, not all combinations of features described in these embodiments and modifications are necessarily essential to the means for solving the problems of the invention. Furthermore, these embodiments and modifications are included within the scope and spirit of the invention, and are included in the scope of the invention and its equivalents as defined in the claims.

[0096] REFERENCE SIGNS LIST 1 Operation device 2 Operation unit 4 Gaze detection unit 6 Control unit 8 Vehicle 9 Occupant 20 Left touchpad 21 Right touchpad 22 Left operation bar 23 Right operation bar 40 Gaze 41 Gaze position display area 81 Steering 84 Display device 410 Pointer 840 Display screen 841 to 844 First display area to fourth display area

Claims

1. An operating device comprising: an operating unit disposed in a vehicle and configured to accept a first operation; a gaze detection unit which detects the gaze of an occupant of the vehicle; and a control unit which, when the operating unit accepts the first operation and determines, among a plurality of display areas which display images, the existence of a gaze position display area in which the gaze of the occupant is located, enables only operation of the gaze position display area via the operating unit.

2. The operating device according to claim 1, wherein the control unit allows only operation via the operation unit with respect to the gaze position display area, and displays a pointer operable via the operation unit in the gaze position display area.

3. The operating device according to claim 1, wherein the operating unit has a left operating unit and a right operating unit arranged on the left and right sides of the steering wheel of the vehicle, and the control unit determines that an operation performed on the left operating unit and the right operating unit, which has at least a partial overlapping period, is the first operation.

4. The operating device of claim 1, wherein the operating unit further accepts a second operation, and the control unit, after enabling only operations on the gaze position display area, when the second operation is performed on the operating unit and then a third operation is performed, moves the gaze position display area to change the layout of the multiple display areas.

5. The operating device according to claim 4, wherein the operating unit comprises a left operating unit and a right operating unit arranged on the left and right sides of the steering wheel of the vehicle, and a specific operating unit provided adjacent to at least one of the left operating unit and the right operating unit and which accepts the second operation.

6. The operating device according to claim 4, wherein the control unit determines that the third operation is an operation performed on one of the left operating unit and the right operating unit, or an operation including a selection operation of a destination display area by changing the occupant's line of sight and a cancellation of the second operation.

7. The operating device according to claim 6, wherein the control unit keeps at least one of the plurality of display areas, which displays the vehicle speed, in front of the occupant, and restricts changes to the layout of the display area and switching of the display position of the display area.

8. The operating device according to claim 7, wherein the control unit has operation pattern information including information on an operation pattern that enables operation of a desired display area, information on an operation pattern for changing the layout of the gaze position display area, and information on an operation pattern for swapping the multiple display areas.

9. The operating device according to claim 5, wherein the left operation unit and the right operation unit are capacitive touch pads that accept touch operations, and the specific operation unit is a push switch or a capacitive touch sensor.

10. The operating device according to claim 1, wherein the gaze detection unit is connected to a camera positioned so that the face of the occupant is within its imaging range, and detects the gaze of the occupant based on image information output from the camera.

Images