User interface system

Abstract

To suppress a decrease in operational subjective feeling in a system in which operational input to a GUI screen displayed on a display device is performed by using a remote-control device.SOLUTION: A remote-control device 11 includes an operated surface 111. A detection device 12 detects a positional relationship between a finger 21 of a user 20 at a predetermined position P and the operated surface 111. A display device 13 three-dimensionally displays a GUI screen 131 associated with a controlled function and a pointer image for performing operational input to the GUI screen 131 so as to correspond to the positional relationship. The display device 13 is disposed so that a center position 131a of the GUI screen 131 is positioned to one of the left and the right of a sagittal plane 22 of the user 20 facing forward at the predetermined position P. The remote-control device 11 is disposed so that a center position 111a of the operated surface 111 is positioned to the other of the left and the right of the sagittal plane 22.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present disclosure relates to a system in which an operation input is made to a GUI (Graphical User Interface) screen displayed on a display device using a remote control device. [Background technology]

[0002] Patent Document 1 discloses a system that realizes operation input to a GUI screen displayed on a display device without using a remote control device such as a remote controller. A virtual surface space is set between the user and the display device, and the position of the user's fingers moving in the virtual surface space is detected by an imaging device. A pointer corresponding to the position is displayed on the screen of the display device, and operation input to the GUI screen can be performed. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] International Publication No. 2018 / 142524 Summary of the Invention [Problem to be solved by the invention]

[0004] 2. Description of the Related Art In a system in which a remote control device is used to input operations onto a GUI screen displayed on a display device, it is required to suppress a decrease in the sense of ownership of the operation. [Means for solving the problem]

[0005] One example aspect provided by the present disclosure is a user interface system, comprising: A remote control device having an operated surface; A detection device that detects a positional relationship between a user's finger located at a predetermined position and the operation surface; a display device that three-dimensionally displays a GUI screen associated with the controlled function and a pointer image for performing an operation input on the GUI screen in accordance with the positional relationship; It is equipped with the display device is disposed such that a center position of the GUI screen is located to the left or right of a sagittal plane of the user facing forward at the predetermined position, The remote control device is disposed so that the center position of the operated surface is located on the other of the left and right sides of the sagittal plane.

[0006] According to the above user interface system, a user can input operations to a GUI screen via a pointer image displayed on a display device by inputting operations with his / her fingers on the operated surface of a remote control device. For example, compared to inputting operations by directly touching a GUI screen equipped with a touch panel function with his / her fingers, the amount of movement of the arm or hand can be reduced.

[0007] On the other hand, since the pointer image that performs operation input on the GUI screen and the fingers on the operated surface are separated, it is inevitable that the feeling that the operation is being input to the GUI screen with one's own fingers (sense of ownership of the operation) will decrease. The inventors of the present application have found that by arranging the remote control device and the display device as described above, it is possible to suppress the occurrence of erroneous operations while suppressing the decrease in the sense of ownership of the operation. [Brief description of the drawings]

[0008] [Figure 1] 1 illustrates an example of a configuration of a user interface system according to an embodiment. [Diagram 2] 2 illustrates an example of a GUI screen and a pointer image displayed on the display device of FIG. 1. [Diagram 3] 2 illustrates an example of a functional configuration of the display control device of FIG. 1; [Figure 4] 1 illustrates a configuration of a user interface system according to a first comparative example. [Diagram 5] The experimental results show the advantages over the comparative example of FIG. [Figure 6] 1 illustrates a configuration of a user interface system according to a second comparative example. [Figure 7] The experimental results show the advantages over the comparative example of FIG. [Figure 8] 2 illustrates a vehicle in which the user interface system of FIG. 1 is installed. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] An embodiment will be described in detail below with reference to the accompanying drawings. Fig. 1 illustrates a configuration of a user interface system 10 according to an embodiment. In each figure, the scale is appropriately changed so that each illustrated element can be recognized.

[0010] The user interface system 10 includes a remote control device 11. The remote control device 11 has an operated surface 111. The operated surface 111 is configured to receive an operation by a finger 21 of a user 20 positioned at a predetermined position P.

[0011] The user interface system 10 includes a detection device 12. The detection device 12 is configured to detect the positional relationship between a finger 21 and an operation surface 111.

[0012] As an example, the detection device 12 can be configured to detect a change in capacitance of the operated surface 111 accompanying the approach or contact of the finger 21. In this case, the detection device 12 can detect the position where the finger 21 approaches or contacts the operated surface 111 and the distance between the operated surface 111 and the finger 21 through a well-known signal processing technique. That is, the three-dimensional positional relationship between the finger 21 and the operated surface 111 can be detected.

[0013] As another example, the detection device 12 can be configured to capture an image of the fingers 21 approaching or touching the operated surface 111. In this case, the detection device 12 can detect the position where the fingers 21 approach or touch the operated surface 111 and the distance between the operated surface 111 and the fingers 21 through a well-known image processing technique. That is, the three-dimensional positional relationship between the fingers 21 and the operated surface 111 can be detected.

[0014] As another example, the detection device 12 may be configured to detect the finger 21 approaching or touching the operated surface 111 using infrared rays or radio waves. In this case, the detection device 12 may detect the position where the finger 21 approaches or touches the operated surface 111 and the distance between the operated surface 111 and the finger 21 through a well-known signal processing technique. That is, the three-dimensional positional relationship between the finger 21 and the operated surface 111 may be detected.

[0015] 3, the detection device 12 is configured to output a detection signal DT corresponding to the detection result. The detection signal DT may be an analog signal or a digital signal depending on the specifications of the detection device 12.

[0016] As illustrated in Fig. 1, the user interface system 10 includes a display device 13. As illustrated in Fig. 2, the display device 13 is configured to display a GUI screen 131. The GUI screen 131 includes a plurality of button images and icon images, each of which is associated with a predetermined controlled function.

[0017] The display device 13 is configured to display a pointer image 132. The pointer image 132 is used to perform an operation input to the GUI screen 131. The pointer image 132 is displayed three-dimensionally so as to reflect the positional relationship between the finger 21 detected by the detection device 12 and the operated surface 111 within the GUI screen 131. The design of the pointer image 132 can be changed as appropriate.

[0018] 1, the user interface system 10 includes a display control device 14. The display control device 14 is configured to cause the display device 13 to display a pointer image 132 reflecting the positional relationship between the finger 21 detected by the detection device 12 and the operated surface 111.

[0019] Specifically, as illustrated in FIG. 3, the display control device 14 includes an input interface 141, a processor 142, and an output interface 143.

[0020] The input interface 141 is configured as a hardware interface that receives the detection signal DT output from the detection device 12. When the detection signal DT is an analog signal, the input interface 141 includes an appropriate conversion circuit including an A / D converter.

[0021] The processor 142 is configured to output, from an output interface 143 configured as a hardware interface, a display control signal DC for displaying a pointer image 132 on the GUI screen 131 in accordance with the positional relationship between the finger 21 and the operated surface 111 indicated by the detection signal DT. The display control signal DC may be an analog signal or a digital signal depending on the specifications of the display device 13. When the display control signal DC is an analog signal, the output interface 143 includes an appropriate conversion circuit including a D / A converter.

[0022] The processor 142 having such functions may be realized by a dedicated integrated circuit such as a microcontroller, ASIC, or FPGA having a storage element in which a computer program for implementing the functions is pre-installed.

[0023] Alternatively, the processor 142 may be realized by a general-purpose microprocessor that operates in cooperation with a general-purpose memory. Examples of the general-purpose microprocessor include a CPU, an MPU, and a GPU. Examples of the general-purpose memory include a ROM and a RAM. In this case, a computer program for realizing the function may be stored in the ROM. The general-purpose microprocessor specifies at least a part of the program stored in the ROM, expands it on the RAM, and works with the RAM to execute the above-mentioned process. The processor 142 may be realized by a combination of a general-purpose microprocessor and a dedicated integrated circuit.

[0024] According to the user interface system 10 configured as described above, the user 20 can input operations to button images and icon images in the GUI screen 131 via the pointer image 132 displayed on the display device 13 by inputting operations with the fingers 21 on the operated surface 111 of the remote control device 11. For example, compared to inputting operations by directly touching a GUI screen equipped with a touch panel function with the fingers, the amount of movement of the arm and hand can be reduced.

[0025] On the other hand, since the pointer image 132 that inputs operations to the GUI screen 131 and the fingers 21 on the operated surface 111 are separated, the feeling that operations are being input to the GUI screen 131 with one's own fingers 21 (sense of ownership of the operation) is inevitably reduced.

[0026] As illustrated in Fig. 1, the display device 13 is disposed so that the center position 131a of the GUI screen 131 is located to the left of the sagittal plane 22 of the user 20 facing forward at a predetermined position P. On the other hand, the remote control device 11 is disposed so that the center position 111a of the operated surface 111 is located to the right of the sagittal plane 22 when viewed from above. Examples of the reference position 22a include the top of the head and the tip of the nose. The inventors of the present application have found that by disposing the remote control device 11 and the display device 13 in this way, it is possible to suppress a decrease in the sense of ownership of the operation.

[0027] 4 shows the arrangement of the remote control device 11 and the display device 13 according to a first comparative example. In this example, the display device 13 is arranged in front of the user 20. Specifically, the display device 13 is arranged so that the center position 131a of the GUI screen 131 is located on the sagittal plane 22 of the user 20 when viewed from above. The arrangement of the remote control device 11 is the same as the configuration exemplified in FIG.

[0028] A number of subjects were instructed to perform specific operational inputs on the GUI screen 131, and the subjects executed the operational inputs by remotely controlling the pointer image 132 using the remote control device 11. After that, each subject was asked to evaluate the feeling of operation of the pointer image 132. The evaluation was performed by selecting one of seven values ​​ranging from "not at all" to "very much" for the question "Did you feel that you had control over the pointer?". A higher evaluation value corresponds to a higher sense of control.

[0029] The results are shown in Fig. 5. Bar graph B51 represents the average of the evaluation values ​​obtained from multiple subjects when the display device 13 is placed in front of the user 20. Bar graph B52 represents the average of the evaluation values ​​obtained from multiple subjects when the display device 13 is placed to the left of the sagittal plane 22.

[0030] The configuration according to the embodiment in which the display device 13 is disposed to the left of the sagittal plane 22 is evaluated as having a higher sense of ownership of the operation than the configuration according to the comparative example in which the display device 13 is disposed in front of the user 20 (a significant tendency). In other words, it is found that in the user interface system 10 in which operation input is performed on the GUI screen 131 displayed on the display device 13 using the remote control device 11, a decrease in the sense of ownership of the operation can be suppressed.

[0031] As illustrated in FIG. 1, the remote control device 11 is disposed at a predetermined position P such that a central position 111a of an operated surface 111 is located to the right of a sagittal plane 22 of a user 20 facing forward.

[0032] 6 shows the arrangement of the remote control device 11 and the display device 13 according to the second comparative example. In this example, the remote control device 11 is arranged in front of the user 20. Specifically, the remote control device 11 is arranged so that the central position 111a of the operated surface 111 is located on the sagittal plane 22 of the user 20 when viewed from above. The arrangement of the display device 13 is the same as the configuration illustrated in FIG.

[0033] A plurality of subjects were instructed to perform a specific operation input on the GUI screen 131, and were made to execute the operation input through remote operation of the pointer image 132 by the remote control device 11. After that, the number of operational errors was tallied.

[0034] The results are shown in Fig. 7. Bar graph B71 shows the average number of erroneous operations obtained from a plurality of subjects when the operated surface 111 is placed in front of the user 20. Bar graph B72 shows the average number of erroneous operations obtained from a plurality of subjects when the operated surface 111 is placed to the right of the sagittal plane 22.

[0035] These results show that the occurrence of erroneous operations is more effectively reduced when the operated surface 111 is positioned to the right of the sagittal plane 22 than when the operated surface 111 is positioned directly in front of the user 20 (there is a significant difference).

[0036] That is, according to the configuration of this embodiment, in a user interface system 10 in which operation input is performed on a GUI screen 131 displayed on a display device 13 using a remote control device 11, it is possible to suppress a decrease in the sense of ownership of the operation while also suppressing the occurrence of erroneous operations.

[0037] 8 illustrates a vehicle 30 equipped with the user interface system 10. In this example, an operated surface 111 of the remote control device 11 is disposed on a palm rest 32 provided on a door 31 for opening and closing the vehicle interior. Although not shown, the display device 13 is disposed on a center cluster located to the left of the steering wheel 33. In this case, the controlled functions of various devices mounted on the vehicle 30 are controlled through operation input to a GUI screen 131 of the display device 13 using the remote control device 11. Examples of such devices include audiovisual devices, air conditioning devices, lighting devices, seat devices, etc.

[0038] In this example, the predetermined position P illustrated in Fig. 1 is the driver's seat of the vehicle 30. That is, the operated surface 111 is installed closer to the door 31 than the driver's seat. The vehicle 30 is an example of a moving body. The passenger compartment is an example of a room. The door 31 is an example of an opening / closing body. The driver's seat is an example of a seat.

[0039] According to this configuration, when the driver as the user 20 performs operation input to the GUI screen 131 of the display device 13 arranged in the center cluster using the remote control device 11 arranged in the palm rest 32, it is possible to suppress a decrease in the sense of ownership of the operation. In addition, it is possible to suppress erroneous operation while reducing the amount of movement of the hands and arms accompanying the operation input.

[0040] Each of the configurations referred to in the above description is merely an example for facilitating understanding of the present disclosure. Each configuration example may be appropriately modified or combined with other configuration examples within the scope of the spirit of the present disclosure.

[0041] In the above embodiment, the display device 13 is disposed to the left of the sagittal plane 22 of the user 20 facing forward at the predetermined position P. However, the display device 13 may be disposed to the right of the sagittal plane 22 depending on circumstances related to the handedness of the user 20, laws and regulations, and the like.

[0042] In the above embodiment, the display device 13 displays the pointer image 132 three-dimensionally within the two-dimensionally displayed GUI screen 131. However, if the pointer image 132 appears three-dimensionally relative to the GUI screen 131 from the user 20 at a predetermined position P, the display device 13 may be configured as a device that projects at least the pointer image 132.

[0043] The user interface system 10 may be installed in a moving body other than the vehicle 30. Examples of other moving bodies include trains, airplanes, and ships. The controlled functions controlled through the remote control device 11 are appropriately determined according to the specifications of the moving body. The moving body may not require a driver. The moving body may have only one seat for passengers to sit in.

[0044] The user interface system 10 does not necessarily need to be mounted on a mobile object. As long as it is possible to arrange the remote control device 11 and the display device 13 associated with a specific controlled function so as to satisfy the above-mentioned positional relationship, they can be installed in an appropriate house or facility. [Explanation of symbols]

[0045] 10: user interface system, 11: remote control device, 111: operated surface, 111a: central position, 12: detection device, 13: display device, 131: GUI screen, 131a: central position, 131b: normal, 132: pointer image, 20: user, 21: finger, 22: sagittal plane, 22a: reference position, 30: vehicle, 31: door, P: predetermined position

Claims

1. A remote control device having an operated surface; A detection device that detects a positional relationship between a user's finger located at a predetermined position and the operation surface; a display device that three-dimensionally displays a GUI screen associated with the controlled function and a pointer image for performing an operation input on the GUI screen in accordance with the positional relationship; Equipped with the display device is disposed such that a center position of the GUI screen is located to the left or right of a sagittal plane of the user facing forward at the predetermined position; The remote control device is disposed so that a center position of the operated surface is located on the other of the left and right sides of the sagittal plane. User interface system.

2. the predetermined position is a seat in a moving body, The operated surface is installed closer to an opening / closing body that opens and closes a room of the moving body than the seat. The user interface system of claim 1 .

Images