User interface systems, control devices, and computer programs

Abstract

To improve the operability of a user interface system that acquires information related to multiple controlled operations through a display device. [Solution] The control device 13 causes the display device 11 to display a first image associated with a first controlled action, and a second image, which is smaller than the first image and associated with a second controlled action. When the control device 13 detects that a part of the user's body is in a predetermined position, it changes the display mode of the second image. When the second image is selected, the control device 13 causes the display device 11 to display a third image on the display device 11, which is related to the second controlled action and provides more information than the second image.

Description

Technical Field

[0001] The present disclosure relates to a user interface system that acquires information related to a plurality of controlled operations through a display device. The present disclosure also relates to a control device that may be included in the user interface system and a computer program executable by a processor mounted on the control device.

Background Art

[0002] Patent Document 1 discloses a user interface system for controlling the operation of a controlled device mounted on a vehicle as an example of a moving body. The system includes a display device and a touch pad. On the screen of the display device, images such as icons and buttons indicating the functions of the controlled device are displayed. The user touches the detection part of the touch pad with a finger to input an operation, thereby selecting the image and executing the function associated with the image.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] There is a need to improve the operability of a user interface system that acquires information related to a plurality of controlled operations through a display device.

Means for Solving the Problems

[0005] One exemplary aspect provided by the present disclosure is a user interface system, a display device, a control device that causes the display device to display a first image associated with a first controlled operation and a second image associated with a second controlled operation and smaller than the first image. It is equipped with, The control device is When it is detected that a part of the user's body is in a predetermined position, the display mode of the second image is changed. When the second image is selected, a third image, which provides more information than the second image in relation to the second controlled operation, is displayed on the display device.

[0006] One embodiment provided herein is a control device, A processor that displays a first image associated with a first controlled action, and a second image associated with a second controlled action, which is smaller than the first image, on a display device, An interface that receives a first signal indicating that a part of the user's body is in a predetermined position, and a second signal indicating that the second image has been selected. It is equipped with, The aforementioned processor, Based on the first signal, the display mode of the second image is changed. Based on the second signal, a third image is displayed on the display device, which provides more information than the second image in relation to the second controlled operation.

[0007] One example of an embodiment provided herein is a computer program executable by a processor mounted on a control device, By being executed, the control device will A first image associated with the first controlled action, and a second image, smaller than the first image and associated with the second controlled action, are displayed on the display device. Based on a first signal indicating that a part of the user's body is in a predetermined position, the display mode of the second image is changed. Based on a second signal indicating that the second image has been selected, the display device is made to display a third image that provides more information than the second image in relation to the second controlled operation.

[0008] As the number of controlled actions controlled through a display device increases, the area of ​​the display device capable of providing information related to those controlled actions is limited. Therefore, it becomes necessary to assign priority and hierarchy to the information provided. In the configurations described in each of the above examples, the second image is displayed smaller than the first image on the display device, thereby increasing the priority of the information related to the first controlled action. In other words, the ease of obtaining information related to the second controlled action is relatively reduced.

[0009] In this situation, when a part of the user's body is detected to be in a predetermined position, the display mode of the second image changes, allowing the user to easily recognize that it is possible to acquire information related to the second controlled action. If the user wishes to continue acquiring this information, they can select the second image to display more detailed information about the second controlled action on the display device. If the user does not wish to acquire this information, they simply do not need to perform this operation. Therefore, the operability of a user interface system that acquires information on multiple controlled actions through a display device can be improved. [Brief explanation of the drawing]

[0010] [Figure 1] This shows a user interface system according to one embodiment. [Figure 2] Figure 1 illustrates a steering system on which the controlled object is mounted. [Figure 3] Figure 1 shows an example of the screen displayed on the display device. [Figure 4] Another example of the screen displayed on the display device shown in Figure 1 is shown. [Figure 5] Another example of the screen displayed on the display device shown in Figure 1 is shown. [Figure 6] Another example of the screen displayed on the display device shown in Figure 1 is shown. [Modes for carrying out the invention]

[0011] While referring to the attached drawings, embodiments will be described in detail below. In each of the drawings used in the following description, the scale is appropriately changed in order to make each illustrated element recognizable in size.

[0012] FIG. 1 illustrates the functional configuration of a user interface system 10 according to an embodiment. The user interface system 10 is configured to be mounted on a vehicle. The vehicle is an example of a moving body.

[0013] The user interface system 10 includes a display device 11 and an operated body 12. The operated body 12 is disposed at a distance from the display device 11. In this example, as illustrated in FIG. 2, the operated body 12 is mounted on a steering device 20 that causes the vehicle to perform an operation (steering) to change the traveling direction. The display device 11 can be disposed, for example, in front of or beside the steering device 20.

[0014] The operated body 12 is configured to receive an operation for causing a controlled device 30 mounted on the vehicle to perform an operation different from steering. The operation is performed by the fingers of a driver who is gripping the steering device 20. The fingers of the driver are an example of a part of the user's body.

[0015] Examples of the controlled device 30 include audio-visual equipment, air conditioning equipment, lighting devices (headlights, direction indicators, fog lights, interior lights, etc.), visibility improvement devices (wipers, cleaners, etc.), devices for adjusting the position of a steering member or seat, power window devices, locking and unlocking devices, and the like.

[0016] The operated body 12 has a detection surface. The operated body 12 is configured to detect contact or approach of the driver's fingers to the detection surface.

[0017] As illustrated in FIG. 1, the operated body 12 can be configured to detect a change in capacitance on the detection surface accompanying contact or approach of the driver's fingers, and output a detection signal DT indicating the position (operated position) where the driver's fingers have contacted or approached based on the change.

[0018] Alternatively, the operated object 12 may be configured to detect pressure changes or displacements on a detection surface caused by contact with the driver's fingers, and to output a detection signal DT indicating the operated position based on such changes or displacements.

[0019] The detection signal DT may be an analog signal or a digital signal, depending on the specifications of the operated object 12.

[0020] The user interface system 10 includes a control device 13. The control device 13 is installed in an appropriate location in the vehicle.

[0021] The control device 13 is equipped with an input interface 131. The input interface 131 is configured as a hardware interface capable of receiving a detection signal DT. If the detection signal DT is an analog signal, the input interface 131 is equipped with an appropriate conversion circuit including an A / D converter that converts the analog signal into a digital signal.

[0022] The control device 13 includes a processor 132 and an output interface 133. The processor 132 is configured to output a display control signal DC from the output interface 133 that changes the screen displayed on the display device 11 based on the operated position indicated by the detection signal DT received by the input interface 131.

[0023] The output interface 133 is configured as a hardware interface capable of outputting a display control signal DC. The display control signal DC may be an analog signal or a digital signal, depending on the specifications of the display device 11.

[0024] If the display control signal DC is an analog signal, the output interface 133 includes an appropriate conversion circuit, including a D / A converter, which converts the digital signal output from the processor 132 into an analog signal. This description also applies to other signals that the output interface 133 can output, as described later.

[0025] Figure 3 shows an example of a screen 40 displayed on the display device 11. The screen 40 includes a first area 41. Images 411, 412, and 413 are displayed in the first area 41. Each of images 411, 412, and 413 is associated with a different operation (controlled operation) of the controlled device 30. The different controlled operations may be different operations relating to the same controlled device, or they may be operations of different controlled devices. The image displayed in the first area 41 is an example of a first image. The controlled operation associated with the image is an example of a first controlled operation.

[0026] Depending on the position of the driver's fingers detected by the operated object 12, one of images 411, 412, or 413 may be selected. In the illustrated example, image 411 is selected. For example, if approach or contact of fingers is detected at the position corresponding to image 411 on the detection surface of the operated object 12, and a predetermined operation (such as double-tapping or holding) is detected, image 411 is selected.

[0027] When image 411 is selected, image 51, which provides more information about the controlled operation associated with image 411, is displayed in the first area 41, as illustrated in Figure 4. Specifically, when the control device 13 receives a detection signal DT corresponding to the operation input that selects image 411, the processor 132 outputs a display control signal DC from the output interface 133 to display image 51 on the display device 11.

[0028] When an operation to control the operation of a specific controlled device is performed through the controlled device 12 based on the information provided by image 51, a detection signal DT corresponding to that operation is received by the input interface 131. As illustrated in Figure 1, the processor 132 outputs an operation control signal OC that controls the operation of the controlled device 30 from the output interface 133.

[0029] Similarly, when image 412 or image 413 is selected, an image providing more information about the controlled operation associated with image 412 or image 413 is displayed in the first region 41.

[0030] As illustrated in Figure 3, screen 40 includes a second area 42. Images 421, 422, and 423 are displayed in the second area 42. Each of images 421, 422, and 423 is associated with a different controlled operation of the controlled device 30. The different controlled operations may be different operations relating to the same controlled device, or they may be operations of different controlled devices. The images displayed in the second area 42 are examples of second images. The controlled operations associated with these images are examples of second controlled operations.

[0031] Each of the images 421, 422, and 423 displayed in the second region 42 is displayed smaller than each of the images 411, 412, and 413 displayed in the first region 41.

[0032] The relationship between the controlled actions associated with the image displayed in the first area 41 and the controlled actions associated with the image displayed in the second area 42 is determined as appropriate. For example, controlled actions that are used more frequently may be associated with the image displayed in the first area 41. Alternatively, controlled actions that require more information for the user to recognize the action may be associated with the image displayed in the first area 41.

[0033] When the input interface 131 of the control device 13 receives a detection signal DT indicating that the driver's fingers are in a position corresponding to the second region 42 on the detection surface of the operated object 12, the processor 132 outputs a display control signal DC from the output interface 133 that changes the display mode of the image displayed in the second region 42, as illustrated in Figure 5. The position corresponding to the second region 42 is an example of a predetermined position. The detection signal DT related to this process is an example of a first signal.

[0034] Specifically, each of the images 421, 422, and 423 displayed in the second area 42 is made larger. This indicates to the driver that they can select any of the images 421, 422, and 423.

[0035] Furthermore, if the driver can be informed that the image displayed in the second area 42 can be selected by changing the display mode, then in addition to or instead of changing the image size, processes such as changing the image color or making the image blink may also be employed.

[0036] Depending on the position of the driver's fingers detected by the operated device 12, one of images 421, 422, or 423 may be selected. For example, if approach or contact of fingers is detected at the position corresponding to image 421 on the detection surface of the operated device 12, and a predetermined operation (such as a double tap or prolonged rest) is detected, image 421 is selected.

[0037] When image 421 is selected, image 61, which provides more information about the controlled operation associated with image 421, is displayed in the first area 41, as illustrated in Figure 6. Specifically, when the control device 13 receives a detection signal DT corresponding to the operation input that selects image 421, the processor 132 outputs a display control signal DC from the output interface 133 to display image 61 on the display device 11. Image 61 is an example of a third image. The detection signal DT related to this process is an example of a second signal.

[0038] When an operation to control the operation of a specific controlled device is performed through the controlled device 12 based on the information provided by image 61, a detection signal DT corresponding to that operation is received by the input interface 131. The processor 132 outputs an operation control signal OC to control the operation of the controlled device 30 from the output interface 133.

[0039] Similarly, when image 422 or image 423 is selected, an image providing more information about the controlled action associated with image 422 or image 423 is displayed in the first region 41.

[0040] As the number of controlled operations controlled through the display device 11 increases, the area of ​​the display device 11 capable of providing information related to the controlled operations is limited. Therefore, it becomes necessary to assign priority and hierarchy to the information provided. In the configuration according to this embodiment, the image in the second region 42 of the screen 40 is displayed smaller than the image in the first region 41, thereby increasing the priority of the information related to the controlled operations associated with the image in the first region 41. In other words, the ease of obtaining information related to the controlled operations associated with the image in the second region 42 is relatively reduced.

[0041] In this situation, when it is detected that a part of the user's body is in a predetermined position, the display mode of the image in the second region 42 changes, so that the user can easily recognize that it is possible to acquire information related to the controlled action associated with that image. If the user wishes to continue acquiring this information, they can select the image to display more detailed information about the controlled action on the display device 11. If the user does not wish to acquire this information, they do not need to perform this operation. Therefore, the operability of the user interface system 10, which acquires information on multiple controlled actions through the display device 11, can be improved.

[0042] The processor 132 of the control device 13, which has the various functions described above, can be realized by at least one dedicated integrated circuit equipped with a memory element on which a computer program for realizing said functions is pre-installed. Examples of dedicated integrated circuits include microcontrollers, ASICs, FPGAs, etc. The memory element is an example of a non-temporary computer-readable medium on which the computer program is stored.

[0043] The processor 132 may be implemented by at least one general-purpose microprocessor operating in cooperation with at least one general-purpose memory. Examples of general-purpose microprocessors include CPUs, MPUs, and GPUs. Examples of general-purpose memory include ROMs and RAMs. In this case, the ROM may store computer programs for implementing the above functions. ROM is an example of a non-temporary computer-readable medium in which computer programs are stored. The general-purpose microprocessor selects at least a portion of the program stored in the ROM and loads it onto the RAM, and then works with the RAM to execute the above-described processes. The processor 132 may also be implemented by a combination of a dedicated integrated circuit and a general-purpose microprocessor.

[0044] The configurations referenced herein are merely examples to facilitate understanding of this disclosure. Each configuration example may be modified or combined with other configuration examples as appropriate within the scope of the intent of this disclosure.

[0045] In the above embodiment, an operator 12 located on the steering device 20 is used to input an operation to select an image displayed on the display device 11. With this configuration, the movement of the driver's fingers and gaze associated with operation input can be minimized.

[0046] However, the operated object 12 does not necessarily have to be located on the steering device 20. The operated object 12 can be located in a suitable location within the vehicle's passenger compartment. In this case, the operation input to the operated object 12 does not necessarily have to be made by the user's fingers. Depending on where the operated object 12 is located, the operation input may be made by other body parts.

[0047] In the above embodiment, an operable body 12 is used, which is positioned away from the display device 11, to input an operation to select an image displayed on the display device 11. In other words, a part of the user's body and the screen 40 displayed on the display device 11 are separated. In this case, the effect of making the user aware of the possibility of obtaining information about the controlled operation associated with the image through changes in the display mode of the image displayed in the second region 42 of the screen 40 can be made more pronounced.

[0048] However, the input for selecting an image displayed on the display device 11 may be made directly through a touch panel device provided on the screen of the display device 11.

[0049] In the above embodiment, the operated object 12 is used to input the operation to select an image after the display mode of the image displayed in the second area 42 of the screen 40 has been changed. However, other methods may be used to select an image after its display mode has been changed.

[0050] For example, as illustrated in Figure 1, the user interface system 10 may include other user interfaces 14. The other user interfaces 14 may be configured to accept voice input, gesture input, etc., and output instruction signals IS corresponding to the inputs. When an instruction signal IS corresponding to a predetermined input is received by the input interface 131 of the control device 13, the processor 132 outputs a display control signal DC from the output interface 133 that selects the image corresponding to the input in the second region 42. The instruction signal IS is an example of a second signal.

[0051] The user interface system 10 can be installed on mobile devices other than vehicles. Examples of other mobile devices include trains, aircraft, and ships. Such mobile devices do not necessarily require a driver.

[0052] The moving object does not necessarily have to be a physical entity. The user interface system 10 may be mounted on a simulator that simulates the movement of the moving object. In this case, the steering device 20 is used to change the direction of travel of the virtual moving object simulated by the simulator. The operated object 12 is used to input operations that control the movement of the virtual controlled object simulated by the simulator.

[0053] The user interface system 10 does not need to be mounted on a mobile device. The user interface system 10 can be used to control the operation of appropriate controlled devices installed in homes or facilities. [Explanation of Symbols]

[0054] 10: User interface system, 11: Display device, 12: Operated object, 13: Control device, 131: Input interface, 132: Processor, 20: Steering device, 411, 412, 413: Image, 421, 422, 423: Image, 61: Image, DT: Detection signal, IS: Instruction signal

Claims

1. Display device and A control device that displays a first image associated with a first controlled action, and a second image associated with a second controlled action, which is smaller than the first image, on the display device, It is equipped with, The control device is When it is detected that a part of the user's body is in a predetermined position, the display mode of the second image is changed. When the second image is selected, a third image, which provides more information than the second image in relation to the second controlled operation, is displayed on the display device. User interface system.

2. It includes an operable object that is positioned at a distance from the display device, The predetermined position is a part of the operated body. The user interface system according to claim 1.

3. The second image above shows the selected state achieved through operation input to the operated object. The user interface system according to claim 2.

4. The operated object is located in the steering device of the moving body. The user interface system according to claim 2 or 3.

5. A processor that displays a first image associated with a first controlled action, and a second image associated with a second controlled action, which is smaller than the first image, on a display device, An interface that receives a first signal indicating that a part of the user's body is in a predetermined position, and a second signal indicating that the second image has been selected. It is equipped with, The aforementioned processor, Based on the first signal, the display mode of the second image is changed. Based on the second signal, a third image is displayed on the display device, which provides more information than the second image in relation to the second controlled operation. Control device.

6. A computer program that can be executed by a processor mounted on a control device, By being executed, the control device will A first image associated with the first controlled action, and a second image, smaller than the first image and associated with the second controlled action, are displayed on the display device. Based on a first signal indicating that a part of the user's body is in a predetermined position, the display mode of the second image is changed. Based on a second signal indicating that the second image is in a selected state, a third image, which provides more information than the second image in relation to the second controlled operation, is displayed on the display device. Computer program.

Images