User interface system, control device, and non-transitory computer-readable medium
The user interface system improves convenience by displaying operation settings as images on a shared display, enabling users to confirm and change settings without direct device interaction, addressing the limitations of existing systems in user interface systems for multiple types of controlled devices.
Patent Information
- Application Number
- PCT/JP2024/017998
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2023-05-16
- Filing Date
- 2024-05-15
- Publication Date
- 2024-11-21
AI Technical Summary
Existing user interface systems that display information for multiple types of controlled devices lack convenience in allowing users to confirm and change operation settings without needing to directly operate the devices or use dedicated notification devices.
A user interface system comprising a display device, an input device, and a control device with a processor that displays operation settings of a controlled device as an image on a general-purpose display when a specific operation is performed, allowing users to confirm and change settings without direct device interaction, using a non-transitory computer-readable medium to execute the necessary computer program.
Enhances user convenience by allowing operation settings to be confirmed and changed through images on a shared display without requiring dedicated devices, reducing unnecessary operations and providing a smooth transition between setting confirmation and control operations.
Smart Images

Figure JP2024017998_21112024_PF_FP_ABST
Abstract
Description
User interface system, controller, and non-transitory computer-readable medium
[0001] The present disclosure relates to a user interface system using a display device capable of displaying information related to multiple types of controlled devices. The present disclosure also relates to a control device that can be installed in the system, and a non-transitory computer-readable medium that stores a computer program executable by a processor installed in the control device.
[0002] Japanese Patent Application Publication No. 2016-126556 discloses a display device mounted on a vehicle as an example of a moving body. The display device displays information related to a plurality of controlled devices mounted on the vehicle. The display device includes a touch panel device capable of receiving touch operations from an occupant of the vehicle. The occupant can control the operation and change the settings of each controlled device through touch operations.
[0003] There is a demand for improving the convenience of user interface systems that use display devices that can display information related to a plurality of types of controlled devices.
[0004] One example aspect provided by the present disclosure is a user interface system comprising: an input device located away from a display device capable of displaying information related to multiple types of controlled devices, and receiving a first operation for controlling the operation of a first controlled device included in the multiple types of controlled devices; and a control device that, when the input device receives a second operation different from the first operation, causes the display device to display a first image indicating the operation settings of the first controlled device.
[0005] One example embodiment provided by the present disclosure is a control device, the control device being located away from a display device capable of displaying information related to multiple types of controlled devices, and comprising: an interface that receives, from an input device that receives a first operation for controlling the operation of a first controlled device included in the multiple types of controlled devices, a control signal based on a second operation different from the first operation; and a processor that causes the display device to display a first image showing the operation settings of the first controlled device based on the control signal.
[0006] One example aspect provided by the present disclosure is a non-transitory computer-readable medium storing a computer program executable by a processor of a control device, wherein execution of the computer program causes the control device to: receive an operation signal based on a second operation different from the first operation from an input device that is located away from a display device capable of displaying information related to multiple types of controlled devices and that accepts a first operation for controlling the operation of a first controlled device included in the multiple types of controlled devices; and display a first image showing the operation settings of the first controlled device on the display device based on the operation signal.
[0007] According to the configurations of the above-described embodiments, the operation settings of the first controlled device can be confirmed through the first image displayed on the display device by performing a second operation on an input device located away from the display device. Since the second operation is different from the first operation for controlling the operation of the first controlled device, it is not necessary to operate the first controlled device to confirm the operation settings. In addition, since the first image is displayed on a general-purpose display device that can also display information related to other controlled devices, there is no need to provide a dedicated device for notifying the operation settings. Therefore, the convenience of a user interface system that uses a display device that can display information related to multiple types of controlled devices can be improved.
[0008] 1 illustrates an example of the configuration of a user interface system according to an embodiment. FIG. 2 illustrates an example of a vehicle equipped with the user interface system of FIG. 1. FIG. 3 illustrates an example of a screen displayed on the display device of FIG. 1. FIG. 4 illustrates another example of a screen displayed on the display device of FIG. 1. FIG. 5 illustrates another example of a screen displayed on the display device of FIG. 1. FIG. 6 illustrates another example of a screen displayed on the display device of FIG. 1.
[0009] The following detailed description of exemplary embodiments will be given with reference to the accompanying drawings. In the drawings used in the following description, the scale has been changed appropriately so that each element can be clearly seen.
[0010] Fig. 1 illustrates a functional configuration of a user interface system 10 according to an embodiment. The user interface system 10 according to this embodiment is configured to be mounted on a vehicle 20 illustrated in Fig. 2. The vehicle 20 is an example of a moving object.
[0011] A display device 22 is installed in a passenger compartment 21 of the vehicle 20. The display device 22 is arranged in a center cluster 23 of the vehicle 20. The display device 22 is configured to be able to display information related to a plurality of types of controlled devices mounted on the vehicle 20.
[0012] As illustrated in FIG. 1 , the user interface system 10 includes an input device 11. The input device 11 includes a wiper switch 111. As illustrated in FIG. 2 , the wiper switch 111 is disposed away from the display device 22. The wiper switch 111 is in the form of a lever switch. The wiper switch 111 is configured to control the operation of wipers mounted on the vehicle 20 by being displaced in the vertical direction of the vehicle 20. The wipers are an example of a first controlled device. The operation of displacing the wiper switch 111 is an example of a first operation.
[0013] 1 , the input device 11 is configured to output a first operation signal S1 when the wiper switch 111 is displaced by the driver of the vehicle 20. The first operation signal S1 may be an analog signal or a digital signal depending on the specifications of the wiper switch 111.
[0014] The user interface system 10 includes a control device 12. The control device 12 includes an input interface 121, a processor 122, and an output interface 123.
[0015] The input interface 121 is configured as a hardware interface that accepts the first operation signal S1. If the first operation signal S1 is an analog signal, the input interface 121 includes an appropriate conversion circuit including an A / D converter. This description also applies to other signals that can be accepted by the input interface 121, which will be described later.
[0016] The processor 122 is configured to output a control signal CT, which causes a controlled device (here, the wiper 24) mounted on the vehicle 20 to perform an operation based on the first operation signal S1, from the output interface 123 configured as a hardware interface. The control signal CT may be an analog signal or a digital signal depending on the specifications of the controlled device.
[0017] If the control signal CT is an analog signal, the output interface 123 includes an appropriate conversion circuit including a D / A converter. This description also applies to other signals that can be output from the output interface 123, which will be described later.
[0018] The input device 11 according to this example includes a detection device that detects a change in capacitance caused by the driver's finger touching the wiper switch 111. The operation of touching the wiper switch 111 with a finger is an example of a second operation. The input device 11 is configured to output a second operation signal S2 corresponding to the change in capacitance caused by the finger touch. The second operation signal S2 may be an analog signal or a digital signal depending on the specifications of the detection device.
[0019] When the second operation signal S2 is received by the input interface 121, the processor 122 is configured to output a display control signal DC from the output interface 123 to cause the display device 22 to display the setting notification image SN illustrated in Figure 3.
[0020] The setting notification image SN is configured to notify the driver of the operation setting of the wiper 24. In this example, the image notifies the driver that the wiper 24 is set to automatic operation. The automatic operation is a setting in which the operation interval of the wiper 24 is automatically adjusted according to the vehicle speed of the vehicle 20, etc. The setting notification image is an example of a first image.
[0021] In this example, an image M providing information related to the music playback device is also displayed on the display device 22. In other words, the setting notification image SN shares the display device 22 with images providing information related to other controlled devices mounted on the vehicle 20. The music playback device is an example of a second controlled device.
[0022] With this configuration, by touching the wiper switch 111 located away from the display device 22, the operation setting of the wiper 24 can be confirmed through the setting notification image SN displayed on the display device 22. This operation is different from the operation for controlling the operation of the wiper 24, so there is no need to operate the wiper 24 to confirm the operation setting. In addition, since the setting notification image SN is displayed on the general-purpose display device 22 that can also display information related to other controlled devices, there is no need to provide a dedicated device for notifying the operation setting. Therefore, the convenience of the user interface system 10 that uses the display device 22 that can display information related to multiple types of controlled devices can be improved.
[0023] 1 , the display device 22 is configured to include a touch panel device 221. The touch panel device 221 is configured to output a detection signal DT corresponding to a change in capacitance caused by a finger of an occupant of the vehicle 20 touching or approaching a detection surface. The detection signal DT may be an analog signal or a digital signal depending on the specifications of the touch panel device 221.
[0024] The processor 122 is configured to output, from the output interface 123, a display control signal DC that changes the screen displayed on the display device 22 in response to the touch operation corresponding to the detection signal DT.
[0025] 3, the setting notification image SN is selected by a touch operation on the setting notification image SN. The operation of selecting the setting notification image SN is an example of a third operation. Based on a detection signal DT indicating that the setting notification image SN has been selected, the processor 122 outputs a display control signal DC from the output interface 123 to cause the display device 22 to display a wiper setting screen WS, as shown in FIG. 4.
[0026] The wiper setting screen WS includes a plurality of button images for accepting an operation to change the operation setting of the wipers 24. As described with reference to FIG. 3, in this example, the wipers 24 are set to operate automatically, so the button image that enables automatic operation is selected. For example, if the occupant desires to set the wipers 24 to operate continuously at a low speed, the button image marked "LO" is pressed, as shown in FIG. 4. The wiper setting screen WS is an example of a first setting screen.
[0027] Note that the operation for selecting a specific image displayed on the display device 22 is not limited to a touch operation on the touch panel device 221. The detection signal DT may be output through an appropriate input device installed in the vehicle interior 21 independently of the display device 22. Examples of such an input device include a mechanical button switch, a lever switch, and a touchpad device.
[0028] According to the configuration of this embodiment, the current operation settings of the wiper 24 can be confirmed by touching the wiper switch 111 before displaying the wiper setting screen WS, which allows the user to change the operation settings of the wiper 24. The wiper setting screen WS only needs to be displayed when it is determined that a setting change is necessary, thereby reducing unnecessary operations. This improves the convenience of the user interface system 10.
[0029] The operation of touching the wiper switch 111 is a preliminary operation for the operation of displacing the wiper switch 111. In this case, a smooth transition can be achieved from the operation for checking the operation setting of the wiper 24 to the operation for controlling the operation of the wiper 24. It is also easy to cancel the displacing operation when it is determined that the operation control of the wiper 24 is unnecessary as a result of checking the operation setting.
[0030] The processor 122 of the control device 12 can output, from the output interface 123, a display control signal DC that causes the display device 22 to display the portal icon image P exemplified in FIG.
[0031] In this case, when the input interface 121 receives a detection signal DT corresponding to the portal icon image P being selected, the processor 122 outputs a display control signal DC, which causes the display device 22 to display the general-purpose setting screen GS exemplified in Fig. 6, from the output interface 123. The portal icon image P is an example of a second image. The operation of selecting the portal icon image P is an example of a fourth operation.
[0032] The general setting screen GS is configured to allow operation settings of other controlled devices in addition to the wiper 24. Fig. 6 shows an example of a screen for setting the operation of the locking / unlocking device of the vehicle 20. The general setting screen GS includes button images for selecting the controlled device for which operation settings are to be made. Fig. 6 shows an example of a button image for selecting the wiper 24 and a button image for selecting the center console position adjustment device. The general setting screen GS is an example of a second setting screen. The locking / unlocking device and the position adjustment device are each an example of a second controlled device.
[0033] When an operation to select the wiper 24 as the target of operation setting is performed on the general setting screen GS, a setting screen similar to the example shown in FIG. 4 is displayed, and the operation setting can be changed.
[0034] When the driver touches the wiper switch 111 while the portal icon image P is displayed, the second operation signal S2 is received by the input interface 121, and the processor 122 outputs a display control signal DC from the output interface 123 to change the portal icon image P to a setting notification image SN.
[0035] When the driver removes his / her finger from the wiper switch 111, the output of the second operation signal S2 from the input device 11 is stopped. When a predetermined time has elapsed since the input of the second operation signal S2 to the input interface 121 was stopped, the processor 122 outputs, from the output interface 123, a display control signal DC that changes the setting notification image SN back to the portal icon image P.
[0036] That is, there are two ways to reach a state where the operation of the wiper 24 can be set. One is a method of selecting the portal icon image P displayed on the display device 22 as described above, transitioning to the general setting screen GS, and then selecting the wiper 24 as the target for operation setting. The other is a method of touching the wiper switch 111 to display the setting notification image SN on the display device 22, and then setting the setting notification image SN to a selected state, as described with reference to FIG. 3.
[0037] The latter allows the user to more quickly reach a state where the operation of the wiper 24 can be set. In other words, the setting notification image SN provides a shortcut route for setting the operation of the wiper 24.
[0038] 3 and 5, the setting notification image SN has a larger area than the portal icon image P. This increases the amount of information related to the operation settings provided for notification, while emphasizing that notification of the operation settings is being made.
[0039] The processor 122 of the control device 12 having the various functions described above can be realized by a dedicated integrated circuit such as a microcontroller, ASIC, FPGA, or the like, which includes a storage element in which a computer program for realizing the functions is pre-installed. In this case, the storage element is an example of a non-transitory computer-readable medium on which a computer program is stored.
[0040] Alternatively, the processor 122 may be realized by a general-purpose microprocessor operating in cooperation with general-purpose memory. Examples of the general-purpose microprocessor include a CPU, an MPU, and a GPU. Examples of the general-purpose memory include a ROM and a RAM. In this case, a computer program for realizing the function may be stored in the ROM. The general-purpose microprocessor specifies at least a portion of the program stored in the ROM, expands it on the RAM, and executes the above-described processing in cooperation with the RAM. In this case, the general-purpose memory is an example of a non-transitory computer-readable medium on which a computer program is stored.
[0041] Processor 122 may be implemented by a combination of a general-purpose microprocessor and a dedicated integrated circuit.
[0042] The configurations described above are merely examples to facilitate understanding of the present disclosure. Each configuration example can be appropriately modified and combined with other configuration examples without departing from the spirit of the present disclosure.
[0043] In the above embodiment, the setting notification image SN is displayed on the display device 22 so as to avoid the area where information related to other controlled devices is displayed. With this configuration, the setting notification image SN, which is temporarily displayed based on the operation of touching the wiper switch 111, can be prevented from impairing the visibility of information related to other controlled devices displayed on the general-purpose display device 22. However, in consideration of the temporary nature of the display, the display position of the setting notification image SN on the screen of the display device 22 can be determined appropriately. For example, the setting notification image SN may be displayed in the central area of the display device 22 so as to overlap with information related to other controlled devices.
[0044] The controlled device for which the operation setting is notified by the setting notification image SN is not limited to the wiper 24. For example, a position adjustment device of the center console can be an example of the first controlled device.
[0045] 2 , the input device 11 may further include a push switch 112. The push switch 112 may be installed on the steering wheel 25 of the vehicle 20.
[0046] The push switch 112 can be configured to be displaceable to indicate an upward, downward, leftward, or rightward direction. As illustrated in Fig. 1 , the input device 11 is configured to output a third operation signal S3 when the push switch 112 is displaced by the driver of the vehicle 20. The third operation signal S3 may be an analog signal or a digital signal depending on the specifications of the push switch 112. The operation of displacing the push switch 112 is an example of a first operation.
[0047] The processor 122 of the control device 12 outputs, from the output interface 123, a control signal CT that causes the center console position adjustment device to perform an operation based on the third operation signal S3 received by the input interface 121. Specifically, when an operation that causes a displacement indicating an upward, downward, leftward, or rightward direction is performed on the push switch 112, the control signal CT that causes the position adjustment device to displace the center console upward, downward, forward, or backward, respectively, is output from the output interface 123.
[0048] The input device 11 may also include a detection device that detects a change in capacitance caused by the driver's finger touching the push switch 112. The operation of touching the push switch 112 with a finger is an example of a second operation. The input device 11 is configured to output a fourth operation signal S4 corresponding to the change in capacitance caused by the finger touch. The fourth operation signal S4 may be an analog signal or a digital signal depending on the specifications of the detection device.
[0049] When the fourth operation signal S4 is received by the input interface 121, the processor 122 outputs, from the output interface 123, a display control signal DC that causes the display device 22 to display a setting notification image SN exemplified in FIG.
[0050] In this example, the setting notification image SN notifies that the center console position is set to "Personal Setting 1." When an occupant of the vehicle 20 performs an operation to select the setting notification image SN, the processor 122 outputs, from the output interface 123, a display control signal DC that causes the display device 22 to display a console setting screen CS, as shown in FIG.
[0051] The console setting screen CS according to this example includes a plurality of button images for accepting an operation to change the position of the center console, a button image for enabling the automatic position adjustment function, and a button image for registering personal settings. The console setting screen CS is an example of a first setting screen.
[0052] In the above embodiment, the operation of displaying the setting notification image SN on the display device 22 is detected based on a change in capacitance caused by the driver's finger touching the switch. However, the operation may also be detected based on a change in pressure or temperature caused by the driver's finger touching the switch. Alternatively, the touch of the driver's finger on the switch may be detected by an imaging device.
[0053] The operation of displaying the setting notification image SN on the display device 22 is not limited to the operation of touching a switch. The operation of displaying the setting notification image SN on the display device 22 can be appropriately determined depending on the type of switch employed, as long as it is different from the operation for controlling the operation of the controlled device associated with the switch. For example, in a configuration in which the operation of the controlled device is controlled by displacing the switch in a specific direction, the setting notification image SN can be displayed by displacing the switch in the opposite direction.
[0054] In the above embodiment, the display device 22 is disposed in the center cluster 23 of the vehicle 20. That is, the setting notification image SN is displayed between the driver's seat and the passenger seat in the left-right direction of the vehicle 20. With this configuration, not only the driver but also the passenger sitting in the passenger seat can change the operational settings of the controlled device by selecting the setting notification image SN, thereby further improving the convenience of the user interface system 10.
[0055] The user interface system 10 may be installed in a moving body other than the vehicle 20. Examples of such moving bodies include trains, airplanes, and ships. The moving body may not require a driver. The moving body may have only one seat for the passengers.
[0056] The user interface system 10 does not necessarily need to be mounted on a mobile object. The user interface system 10 can be configured as a system that allows operation control and operation setting changes of appropriate controlled devices installed in a home or facility.
[0057] The contents of Japanese Patent Application No. 2023-080789 filed on May 16, 2023 are incorporated by reference as part of this disclosure.
Claims
1. A user interface system comprising: an input device positioned away from a display device capable of displaying information related to multiple types of controlled devices, and accepting a first operation for controlling the operation of a first controlled device included in the multiple types of controlled devices; and a control device that, when the input device accepts a second operation different from the first operation, causes the display device to display a first image indicating the operation settings of the first controlled device.
2. The user interface system according to claim 1, wherein the control device causes the display device to display a first setting screen for changing the operation settings based on a third operation of selecting the first image.
3. The user interface system of claim 2, wherein the control device changes the second image displayed on the display device to the first image based on the second operation, and, based on a fourth operation of selecting the second image, causes the display device to display a second setting screen which allows the settings of a second controlled device included in the multiple types of controlled devices to be changed in addition to the first controlled device.
4. The user interface system of claim 3, wherein the first image has a larger area than the second image.
5. A user interface system as described in any one of claims 1 to 4, wherein the control device displays the first image on the display device so as to avoid an area in which information related to a second controlled device included in the multiple types of controlled devices is displayed.
6. The user interface system according to any one of claims 1 to 5, wherein the second operation is a preliminary operation of the first operation.
7. A user interface system according to any one of claims 1 to 6, configured to be mounted on a moving object.
8. The user interface system according to claim 7, wherein the moving body is a vehicle equipped with a driver's seat and a passenger seat, and the first image is displayed between the driver's seat and the passenger seat in the left-right direction of the vehicle.
9. A control device comprising: an interface that is positioned away from a display device capable of displaying information related to multiple types of controlled devices, and that receives an operation signal based on a second operation different from the first operation from an input device that receives a first operation for controlling the operation of a first controlled device included in the multiple types of controlled devices; and a processor that displays a first image indicating the operation settings of the first controlled device on the display device based on the operation signal.
10. A non-transitory computer-readable medium storing a computer program executable by a processor of a control device, wherein, when the computer program is executed, the control device: receives an operation signal based on a second operation different from the first operation from an input device that is positioned away from a display device capable of displaying information related to multiple types of controlled devices and that accepts a first operation for controlling the operation of a first controlled device included in the multiple types of controlled devices; and displays a first image indicating the operation setting of the first controlled device on the display device based on the operation signal.
Citation Information
Patent Citations
Operating device
JP2009078729A
Operation device
JP2014067253A
Automotive touch-sensitive operating devices and automobiles
JP2015508356A
Steering Wheel Mounted Control Using Touch Switch
US20100101921A1
