User interface system, control device and computer program
The user interface system addresses usability issues by transitioning display states based on non-contact operations and notifying users, enhancing the reliability of gesture input.
Patent Information
- Application Number
- JP2023080051
- Authority / Receiving Office
- JP · JP
- Patent Type
- Applications
- Current Assignee / Owner
- Filing Date
- 2023-05-15
- Publication Date
- 2024-11-27
AI Technical Summary
Existing user interface systems that allow gesture input face challenges in improving usability due to ambiguity in detecting non-contact operations, leading to user hesitation and potential failure in executing intended actions.
A user interface system equipped with a detection device and control device that transitions the display screen from one state to another based on non-contact operations, accompanied by a notification to the user that the operation has started, thereby enhancing user confidence and reducing hesitation.
The system improves usability by providing clear feedback on the initiation of non-contact operations, ensuring smooth execution of intended actions without interruption.
Smart Images

Figure 2024164515000001_ABST
Abstract
Description
[Technical field]
[0001] The present disclosure relates to a user interface system that accepts gesture input. The present disclosure also relates to a control device that may be included in the system, and to a computer program executable by a processor included in the control device. [Background technology]
[0002] Patent Document 1 discloses a user interface system that detects a non-contact action (gesture) associated with a predetermined controlled action based on the movement of a user's hand captured by a camera. When the predetermined gesture is detected, the device executes the controlled action associated with the gesture. [Prior art documents] [Patent documents]
[0003] [Patent Document 1] JP 2001-306243 A Summary of the Invention [Problem to be solved by the invention]
[0004] There is a demand for improving the convenience of user interface systems that allow gesture input. [Means for solving the problem]
[0005] One example aspect provided by the present disclosure is a user interface system, comprising: a detection device for detecting a non-contact motion made by a user to perform a controlled action; a control device that transitions a screen displayed on a display device from a first state corresponding to a state before the controlled action is executed to a second state corresponding to a state after the controlled action is executed based on the non-contact action; and Equipped with The control device causes a notification device to notify the user that detection of the non-contact motion has commenced.
[0006] One example aspect provided by the present disclosure is a control device, comprising: an interface for receiving a detection signal corresponding to a non-contact action performed by a user to perform a controlled action; a processor that transitions a screen displayed on a display device from a first state corresponding to a state before the controlled operation is executed to a second state corresponding to a state after the controlled operation is executed based on the detection signal; Equipped with The processor causes a notification device to notify the user that detection of the non-contact motion has commenced.
[0007] One example of an aspect provided by the present disclosure is a computer program executable by a processor installed in a control device, By being executed, the control device receiving a detection signal corresponding to a non-contact action performed by a user to perform a controlled action; transitioning a screen displayed on a display device from a first state corresponding to a state before the controlled operation is executed to a second state corresponding to a state after the controlled operation is executed based on the detection signal; A notification device is caused to notify the user that detection of the non-contact motion has commenced.
[0008] For example, when a user performs a predetermined contacted action to execute a specific controlled action, but the controlled action is not executed, it is difficult for the user to determine whether the cause is the detection device or the user's own action. However, according to the configurations according to the above-mentioned embodiments, the user is notified that the detection of the non-contact action has started, so that the user can continue the non-contact action without hesitation. Alternatively, by repeating the initial stage of the non-contact action so that the notification is given, the occurrence of a situation in which the non-contact action ends without being detected can be suppressed. Therefore, the convenience of the user interface system that allows gesture input can be improved. [Brief description of the drawings]
[0009] [Figure 1] 1 illustrates an example of a configuration of a user interface system according to an embodiment. [Diagram 2] 2 illustrates a vehicle in which the user interface system of FIG. 1 is installed. [Diagram 3] 2 shows an example of a screen displayed on the display device of FIG. 1. [Figure 4] 2 shows another example of the screen displayed on the display device of FIG. 1. [Diagram 5] 2 shows another example of the screen displayed on the display device of FIG. 1. [Figure 6] 2 shows another example of the screen displayed on the display device of FIG. 1. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0010] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS An embodiment will be described in detail below with reference to the accompanying drawings. In each of the drawings used in the following description, the scale is appropriately changed so that each element can be clearly seen.
[0011] Fig. 1 illustrates a functional configuration of a user interface system 10 according to an embodiment. The user interface system 10 according to this embodiment is configured to be mounted on a vehicle 20 illustrated in Fig. 2. The user interface system 10 is used to control the operation of a controlled device mounted on the vehicle 20 and to obtain information related to the driving environment of the vehicle 20. Examples of the controlled device mounted on the vehicle 20 include audiovisual equipment, air conditioning equipment, lighting equipment, and seat devices. The vehicle 20 is an example of a moving body.
[0012] A display device 22 is installed in a passenger compartment 21 of the vehicle 20. The display device 22 is disposed in a center cluster 23 of the vehicle 20.
[0013] Display device 22 is configured to display a portal screen P as exemplified in Fig. 3. Portal screen P includes a first tile image T1, a second tile image T2, a third tile image T3, and a fourth tile image T4. First tile image T1 is associated with a function of acquiring map information and traffic information. Second tile image T2 is associated with a music playback function. Third tile image T3 is associated with a function of acquiring weather information. Fourth tile image T4 is associated with a function of acquiring remaining battery power and mileage information.
[0014] 1, the display device 22 is configured to include a touch panel device 221. The touch panel device 221 is configured to output a detection signal DT corresponding to a change in capacitance caused by a finger of an occupant of the vehicle 20 touching or approaching a detection surface. The detection signal DT may be an analog signal or a digital signal depending on the specifications of the touch panel device 221.
[0015] The user interface system 10 includes a controller 12. The controller 12 includes an input interface 121, a processor 122, and an output interface 123.
[0016] The input interface 121 is configured as a hardware interface that receives the detection signal DT. When the detection signal DT is an analog signal, the input interface 121 includes an appropriate conversion circuit including an A / D converter.
[0017] The processor 122 is configured to output a display control signal DC, which changes a screen displayed on the display device 22 in response to a touch operation corresponding to the detection signal DT, from an output interface 123 configured as a hardware interface. The display control signal DC may be an analog signal or a digital signal depending on the specifications of the display device 22. When the display control signal DC is an analog signal, the output interface 123 includes an appropriate conversion circuit including a D / A converter. This description is similarly applied to other signals that can be output from the output interface 123, which will be described later.
[0018] In the example of FIG. 3, the touch panel device 221 is configured to receive touch operations on each of a first tile image T1, a second tile image T2, a third tile image T3, and a fourth tile image T4.
[0019] For example, when a touch operation is performed on second tile image T2, a detection signal DT corresponding to the position where the touch operation was performed is output from touch panel device 221. When detection signal DT is received by input interface 121 of control device 12, processor 122 outputs, from output interface 123, a display control signal DC that causes display device 22 to make a change to enlarge second tile image T2, as illustrated in FIG.
[0020] The enlarged second tile image T2 provides more detailed information about the music playback function and can receive operation inputs related to more detailed function selection. An occupant of vehicle 20 can cause the audio device to perform a desired operation by performing an appropriate touch operation on a button image or thumbnail image displayed in the enlarged second tile image T2. This description is similarly applied to other tile images.
[0021] As a result of second tile image T2 being enlarged, the other tile images that were not initially selected disappear from the screen of display device 22, leaving only third tile image T3. Third tile image T3 is displayed in a grayed-out state to indicate that it is not selected.
[0022] The processor 122 of the control device 12 is configured to detect a non-contact action (gesture) made by an occupant of the vehicle 20 based on a detection signal DT corresponding to a change in capacitance of the detection surface of the touch panel device 221 accompanying the approach and movement of the hand of the occupant. In this example, the touch panel device 221 is an example of a detection device.
[0023] 5 and 6, when a sweeping motion is performed with the occupant's hand approaching the screen of display device 22, the enlarged display of second tile image T2 is canceled, and the state returns to the state where portal screen P is displayed. The sweeping motion is an example of a non-contact motion. Returning to the portal screen P is an example of a controlled motion.
[0024] That is, when a detection signal DT corresponding to a sweeping action is received by input interface 121, processor 122 outputs a display control signal DC from output interface 123, which transitions the screen displayed on display device 22 from the state illustrated in Fig. 4 to the state illustrated in Fig. 6. The screen state illustrated in Fig. 4 is an example of a first state corresponding to the state before the controlled action is executed. The screen state illustrated in Fig. 6 is an example of a second state corresponding to the state after the controlled action is executed.
[0025] More specifically, when the occupant brings his / her hand close to the screen of display device 22 prior to the sweeping motion, processor 122 determines that detection of the sweeping motion has started based on detection signal DT corresponding to that state. In this case, processor 122 outputs, from output interface 123, a display control signal DC that causes display device 22 to cancel the grayed-out state of third tile image T3.
[0026] As a result, the grayed-out state of third tile image T3 is released, as illustrated in Fig. 4 and Fig. 5. This allows the occupant to recognize that detection of a waving motion has started. In other words, processor 122 causes display device 22 to notify the occupant that detection of a waving motion has started. In this example, display device 22 is an example of a notification device. The screen state illustrated in Fig. 5 is an example of a third state indicating that detection of a non-contact motion has started.
[0027] For example, if an occupant performs a sweeping motion to return to the portal screen P but the return does not occur, it is difficult for the occupant to determine whether the cause is the detection device or his / her own motion. However, according to the configuration of this embodiment, the occupant is notified that detection of the sweeping motion has started, so the occupant can continue the sweeping motion without hesitation. Alternatively, by repeating the initial stage of the sweeping motion so that a notification is given, it is possible to prevent the occurrence of a situation in which the sweeping motion ends without being detected. Therefore, the convenience of the user interface system 10 that allows gesture input can be improved.
[0028] The processor 122 of the control device 12 having the various functions described above can be realized by a dedicated integrated circuit such as a microcontroller, ASIC, FPGA, etc., including a storage element in which a computer program for implementing the functions is preinstalled. In this case, the storage element is an example of a non-transitory computer-readable medium in which a computer program is stored.
[0029] Alternatively, the processor 122 may be realized by a general-purpose microprocessor that operates in cooperation with a general-purpose memory. Examples of the general-purpose microprocessor include a CPU, an MPU, and a GPU. Examples of the general-purpose memory include a ROM and a RAM. In this case, a computer program for realizing the function may be stored in the ROM. The general-purpose microprocessor specifies at least a part of the program stored in the ROM, expands it on the RAM, and executes the above-mentioned process in cooperation with the RAM. In this case, the general-purpose memory is an example of a non-transitory computer-readable medium in which a computer program is stored.
[0030] The processor 122 may be implemented by a combination of a general purpose microprocessor and a dedicated integrated circuit.
[0031] The configurations described above are merely examples for facilitating understanding of the present disclosure. Each configuration example may be appropriately modified and combined with other configuration examples without departing from the spirit of the present disclosure.
[0032] In the above embodiment, the start of detection of the sweeping motion is notified by canceling the grayed-out state of the third tile image T3. However, as long as the state is different from the state before and after the sweeping motion is performed, an appropriate display mode can be selected. For example, the enlarged second tile image T2 may be changed to a grayed-out state, or an appropriate mark, an image resembling a hand performing a sweeping motion, or the like may be additionally displayed.
[0033] In the above embodiment, the display device 22 functions as a notification device. By sharing the function with the display device 22, an increase in the number of devices constituting the user interface system 10 can be suppressed.
[0034] However, in addition to or instead of the display device 22, as illustrated in Fig. 1, the user interface system 10 may include an independent notification device 13. The notification device 13 may be installed at an appropriate position in the passenger compartment 21 of the vehicle 20 so as to notify the occupant that a brushing operation has been started through at least one of a visual notification, an audible notification, and a tactile notification. In this case, the processor 122 of the control device 12 outputs a control signal CT from the output interface 123 based on the detection signal DT received by the input interface 121, for causing the notification device 13 to perform a predetermined notification operation.
[0035] In the above embodiment, the touch panel device 221 functions as a detection device. By sharing the function with the touch panel device 221 as an input device, an increase in the number of devices constituting the user interface system 10 can be suppressed.
[0036] However, in addition to or instead of the touch panel device 221, as exemplified in FIG. 1, the user interface system 10 may include an independent detection device 14. For example, the detection device 14 may be realized by an imaging device installed at an appropriate position in the cabin 21 of the vehicle 20. The detection device 14 is configured to output a detection signal DT corresponding to an image in which the occupant's hand movement is captured. In this case, the processor 122 of the control device 12 detects that a sweeping action has been started and that a sweeping action has been performed based on the detection signal DT received by the input interface 121. The detection device 14 may detect the occupant's hand movement using infrared rays or radio waves.
[0037] In the above embodiment, the touch panel device 221 is disposed in the center cluster 23 of the vehicle 20. That is, the touch panel device 221 is disposed between the driver's seat and the passenger seat in the left-right direction of the vehicle 20. With such a configuration, non-contact actions by not only the driver but also the passenger seat can be detected, so that the convenience of the user interface system 10 that allows gesture input can be further improved. From this perspective, it is preferable that the above-mentioned detection device 14 is disposed in a position where it can detect non-contact actions by the passenger seat.
[0038] The user interface system 10 may be installed in a moving object other than the vehicle 20. Examples of such moving objects include trains, airplanes, and ships. The moving object may not require a driver. The moving object may have only one seat for passengers to sit on.
[0039] The user interface system 10 does not necessarily need to be mounted on a mobile object. The user interface system 10 can be configured as a system that allows input of a predetermined controlled action so that an appropriate device installed in a house or facility executes a desired controlled action.
[0040] Each of the configurations listed below also constitutes part of this disclosure. Item 1: a detection device for detecting a non-contact motion made by a user to perform a controlled action; a control device that transitions a screen displayed on a display device from a first state corresponding to a state before the controlled action is executed to a second state corresponding to a state after the controlled action is executed based on the non-contact action; and Equipped with The control device causes a notification device to notify the user that detection of the non-contact operation has started. User interface system. Item 2: the control device causes the display device to function as the notification device by transitioning the screen to a third state, which indicates that detection of the non-contact motion has started. Item 1. A user interface system according to item 1. Item 3: the detection device includes a touch panel device that detects an operation on the display device based on a change in capacitance, The non-contact operation is detected based on the change in capacitance. 3. A user interface system according to item 1 or 2. Item 4: Configured to be mounted on a moving object, 4. A user interface system according to any one of items 1 to 3. Item 5: The moving body is a vehicle equipped with a driver's seat and a passenger seat, The detection device is disposed between a driver's seat and a passenger seat in the left-right direction of the vehicle. 5. The user interface system according to item 4. [Explanation of symbols]
[0041] 10: User interface system, 12: Control device, 121: Input interface, 122: Processor, 13: Notification device, 14: Detection device, 20: Vehicle, 22: Display device, 23: Center cluster, 221: Touch panel device
Claims
1. a detection device for detecting a non-contact motion made by a user to perform a controlled action; a control device that transitions a screen displayed on a display device from a first state corresponding to a state before the controlled action is executed to a second state corresponding to a state after the controlled action is executed based on the non-contact action; and Equipped with The control device causes a notification device to notify the user that detection of the non-contact operation has started. User interface system.
2. the control device causes the display device to function as the notification device by transitioning the screen to a third state, which indicates that detection of the non-contact motion has started. The user interface system of claim 1 .
3. the detection device includes a touch panel device that detects an operation on the display device based on a change in capacitance, The non-contact operation is detected based on the change in capacitance. The user interface system of claim 1 .
4. Configured to be mounted on a moving object, The user interface system of claim 1 .
5. The moving body is a vehicle equipped with a driver's seat and a passenger seat, The detection device is disposed between a driver's seat and a passenger seat in the left-right direction of the vehicle. The user interface system of claim 4.
6. an interface for receiving a detection signal corresponding to a non-contact action performed by a user to perform a controlled action; a processor that transitions a screen displayed on a display device from a first state corresponding to a state before the controlled operation is executed to a second state corresponding to a state after the controlled operation is executed based on the detection signal; and Equipped with and causing a notification device to notify the user that detection of the non-contact motion has commenced. Control device.
7. A computer program executable by a processor mounted on the control device, By being executed, the control device receiving a detection signal corresponding to a non-contact action performed by a user to perform a controlled action; transitioning a screen displayed on a display device from a first state corresponding to a state before the controlled operation is executed to a second state corresponding to a state after the controlled operation is executed based on the detection signal; causing a notification device to notify the user that detection of the non-contact motion has started; Computer program.
Citation Information
Patent Citations
Information processor
JP2016153250A
Virtual image display apparatus, control method, program, and storage medium
JP2021014262A
Finger strip and use of said finger strip
US20170368938A1
Vehicle sensor system
US20220413596A1
Detection device and gesture input device
WO2015133057A1
