Interface device, interface method, and program

Abstract

The present invention provides an interface device that can suppress the creation of feelings of discomfort or distrust in users when performing interactive response processing for users while they are on board a mobile vehicle. [Solution] The interface device 1 includes a user state recognition unit 12 that recognizes the state of user U riding on the mobile body 100, a response output control unit 14 that controls the output mode of the notification output by first output devices 55, 57 used in the mobile body 100 according to the state of user U recognized by the user state recognition unit 12, and an action recognition notification unit 15 that performs action recognition notification processing, in which, when a predetermined action of user U is recognized by the user state recognition unit 12, an action recognition notification with an output mode changed in conjunction with the predetermined action is output by second output devices 55, 56, 58 used in the mobile body 100.

Description

Technical Field

[0001] The present invention relates to an interface device, an interface method, and a program.

Background Art

[0002] Conventionally, a technique is known in which the line-of-sight direction of a passenger riding in a vehicle is detected, and the sound image localization position of the sound output inside the vehicle is adjusted so as to be in the line-of-sight direction (see, for example, Patent Document 1). Further, a technique is known in which the direction of the driver's face is detected, an audio-visual direction for turning the driver's face toward the front of the vehicle is determined, and sound corresponding to the frequency of the direction determination band corresponding to the determined audio-visual direction is output to an audio output unit (see, for example, Patent Document 2).

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Patent Document 2

Summary of the Invention

Problems to be Solved by the Invention

[0004] By the way, when providing an interactive HMI (Human Machine Interface) to a user riding in a moving body such as a vehicle, as in the above background art, it is conceivable to recognize the user's state such as the line of sight or the direction of the face, and set the mode of the response voice according to the user's state. However, if the user does not recognize that the HMI is responding due to such a function, the user cannot know based on what information the HMI is responding, and thus there is a risk of giving a sense of discomfort or distrust to the HMI. Therefore, the objective of this invention is to provide an interface device that can suppress the creation of feelings of discomfort or distrust in users when performing interactive response processing for users while they are on board a moving vehicle. Ultimately, this will contribute to further improving traffic safety and the development of a sustainable transportation system. [Means for solving the problem]

[0005] As a first embodiment for achieving the above objective, an interface device is provided which includes: a user state recognition unit that recognizes the state of a user riding in a mobile vehicle; a response output control unit that controls the output mode of a notification output by a first output device used in the mobile vehicle according to the state of the user recognized by the user state recognition unit; and an action recognition notification unit that performs action recognition notification processing, in which, when a predetermined action of the user is recognized by the user state recognition unit, an action recognition notification with an output mode changed in conjunction with the predetermined action is output by a second output device used in the mobile vehicle.

[0006] In the above interface device, the operation recognition notification unit may be configured to change the output mode of the operation recognition notification by the second output device according to the degree to which the user state recognition unit recognizes the predetermined operation.

[0007] In the above-described interface device, the second output device is a display device, and the operation recognition notification unit may be configured to output a blinking of the display device as an operation recognition notification when the user state recognition unit recognizes the user blinking as a predetermined operation.

[0008] In the above interface device, the second output device is a seat vibration device that vibrates multiple locations on the seat in which the user sits, and the motion recognition notification unit may be configured to vibrate the locations corresponding to the direction of the user's body displacement as motion recognition notification when the user state recognition unit recognizes the displacement of the user's body as a predetermined motion.

[0009] The above interface device may be configured to include a boarding recognition unit that recognizes when the user has boarded the mobile vehicle, and the motion recognition notification unit executes the motion recognition notification process when the boarding recognition unit recognizes that the user has boarded the mobile vehicle.

[0010] As a second embodiment for achieving the above objective, an interface method executed by a computer is provided, which includes: a user state recognition step for recognizing the state of a user riding in a mobile body; a response output control step for controlling the output mode of a notification to be output by a first output device used in the mobile body according to the state of the user recognized by the user state recognition step; and an action recognition notification step for executing an action recognition notification process in which, when a predetermined action of the user is recognized by the user state recognition step, an action recognition notification with an output mode changed in conjunction with the predetermined action is output by a second output device used in the mobile body.

[0011] A third embodiment for achieving the above objective is a program that causes the computer to function as: a user state recognition unit that recognizes the state of a user riding in a mobile vehicle; a response output control unit that controls the output mode of a notification output by a first output device used in the mobile vehicle according to the state of the user recognized by the user state recognition unit; and an action recognition notification unit that, when a predetermined action of the user is recognized by the user state recognition unit, outputs an action recognition notification with an output mode changed in conjunction with the predetermined action, using a second output device used in the mobile vehicle. [Effects of the Invention]

[0012] According to the above interface device, interface method, and program, it is possible to suppress causing discomfort or distrust in the user when performing interactive response processing. [Brief explanation of the drawing]

[0013] [Figure 1] Figure 1 is an explanatory diagram of how the interface device is used. [Figure 2] Figure 2 is a diagram showing the configuration of the interface device. [Figure 3] Figure 3 is a flowchart of the control process for the agent's output mode. [Modes for carrying out the invention]

[0014] [1. Usage of Interface Devices] Referring to Figure 1, the usage of the interface device 1 of this embodiment will be described. The interface device 1 provides a Human-Machine Interface (HMI) using an anthropomorphic agent with emotions to the user U, who is the driver of the vehicle 100. The interface device 1 may be configured as part of the functions of an in-vehicle device such as a navigation device mounted on the vehicle 100, or it may be configured as a dedicated in-vehicle device. The vehicle 100 corresponds to the mobile body in this disclosure.

[0015] Interface device 1 has a communication function and performs wireless communication between a wearable device 70 (such as a smartwatch) worn by user U and a mobile terminal 60 (such as a smartphone, tablet, or mobile phone) brought into the vehicle 100 by user U. Interface device 1 also communicates with external communication systems such as a content server 210 via the communication network 200.

[0016] Vehicle 100 is equipped with a driver monitor camera 54 for photographing user U, a touch panel display 55, a speaker 56, a microphone 57 for collecting user U's speech, a seat vibration device 58 mounted on the backrest of the driver's seat 5 (corresponding to the seat in this disclosure), a front camera 50 for photographing the area in front of vehicle 100, an ADAS (Advanced Driver-Assistance Systems) 2 that performs various driving assistance controls based on the image captured by the front camera 50, and a navigation device 3 that provides route guidance to the destination. The seat vibration device 58 is equipped with vibration units 58a and 58b arranged in parallel at multiple locations in the left-right direction (vehicle width direction of vehicle 100) on the backrest of the driver's seat 5. A HUD (Head-Up Display) that projects images onto the windshield of vehicle 100 may also be provided as an image display device.

[0017] The display 55 and the speaker 56 and other speakers provided in the vehicle 100 correspond to the first and second output devices of this disclosure. The seat vibration device 58 corresponds to the second output device of this disclosure.

[0018] The agent installed in interface device 1 is an interface agent configured, for example, by an AI (Artificial Intelligence) program. The agent is an interactive interface that analyzes the user U's speech input into microphone 57 using speech recognition and outputs information either by outputting audio from speaker 56 or displaying images on display 55, depending on the content of the speech. In response to the user U's speech, the agent provides output such as suggesting destinations for route guidance by navigation device 3, suggesting content to download and play from content server 210, and issuing warnings about predetermined risks (such as approaching obstacles) recognized by ADAS 2.

[0019] The interface device 1 recognizes the state of the user U based on the captured image of the user U by the driver monitoring camera 54, the voice of the user U input to the microphone 57, the biometric information (heart rate, body temperature, blood pressure, etc.) of the user U detected by the wearable device 70, and the like. Then, the interface device 1 performs control to set the output mode by the agent according to the state of the user U.

[0020] Also, although details will be described later, the interface device 1 executes an operation recognition notification process for notifying the user U that the operation of the user U is recognized by the interface device 1.

[0021] [2. Configuration of Interface Device] Referring to FIG. 2, the configuration of the interface device 1 will be described. Referring to FIG. 2, the interface device 1 is a control unit including a processor 10, a memory 20, a communication unit 30, and the like. The interface device 1 is connected to the ADAS 2, speed sensor 51, gyro sensor 52, GNSS (Global Navigation Satellite System) sensor 53, driver monitoring camera 54, display 55, speaker 56, microphone 57, and seat vibration device 58 mounted on the vehicle 100.

[0022] The communication unit 30 performs short-range wireless communication with the mobile terminal 60 and the wearable device 70 according to specifications such as Bluetooth (registered trademark) and UWB (Ultra Wide Band). Also, the communication unit 30 communicates with the content server 210 and the like via the communication network 200.

[0023] Memory 20 stores a program 21 for controlling the interface device 1, etc. The processor 10 reads and executes the program 21 and functions as a boarding recognition unit 11, a user status recognition unit 12, a risk recognition unit 13, an agent control unit 14, and an action recognition notification unit 15. The configuration in which the agent control unit 14 controls the output mode of notifications from the display 55 or speaker 56 corresponds to the response output control unit of this disclosure.

[0024] The processing performed by the user state recognition unit 12 corresponds to the user state recognition step in the interface method of this disclosure, and the control processing of the notification output mode performed by the agent control unit 14 corresponds to the response output control step in the interface method of this disclosure. The processing performed by the operation recognition notification unit 15 corresponds to the operation recognition notification step in the interface method of this disclosure.

[0025] The passenger recognition unit 11 recognizes that user U has boarded the vehicle 100 by turning on the power switch provided in the vehicle 100, recognizing user U from images captured by the driver monitor camera 54, etc. The user state recognition unit 12 recognizes the state of user U who is on board the vehicle 100 based on at least one of the following: information entered by touch operation on the display 55, an image of user U captured by the driver monitor camera 54, the user's voice entered into the microphone 57, or biometric information of user U detected by the wearable device 70.

[0026] The user state recognition unit 12 recognizes, for example, the following elements as the state of user U. Element 1: The user U's response to a question such as "How are you feeling today?" displayed on the display 55 when the user U boards the vehicle 100, which is entered via touch operation on the display 55. Second element: User U's facial expressions and behavior as recognized from User U's image. Third element: User U's speech content, intonation, pitch, volume, and intonation. Fourth element: User U's biometric information (heart rate, blood pressure, body temperature, etc.).

[0027] The risk recognition unit 13 recognizes a predetermined risk facing the vehicle 100 based on risk information input from the ADAS2. An example of a predetermined risk facing the vehicle 100 is approaching a monitored object (other vehicles, pedestrians, obstacles, etc.) in the vicinity of the vehicle 100. As the distance between the vehicle 100 and the monitored object decreases, the degree of risk recognized by the risk recognition unit 13 increases. Alternatively, the risk recognition unit 13 may recognize the degree of concentration of the user U on driving from the user U's facial image captured by the driver monitor camera 54 or the user U's biometric information detected by the wearable device 70, and recognize a state where the degree of concentration has fallen below a predetermined level as a predetermined risk. Alternatively, the risk level recognition unit 15 may calculate the risk level by using a vehicle-to-vehicle communication device, a GNSS unit, and V2X (vehicle-to-infrastructure or pedestrian-to-vehicle) communication to determine the distance to the monitored object in the vicinity of the vehicle 100.

[0028] The agent control unit 14 provides the user U with an HMI (Human-Machine Interface) provided by an anthropomorphic agent with emotions. The agent control unit 14 analyzes the user U's speech input to the microphone 57 and responds to the speech by outputting the agent's voice from the speaker 56 or displaying an image of the agent on the display 55. Furthermore, when a predetermined risk is recognized by the risk recognition unit 13, the agent control unit 14 provides a warning notification by outputting a warning voice from the speaker 56 and displaying a warning image on the display 55. The agent control unit 14 controls the output manner of the agent's voice output from the speaker 56 (intonation, pitch, volume, etc.) and the display manner of the agent's image displayed on the display 55 (movement, display position, size, etc.) according to the state of the user U recognized by the user state recognition unit 12.

[0029] When the boarding recognition unit 11 recognizes that user U has boarded the vehicle 100, the motion recognition notification unit 15 performs motion recognition notification processing to notify user U that user U's actions have been recognized by the interface device 1. Based on the images captured by the driver monitor camera 54, the motion recognition notification unit 15 performs motion confirmation notification according to, for example, the following first to fourth notification examples.

[0030] First notification example…When the motion recognition notification unit 15 recognizes a change in the direction of the user U's gaze from the image captured by the driver monitor camera 54, it provides an motion confirmation notification by displaying an image on the display 55 (for example, the agent image 55a shown in Figure 1) in accordance with the direction of the user U's gaze. For example, when the user U's gaze is directed to the left, the motion recognition notification unit 15 displays image 55a on the left side of the display 55, and when the gaze is directed to the right, it displays image 55a on the right side of the display 55. The change in the direction of the user U's gaze corresponds to a predetermined user action in this disclosure, the degree of change in the direction of the gaze corresponds to the degree of recognition of the predetermined action in this disclosure, and the change in the position where image 55a is displayed corresponds to the output mode of the motion confirmation notification in this disclosure. If a HUD is provided, the position of the image projected onto the windshield by the HUD may be changed in accordance with the direction of the user U's gaze.

[0031] Second notification example: When the motion recognition notification unit 15 recognizes that user U has blinked from the image captured by the driver monitor camera, it flashes the screen display of the display 55 as an motion confirmation notification. In this case, depending on the speed of user U's blinking, the faster the blinking, the shorter the interval at which the screen display of the display 55 flashes. User U's blinking corresponds to a predetermined action of the user in this disclosure, the speed of the blinking corresponds to the degree of recognition of the predetermined action in this disclosure, and the flashing interval of the screen display of the display 55 corresponds to the output mode of the motion confirmation notification in this disclosure.

[0032] Third notification example…When the motion recognition notification unit 15 recognizes a change in the direction of user U's face (head direction) from the image captured by the driver monitor camera, it performs motion confirmation notification by outputting notification sound from multiple speakers (may include speaker 56) provided in the vehicle 100, adjusting the sound image position to match the direction of user U's face according to the direction of user U's face.For example, when user U turns their face to the left, the motion recognition notification unit 15 sets the sound image position to the left of user U and outputs notification sound, and when user U turns their face to the right, it sets the sound image position to the right of user U and outputs notification sound.In this case, the greater the change in the direction of user U's face, the louder the notification sound may be.The action of user U changing the direction of their face corresponds to the predetermined action of the user in this disclosure, the magnitude of the change in the direction of the face corresponds to the degree of recognition of the predetermined action in this disclosure, and the loudness of the notification sound corresponds to the output mode of the motion confirmation notification in this disclosure.

[0033] Fourth notification example…When the motion recognition notification unit 15 recognizes the displacement of user U's body (displacement of head, torso, arms, hands, etc.) from the image captured by the driver monitor camera, it makes a motion confirmation notification by vibrating the vibrating part 58a or 58b of the seat vibration device 58 that is closer to user U's center of gravity. For example, when user U tilts their head to the left, user U's center of gravity is displaced to the left, so the motion recognition notification unit 15 vibrates the left vibrating part 58a. Similarly, when user U tilts their head to the right, user U's center of gravity is displaced to the right, so the motion recognition notification unit 15 vibrates the right vibrating part 58b. In this case, depending on the magnitude of user U's body displacement, the vibration of the vibrating parts 58a and 58b may be increased as the displacement of user U's body increases. The displacement of user U's body corresponds to a predetermined action of the user in this disclosure, the magnitude of the body displacement corresponds to the degree of recognition of the predetermined action in this disclosure, and the magnitude of the vibration of the vibration units 58a and 58b corresponds to the output mode of the operation confirmation notification in this disclosure.

[0034] [3. Control processing for the agent's output mode] The procedure for controlling the output mode of the HMI agent, which is performed by the interface device 1, will be explained according to the flowchart shown in Figure 3.

[0035] In step S1 of Figure 3, when the boarding recognition unit 11 recognizes that user U has boarded the vehicle 100, it proceeds to step S2. In step S2, the motion recognition notification unit 15 executes the motion confirmation notification process described above. By executing the motion confirmation notification process at the time user U boards the vehicle 100 in this way, user U, who is about to start driving the vehicle 100, can be notified in advance that the user's status has been recognized by the interface device 1. As a result, user U can recognize that the notification output from the agent of the interface device 1 is based on the recognition result of user U's status, and this can prevent user U from feeling any discomfort or distrust towards the notification output from the agent.

[0036] In the following step S3, the user state recognition unit 12 recognizes the state of user U as described above. In the next step S4, the risk recognition unit 13 proceeds to step S20 if it recognizes a predetermined risk, and proceeds to step S5 if it does not recognize a predetermined risk. In step S20, the agent control unit 14 outputs a warning voice and a warning image corresponding to the predetermined risk recognized by the risk recognition unit 13 through the speaker 56 and display 55, respectively, in an output mode corresponding to the state of user U, and proceeds to step S6.

[0037] Furthermore, in step S5, the agent control unit 14 proceeds to step S30 if it recognizes the voice of user U input to microphone 57, and proceeds to step S6 if it does not recognize the voice of user U input to microphone 57. In step S30, the agent control unit 14 outputs a response voice to the input voice of user U in an output manner that corresponds to the state of user U. For example, if the input voice of user U requests the playback of cheerful music, the agent control unit 14 outputs the title of the music extracted as a playback candidate from speaker 56 in a bright and cheerful voice, and proceeds to step S6.

[0038] In step S6, the agent control unit 14 determines whether or not the vehicle 100 has been powered off. If the power has not been powered off, the agent control unit 14 proceeds to step S3 and repeats the processes in steps S3-S6, S20, and S30. On the other hand, if the power has been powered off, the agent control unit 14 proceeds to step S7 and terminates the control process for the agent's output mode.

[0039] [4. Other Embodiments] In the above embodiment, a vehicle 100 was shown as the mobile body of the disclosure, but the mobile body of the disclosure may be any mobile body for which an interface to a passenger is used, and may be an aircraft, a ship, or the like.

[0040] In the above embodiment, the interface device 1 of the present disclosure is exemplified by an interface device 1 configured by an in-vehicle device mounted on a vehicle 100. In other embodiments, part or all of the interface device of the present disclosure may be configured by a mobile terminal 60 brought into the vehicle 100, or by an external communication system such as a content server 210.

[0041] When the interface device is configured using the mobile terminal 60, the camera, microphone, speaker, and display provided on the mobile terminal 60 can be used to provide an HMI using an agent. Alternatively, biometric information of user U may be acquired through communication between the wearable device 70 and the mobile terminal 60, and the mobile terminal 60 may recognize the state of user U based on the biometric information. Furthermore, communication between the in-vehicle device of the vehicle 100 and the mobile terminal 60 may allow the mobile terminal 60 to control the output of the display 55, speakers 56 and other speakers provided in the vehicle 100, and the seat vibration device 58.

[0042] Furthermore, if the interface device is configured using an external communication system, for example, the vehicle 100 transmits images of user U captured by the driver monitor camera 54 of the vehicle 100 and audio data input to the microphone 57 from the vehicle 100 to the external communication system, which then recognizes the status of user U and executes operation confirmation notification processing.

[0043] In the above embodiment, the system includes a boarding recognition unit 11, and when the boarding recognition unit 11 recognizes that user U has boarded the vehicle 100, the motion recognition notification unit 15 executes motion confirmation notification processing. In other embodiments, the motion confirmation notification processing may be executed at other timings, for example, at predetermined intervals while user U is on board the vehicle 100. Furthermore, when executing motion recognition notification processing, voice output or screen display may be provided to prompt user U to perform predetermined actions such as "Please blink" or "Please move your head."

[0044] Figure 2 is a schematic diagram showing the configuration of the interface device 1 divided by its main processing content, in order to facilitate understanding of the present invention. The configuration of the interface device 1 may be divided into other categories. Furthermore, the processing of each component may be performed by one hardware unit or by multiple hardware units. Also, the processing of each component shown in Figure 3 may be performed by one program or by multiple programs.

[0045] [5. Configurations supported by the above embodiment] The above embodiment is a specific example of the following configuration.

[0046] (Configuration 1) An interface device comprising: a user state recognition unit that recognizes the state of a user riding in a mobile vehicle; a response output control unit that controls the output mode of a notification output by a first output device used in the mobile vehicle according to the state of the user recognized by the user state recognition unit; and an action recognition notification unit that performs action recognition notification processing, in which, when a predetermined action of the user is recognized by the user state recognition unit, an action recognition notification with an output mode changed in conjunction with the predetermined action is output by a second output device used in the mobile vehicle.

[0047] According to the interface device of Configuration 1, the interface device can inform users who are on board a mobile vehicle that their actions have been recognized by the interface device, through the output of an action confirmation notification from the second output device. This helps to prevent users from feeling uncomfortable or distrustful during interactive response processing.

[0048] (Configuration 2) The interface device according to Configuration 1, wherein the operation recognition notification unit changes the output mode of the operation recognition notification by the second output device according to the degree of recognition of the predetermined operation by the user state recognition unit.

[0049] According to the interface device of configuration 2, the output mode of the operation recognition notification from the second output device is changed in accordance with changes in the user's actions, so the user can easily confirm that they are being recognized by the interface device.

[0050] (Configuration 3) The interface device according to Configuration 1 or Configuration 2, wherein the second output device is a display device, and the operation recognition notification unit outputs a blinking of the display device as an operation recognition notification when the user status recognition unit recognizes the user blinking as a predetermined operation.

[0051] According to the interface device of configuration 3, the user can easily confirm that they are being recognized by the interface device by the flashing of a display device that is linked to their blinking.

[0052] (Configuration 4) The interface device according to any one of Configurations 1 to 3, wherein the second output device is a seat vibration device that vibrates multiple locations on the seat in which the user sits, and the motion recognition notification unit, when the user state recognition unit recognizes the displacement of the user's body as a predetermined action, notifies by vibrating the location corresponding to the direction of the user's body displacement as motion recognition notification.

[0053] According to the interface device of configuration 4, the user can easily confirm that they are being recognized by the interface device by the vibration of the seat at a point corresponding to the direction of their body's displacement.

[0054] (Configuration 5) An interface device according to any one of Configurations 1 to 4, comprising a boarding recognition unit that recognizes that the user has boarded the mobile vehicle, and the motion recognition notification unit that executes the motion recognition notification process when the boarding recognition unit recognizes that the user has boarded the mobile vehicle.

[0055] According to the interface device of configuration 5, when a user boards a mobile vehicle, the user can be notified that they are being recognized by the interface device.

[0056] (Configuration 6) An interface method executed by a computer, comprising: a user state recognition step of recognizing the state of a user riding in a mobile body; a response output control step of controlling the output mode of a notification to be output by a first output device used in the mobile body according to the state of the user recognized by the user state recognition step; and an action recognition notification step of executing an action recognition notification process in which, when a predetermined action of the user is recognized by the user state recognition step, an action recognition notification with an output mode changed in conjunction with the predetermined action is output by a second output device used in the mobile body.

[0057] By executing the interface method of Configuration 6 using a computer, the same effects and benefits as those of the interface device of Configuration 1 can be obtained.

[0058] (Configuration 7) A program that causes a computer to function as: a user state recognition unit that recognizes the state of a user riding in a mobile vehicle; a response output control unit that controls the output mode of a notification output by a first output device used in the mobile vehicle according to the state of the user recognized by the user state recognition unit; and an action recognition notification unit that, when a predetermined action of the user is recognized by the user state recognition unit, performs an action recognition notification process that outputs an action recognition notification with an output mode changed in conjunction with the predetermined action, using a second output device used in the mobile vehicle.

[0059] By executing the program of configuration 7 on a computer, the configuration of the interface device of configuration 1 can be realized. [Explanation of Symbols]

[0060] 1…Interface device, 2…ADAS, 10…Processor, 11…Boarding recognition unit, 12…User status recognition unit, 13…Risk recognition unit, 14…Agent control unit (response output control unit), 15…Motion recognition notification unit, 20…Memory, 21…Program, 30…Communication unit, 50…Front camera, 51…Speed ​​sensor, 52…Gyro sensor, 53…GNSS sensor, 54…Driver monitor camera, 55…Display, 56…Speaker, 57…Microphone, 58…Seat vibration device, 60…Mobile terminal, 70…Wearable device, 100…Vehicle (mobile object), 200…Communication network, 210…Content server, U…User.

Claims

1. A user status recognition unit that recognizes the status of the user riding in the mobile vehicle, A response output control unit controls the output mode of the notification output by the first output device used in the mobile body, according to the user's state recognized by the user state recognition unit, The system includes an action recognition notification unit that, when the user state recognition unit recognizes a predetermined action of the user, performs an action recognition notification process that outputs an action recognition notification, whose output mode changes in conjunction with the predetermined action, via a second output device used in the mobile unit. Interface device.

2. The operation recognition notification unit changes the output mode of the operation recognition notification by the second output device according to the degree to which the user state recognition unit recognizes the predetermined operation. The interface device according to claim 1.

3. The aforementioned second output device is a display device, The motion recognition notification unit outputs a blinking display device as a motion recognition notification when the user state recognition unit recognizes the user blinking as a predetermined action. The interface device according to claim 1 or claim 2.

4. The second output device is a seat vibration device that vibrates multiple locations on the seat in which the user sits, When the user state recognition unit recognizes a displacement of the user's body as a predetermined action, the motion recognition notification unit vibrates the part corresponding to the direction of the user's body displacement as a motion recognition notification. The interface device according to claim 1 or claim 2.

5. The vehicle includes a boarding recognition unit that recognizes that the user has boarded the vehicle. The motion recognition notification unit executes the motion recognition notification process when the boarding recognition unit recognizes that the user has boarded the mobile vehicle. The interface device according to any one of claims 1 to 4.

6. A computer-based interface method, A user status recognition step that recognizes the status of the user riding in the mobile vehicle, A response output control step controls the output mode of the notification output by the first output device used in the mobile body, according to the user's state recognized by the user state recognition step, The process includes an action recognition notification step in which, when a predetermined action of the user is recognized by the user state recognition step, an action recognition notification is output by a second output device used in the mobile body, which changes the output mode in conjunction with the predetermined action. Interface method.

7. Computers, A user status recognition unit that recognizes the status of the user riding in the mobile vehicle, A response output control unit controls the output mode of the notification output by the first output device used in the mobile body, according to the user's state recognized by the user state recognition unit, The operation recognition notification unit functions as an operation recognition notification unit that, when the user state recognition unit recognizes a predetermined action of the user, outputs an operation recognition notification, whose output mode changes in conjunction with the predetermined action, via a second output device used in the mobile unit. program.

Images