On-vehicle device

Abstract

To solve the problem that it is difficult that a conventional on-vehicle device voice-operating an operation object estimates an operation object.SOLUTION: An on-vehicle device includes a voice recognition device 1, a sight line recognition device 2, and a controller 4 for controlling a plurality of operation objects in a vehicle on the basis of signals from the devices 1 and 2, wherein the sight line recognition device 2 has a function of recognizing each sight line direction to at least a driver seat side region A and a front passenger seat side region B, and the controller 4 includes an object specification part 8 for specifying the operation object, a command generation part 9 for generating a command, and a transmission part 10 for transmitting the command to the specified operation object, and accurately operates the operation object by sight line movement for a short time and simple voice on the basis of the fact that the sight line of an occupant during driving always moves.SELECTED DRAWING: Figure 2

Description

Technical Field

[0001] The present invention relates to an in-vehicle device used to operate various objects mounted in a vehicle by recognizing the voice and line of sight of an occupant.

Background Art

[0002] Conventionally, as the in-vehicle device, for example, there is one described in Patent Document 1. Patent Document 1 discloses an in-vehicle device including voice recognition means for recognizing the voice of an occupant, line-of-sight direction recognition means for recognizing the line-of-sight direction of the occupant, command detection means for detecting a command from the voice recognized by the voice recognition means, and command processing means for processing the command according to the line-of-sight direction recognized by the line-of-sight direction recognition means.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] However, in the above conventional in-vehicle device, since the line of sight of the occupant during driving always moves, it may be difficult to estimate which object to be operated is being attempted to be operated by voice. Although voice operation using the line of sight can be made possible by continuously looking at the object to be operated for a certain period of time, it is not preferable as an operation during driving to focus on one place.

[0005] The present invention has been made in view of the above conventional situation, and an object of the present invention is to provide an in-vehicle device that can accurately operate an object to be operated with a short line-of-sight movement and simple voice, taking into account that the line of sight of the occupant during driving always moves.

Means for Solving the Problems

[0006] The in-vehicle device according to the present invention is an in-vehicle device that receives voice operations of vehicle occupants, and includes a voice recognition device that recognizes the voices of the occupants, a gaze recognition device that recognizes the gaze directions of the occupants, and a controller for controlling a plurality of operation objects in the vehicle based on signals from the voice recognition device and the gaze recognition device. The gaze recognition device has a function of recognizing the respective gaze directions at least for the driver's seat side area and the passenger seat side area. And, the above in-vehicle device is characterized in that the controller includes an object identification unit that identifies an operation object from the voice recognized by the voice recognition device, a command generation unit that generates a command from the voice recognized by the voice recognition device, and a transmission unit that transmits the command to the identified operation object according to the gaze direction recognized by the gaze recognition device.

Effects of the Invention

[0007] By adopting the above configuration, the in-vehicle device according to the present invention can accurately operate the operation object with a short-time gaze movement and a simple voice, taking into account that the gaze of the occupant during driving always moves.

Brief Description of the Drawings

[0008]

Figure 1

Figure 2

Figure 3

Figure 4

Modes for Carrying Out the Invention

[0009] <First Embodiment> The in-vehicle device accepts voice operations of the vehicle occupants. As shown in FIG. 1, generally, it includes a voice recognition device 1 that recognizes the voices of the occupants, a gaze recognition device 2 that recognizes the gaze direction of the occupants, and a main control device 3. The main control device 3 includes a controller 4 for controlling a plurality of operation objects in the vehicle based on signals from the voice recognition device 1 and the gaze recognition device 2.

[0010] The main control device 3 includes a voice data reception unit 5 that inputs a signal from the voice recognition device 1, a gaze data reception unit 6 that inputs a signal from the gaze recognition device 2, and a storage unit 7 for various data. The controller 4 includes an object identification unit 8 that identifies an operation object from the voice recognized by the voice recognition device 1, a command generation unit 9 that generates a command from the voice recognized by the voice recognition device 1, and a transmission unit 10 that transmits the command generated by the command generation unit 9 to the operation object identified by the object identification unit 8 according to the gaze direction recognized by the gaze recognition device 2.

[0011] The command from the controller 4 is transmitted to a notification unit 12 that notifies the content of the command by display or sound and an operation object group 11. The operation object group 11 typically consists of operation objects such as doors, windows, front and rear wipers, etc. The command is transmitted to the drive unit of each operation object. The operation object group 11 can also include an audio, an air conditioner, etc.

[0012] The voice recognition device 1 uses a microphone as an input source and recognizes a voice that combines the name of an operation object and its operation from the voice of the occupants collected. This voice recognition device 1 may be a single unit, or it may be a configuration that includes a part of the functions of the main control device 3.

[0013] The gaze recognition device 2 is a well-known device using an infrared camera or the like as an input source and recognizes the gaze direction from the movement of the occupants' eyes. The recognition of the gaze direction can be determined by the movement of only the eyeballs or the direction of the face. This gaze recognition device 1, like the voice recognition device 2, may be a single unit, or it may be a configuration that includes a part of the functions of the main control device 3.

[0014] The input sources (microphone and camera) of the voice recognition device 1 and the line-of-sight recognition device 2 may be arranged to be able to acquire the voice and line of sight of the occupant in the driver's seat. For example, as shown in Fig. 2(A), it is mounted on the upper part of the rearview mirror RM.

[0015] Here, as shown in Fig. 2(A), the line-of-sight recognition device 2 has a function of recognizing at least the respective line-of-sight directions with respect to the driver's seat side area A and the passenger seat side area B. In this embodiment, it has a function of recognizing the respective line-of-sight directions with respect to the upper area C inside the vehicle around the rearview mirror RM and the rear area inside the vehicle.

[0016] Regarding the above line-of-sight recognition device 2, it is possible to recognize the line-of-sight direction from the movement of the occupant's eyes for the driver's seat side area A, the passenger seat side area B, and the upper area C inside the vehicle. For the rear area inside the vehicle, it is possible to recognize from the direction of eye movement and the orientation of the face when the occupant turns around.

[0017] When identifying an operation object and generating a command, the controller 4 performs a process of recognizing the voice by the voice recognition device 1 after recognizing the line-of-sight direction by the line-of-sight recognition device 2, and also performs a process of recognizing the line-of-sight direction by the line-of-sight recognition device 2 when the occupant's voice is uttered or before that.

[0018] Fig. 3 is a flowchart showing the processing steps of the main control device 3. That is, when starting the processing, the main control device 3 inputs the line-of-sight data acquired by the line-of-sight recognition device 2 in step S1 and stores the line-of-sight data in step S2. Next, after receiving the voice data acquired by the voice recognition device 1 in step S3, it determines in step S4 whether voice data has been input.

[0019] If there is no input of voice data in step S4 (NO), it returns to step S3. If there is input of voice data (YES), it proceeds to step S5 and determines whether the operation object is included in the voice data. If not included (NO), it returns to step S3.

[0020] If the voice data contains the object to be operated on in step S5 (YES), the process proceeds to step S6 to identify the object to be operated on, then proceeds to step S7 to generate a command which is a control instruction, and further, in step S8, the command is transmitted to the object to be operated on to end the process.

[0021] Next, the operation of the in-vehicle device will be described in more detail. While the vehicle is running, the line-of-sight direction of the occupant (driver) is always moving as shown in the line-of-sight data of FIG. 2(B). Therefore, as shown by the dotted arrow in FIG. 2(C), when the occupant M looks at the left side of the vehicle, that is, the passenger seat side area B, as shown in the voice data of FIG. 2(B), the occupant utters "Open the window."

[0022] At this time, the in-vehicle device recognizes that the line-of-sight direction of the occupant M is the passenger seat area B and also recognizes that the object to be operated on is the window. As a result, the in-vehicle device identifies from the line-of-sight data and the voice data that the object to be operated on is the left window, generates a command, and transmits the command to the drive unit of the left window. As a result, as shown in FIG. 2(B), the in-vehicle device can automatically open the left window on the passenger seat side even when the line-of-sight direction when the occupant M utters is near the door.

[0023] Note that the voice during operation only needs to contain the name of the object to be operated on and a phrase indicating its operation. If the estimated phrases are programmed in the storage unit 7, not only "Open the window," but also "Window, open," "Open the window," or "Window, open (open)" etc. can be used to operate.

[0024] FIG. 4(A) is an explanatory diagram showing the state when the rear door is opened. In this case, the vehicle is stopped, and the occupant M turns around to the left rear and utters "Open the door." At this time, the in-vehicle device recognizes that the line-of-sight direction of the occupant M (the dotted arrow in the figure) is the left rear area inside the vehicle and also recognizes that the object to be operated on is the door.

[0025] After that, the in-vehicle device identifies that the object to be operated is the rear door on the left side from the line-of-sight data and voice data, generates a command, and transmits the command to the drive unit of the left door. As a result, the in-vehicle device can automatically open the rear door on the left side only by the voice of "open the door" without the occupant uttering "open the rear door on the left side". Naturally, it is also possible to close the opened door.

[0026] Figure 4(B) is an explanatory diagram showing the state when the front wiper is operating. In this case, the occupant M utters "wiper operation" while looking ahead. At this time, the in-vehicle device recognizes that the line-of-sight direction of the occupant M is the driver's seat side area A and that the object to be operated is the wiper.

[0027] After that, the in-vehicle device identifies that the object to be operated is the front wiper from the line-of-sight data and voice data, generates a command, and transmits the command to the drive unit of the front wiper. As a result, the in-vehicle device can automatically operate the front wiper only by the voice of "wiper operation" without the occupant M uttering "front wiper operation".

[0028] Figure 4(C) is an explanatory diagram showing the state when the rear wiper is operating. In this case, the occupant M utters "wiper operation" while looking at the upper area C inside the vehicle near the rearview mirror RM. At this time, the in-vehicle device recognizes that the line-of-sight direction (dotted arrow) of the occupant M is the upper area C inside the vehicle and that the object to be operated is the wiper.

[0029] After that, the in-vehicle device identifies that the object to be operated is the rear wiper from the line-of-sight data and voice data, generates a command, and transmits the command to the drive unit of the rear wiper.

[0030] As a result, the in-vehicle device can automatically activate the rear windshield merely by the utterance of "wiper operation", without the occupant having to say "rear windshield" or even look back. Therefore, it is extremely convenient especially while the vehicle is in motion. Also, it is of course possible to stop the front or rear windshield that was in operation.

[0031] Since the above in-vehicle device employs the line-of-sight recognition device 2 having a function of recognizing the respective line-of-sight directions for at least the driver's seat side area A and the passenger seat side area B, the specified area will be roughly set. Taking into account that the line of sight of the occupant M is constantly moving, it is possible to accurately operate the object to be operated with a short-time line-of-sight movement (so-called "glancing") and a simple voice. Furthermore, since the above in-vehicle device does not need to specify the operation object from a plurality of candidates obtained from the line-of-sight data, it is possible to accurately operate the operation object only with the line-of-sight direction and voice without using a high-precision and expensive line-of-sight recognition device.

[0032] Also, since the above in-vehicle device recognizes the voice by the voice recognition device 1 after recognizing the line-of-sight direction by the line-of-sight recognition device 2, it is possible to specify the operation object and reliably operate it.

[0033] Furthermore, in the above in-vehicle device, the voice recognition device 1 recognizes a voice combining the name and operation of the operation object. That is, since the combination of the target word and the verb is natural as human speech, even while driving, by simply uttering a simple content, it is possible to specify the operation object and reliably operate it.

[0034] Furthermore, the above in-vehicle device recognizes the line-of-sight direction by the line-of-sight recognition device 2 at the time when the occupant utters or before that, and by recognizing including the past line-of-sight data, it is not necessary to look at the operation object for a certain period of time, and it is possible to specify the operation object with the line-of-sight data and voice data and reliably operate it.

[0035] Furthermore, since the in-vehicle device described above enables the line-of-sight recognition device 2 to recognize the line-of-sight directions with respect to the upper interior region C and the rear interior region in addition to the driver's seat side region A and the passenger seat side region B, it is possible to surely activate an operation target object arranged behind the occupant, thereby achieving a further improvement in convenience.

[0036] The configuration of the in-vehicle device according to the present invention is not limited to the above-described embodiment, and can be appropriately changed without departing from the gist of the present invention.

Explanation of Reference Numerals

[0037] 1 Voice recognition device 2 Line-of-sight recognition device 4 Controller 8 Object identification unit 9 Command generation unit 10 Transmitter A Driver's seat side region B Passenger seat side region C Upper interior region M Occupant (driver)

Claims

1. An in-vehicle device that receives voice operations of vehicle occupants, comprising: a voice recognition device that recognizes the voice of the occupant; a line-of-sight recognition device that recognizes the line-of-sight direction of the occupant; a controller for controlling a plurality of operation targets in the vehicle based on signals from the voice recognition device and the line-of-sight recognition device, wherein the line-of-sight recognition device has a function of recognizing the respective line-of-sight directions for at least the driver's seat side area and the passenger seat side area, and the controller an object identification unit that identifies the operation target from the voice recognized by the voice recognition device; a command generation unit that generates a command from the voice recognized by the voice recognition device; and a transmission unit that transmits the command to the identified operation target according to the line-of-sight direction recognized by the line-of-sight recognition device. The in-vehicle device is characterized by this.

2. The in-vehicle device according to claim 1, wherein the controller recognizes the voice by the voice recognition device after recognizing the line-of-sight direction by the line-of-sight recognition device.

3. The in-vehicle device according to claim 1, wherein the voice recognition device recognizes a voice combining the name and operation of the operation target.

4. The in-vehicle device according to claim 1, wherein the controller recognizes the line-of-sight direction by the line-of-sight recognition device at the time when the voice of the occupant is uttered or before that.

5. The in-vehicle device according to claim 1, wherein the line-of-sight recognition device is capable of recognizing the respective line-of-sight directions for the upper area inside the vehicle and the rear area inside the vehicle in addition to the driver's seat side area and the passenger seat side area.

Images