Audio output device, audio output method, and audio output program

Abstract

To provide an audio output device, etc., that can prevent the driver of a moving vehicle from being distracted by an object outside the vehicle and driving while looking away. [Solution] When it is determined that the driver of a moving vehicle is distracted, the system provides the driver with an audible notification of information about the object of their distraction and a warning to stop distraction.

Description

Technical Field

[0001] This application relates to the technical field of voice output devices and the like that can prevent a driver of a moving body from looking around (including side glances) during driving.

Background Art

[0002] Patent Document 1 discloses a side glance driving detection device including a main camera that images the face of a driver's seat in a vehicle, a side glance determination unit that determines the presence or absence of side glance driving based on the direction of the driver's line of sight obtained from an image captured by the main camera, and an alarm output unit that outputs an alarm when the side glance determination unit determines that it is side glance driving. This driving detection device determines that it is not side glance driving when the driver's line of sight is directed toward a specific in-vehicle component and the time directed toward the direction of the in-vehicle component does not exceed a reference time. Thereby, for example, when the driver checks the driving route displayed on the display unit of the car navigation for only a short time, an alarm is not output, preventing an alarm from being output in a case that does not correspond to side glance driving.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] On the other hand, for example, when a driver of a moving body such as a vehicle is distracted by an object outside the moving body such as a roadside store and drives while looking around, there are safety problems.

[0005] Therefore, in view of such an example of the problem, an object of the present invention is to provide a voice output device and the like that can prevent a driver of a moving body from being distracted by an object outside the moving body and driving while looking around.

Means for Solving the Problems

[0006] The invention described in claim 1 is characterized by comprising: determination means for determining whether or not the driver of a moving object is distracted; and, if it is determined that the driver is distracted, voice output means for providing the driver with a voice notification that information regarding the object of the distraction will be provided later, and a warning to stop the distraction.

[0007] The invention described in claim 11 is a voice output method using a voice output device, characterized in that it includes a determination step of determining whether or not the driver of a moving object is distracted, and a voice output step of, if it is determined that the driver is distracted, notifying the driver that information regarding the object of the distraction will be provided later, and giving a warning to stop the distraction.

[0008] The invention described in claim 12 is characterized in that the computer included in the voice output device functions as a determination means for determining whether or not the driver of a mobile vehicle is looking away, and, if it is determined that the driver is looking away, a voice output means that provides the driver with a voice notification that information regarding the object of the distraction will be provided later, and a warning to stop the distraction. [Brief explanation of the drawing]

[0009] [Figure 1] This is a block diagram showing an example configuration of the audio output device 100 in an embodiment. [Figure 2] This is a block diagram showing an example configuration of the distraction warning device 200 in this embodiment. [Figure 3] This flowchart shows an example of warning processing by the control unit 211 in this embodiment. [Modes for carrying out the invention]

[0010] A description of an embodiment for carrying out the present invention will be made using Figure 1.

[0011] As shown in Figure 1, the audio output device 100 includes a determination means 101 and an audio output means 102.

[0012] The determination means 101 determines whether or not the driver of the moving vehicle is distracted.

[0013] If the audio output means 102 determines that the driver is distracted, it provides the driver with an audio notification of information about the object of the distraction and a warning to stop the distraction.

[0014] According to the voice output device 100, when the driver of a moving vehicle is distracted, the driver is notified by voice that information about the object of their distraction will be provided to them, and a warning is issued to stop them from distracting themselves. As a result, the driver will know that they can obtain information about the object of their distraction and will stop distracting themselves from that object, thus preventing the driver from driving while distracted. [Examples]

[0015] Next, embodiments corresponding to the embodiments described above will be explained. The embodiments described below are examples in which the present invention is applied to a distraction warning device 200 mounted on an automobile.

[0016] [1. Configuration of the distraction warning device 200] The configuration of the distraction warning device 200 in this embodiment will be explained using Figure 2. Figure 2 is a block diagram showing an example configuration of the distraction warning device 200.

[0017] As shown in Figure 2, the distraction warning device 200 is comprised of a control unit 211, a storage unit 212 consisting of an HDD (Hard Disk Drive) or SSD (Solid State Drive), a communication unit 213, a display unit 214, a touch panel 215, an in-vehicle microphone 216, an eye-tracking device 217, an in-vehicle camera 218, an exterior camera 219, and a speaker 220.

[0018] The control unit 211 is composed of a CPU 211a that controls the entire control unit 211, a ROM 211b in which a control program for controlling the control unit 211 and the like are stored in advance, and a RAM 211c that temporarily stores various data. Then, the CPU 211a functions as a "determination means, voice output means" by reading and executing various programs stored in the ROM 211b and the storage unit 212. The control unit 211 or the CPU 211a corresponds to a "computer".

[0019] The storage unit 212 stores various programs such as an OS (Operating System), application programs, etc., as well as data and information used in various programs. In addition, the storage unit 212 stores a voice output program for notifying the driver of a vehicle by voice and warning to stop looking around when the driver of the vehicle is looking around, and providing information about the object of looking around. Note that the various programs may be obtained from, for example, a server device or the like via a network, or may be read from those recorded on a recording medium such as a USB memory.

[0020] The communication unit 213 has a wireless communication device or the like, and communicates with an external server or terminal to mutually transmit and receive data.

[0021] The display unit 214 includes a graphics controller 214a and a buffer memory 214b composed of a memory such as a VRAM (Video RAM). In this configuration, the graphics controller 214a controls the display unit 214 and the touch panel 215 based on the control information sent from the control unit 211. In addition, the buffer memory 214b temporarily stores image data that can be immediately displayed on each touch panel 215. Then, an image is displayed on the touch panel 215 based on the image data output from the graphics controller 214a.

[0022] The touch panel 215 displays an image based on the image data received from the display unit 214. Also, the touch panel 215 detects a touch operation by the operator and transmits touch operation data indicating the touched position and the like to the control unit 211. The control unit 211 discriminates which operation has been performed based on the touch operation data and performs processing according to the operation content.

[0023] The in-vehicle microphone 216 is provided inside the automobile, converts the voices of the driver and passengers inside the vehicle into electrical signals, and outputs them to the control unit 211.

[0024] The eye gaze measurement (eye tracking) device 217 measures the driver's eye gaze and detects where the driver is looking. For eye gaze measurement, a conventionally well-known technique can be used. For example, the driver is made to wear a glasses-type wearable terminal, the wearable terminal irradiates the driver's eyeball with weak infrared light, the reflected light (Purkinje image) and the pupil are photographed with a camera, and the direction of the eye gaze is measured based on the positional relationship between the center of the pupil and the position of the Purkinje image.

[0025] The in-vehicle camera 218 is provided inside the automobile and outputs an image of the photographed driver to the control unit 211. The control unit 211 determines whether the driver is looking around based on the image photographed by the in-vehicle camera 218.

[0026] Note that the looking-around warning device 200 may include at least one of the eye gaze measurement device 217 and the in-vehicle camera 218. Also, the control unit 211 may determine whether the driver is looking around based on at least one of the driver's eye gaze measured by the eye gaze measurement device 217 and the driver's image photographed by the in-vehicle camera 218. At this time, the control unit 211 may determine that the driver is looking around when the driver is looking in a direction other than the traveling direction of the automobile.

[0027] The external camera 219 is installed inside or outside the vehicle and outputs images of the scenery outside the vehicle to the control unit 211. Preferably, the external camera 219 is configured to capture images in all directions outside the vehicle, and may be composed of multiple cameras, for example.

[0028] The speaker 220 is installed inside the vehicle and outputs sound under the control of the control unit 211.

[0029] The control unit 211 acquires information about the object the driver is looking at when the driver is distracted, and provides the driver with information about that object.

[0030] Specifically, if the driver is distracted by an object located on the left side of the vehicle, the control unit 211 acquires image data of the left side of the vehicle using the external camera 219 and provides the image data. The image data can be provided, for example, by displaying it on the touch panel 215 or transmitting it to the driver's terminal device when safety is ensured, such as when the vehicle is stopped (transmission can be via Bluetooth®, email, or any other method; the same applies hereinafter).

[0031] Furthermore, the control unit 211 may, for example, identify a feature in the direction the driver is looking based on map data when the driver is distracted by an object located to the left of the vehicle, and obtain and provide image data, text data, or audio data related to that feature from a server on the Internet or a private network. Methods for providing text data include, for example, displaying the text data on the touch panel 215, outputting it as audio from the speaker 220, or transmitting it to the driver's terminal device. The method for providing audio data is similar, but when displaying it on the touch panel 215, the audio data is converted to text data before display. Similarly, when transmitting to the driver's terminal device, the audio data is converted to text data before transmission or transmitted as attached data.

[0032] Furthermore, the driver should be able to select the type of information provided regarding the subject (image data, text data, audio data), and it is preferable that the control unit 211 provides the type of information selected by the driver. Also, when the driver is distracted, there is a desire to see the object of their distraction, so if the driver does not select the type of information, it is preferable to provide image data as information regarding the subject.

[0033] Furthermore, if the driver is distracted, the control unit 211 will provide the driver with an audible notification that information about the object of their distraction will be provided, and a warning to stop distraction. For example, the control unit 211 will output a message from the speaker 220 such as, "We will show you an image of that building later, so please stop distraction and concentrate on driving." In addition, when providing image data to the driver, the control unit 211 may also provide a warning along with notification that image data related to the object will be provided.

[0034] [2. Warning processing by the control unit 211] Next, the warning processing by the control unit 211 will be explained using the flowchart shown in Figure 3. Figure 3 is a flowchart showing an example of warning processing by the control unit 211.

[0035] First, the control unit 211 determines whether the driver is looking away (step S101). Specifically, the control unit 211 determines whether the driver is looking away based on at least one of the driver's gaze measured by the gaze measurement device 217 and the image of the driver taken by the in-vehicle camera 218. Note that if the vehicle is stopped, it is not necessary to determine whether the driver is looking away. Also, if the driver is moving their gaze from side to side to check for safety at an intersection or railroad crossing, it is preferable to determine that they are not looking away. This is because the driver should only be warned if they are looking away and pose a safety risk.

[0036] If the control unit 211 determines that the driver is not distracted (step S102: NO), it terminates the warning process. On the other hand, if the control unit 211 determines that the driver is distracted (step S102: YES), it acquires information about the object the driver is distracted by (step S103). Specifically, the control unit 211 acquires image data of the object captured by the external camera 219, or acquires image data, text data, and audio data related to the object from the server.

[0037] Next, the control unit 211 provides voice notification of information regarding the object the driver is looking at, and a warning to stop looking away (step S104).

[0038] Next, the control unit 211 provides the driver with the information about the acquired object (step S105) and terminates the warning process. If the information about the acquired object is image data and the method of provision is display on the touch panel 215, it is preferable that the control unit 211 displays it in a situation where it is not a safety issue for the driver to look at the image displayed on the touch panel 215, such as when the vehicle is stopped.

[0039] As described above, in this embodiment, the distraction warning device 200 has a control unit 211 (an example of a "determination means" and an "audio output means") which determines whether the driver of the automobile (an example of a "mobile body") is distracted, and if it is determined that the driver is distracted, it provides the driver with information about the object of their distraction and a warning to stop them, both by voice.

[0040] Therefore, according to the distraction warning device 200 in this embodiment, when the driver of a vehicle is distracted, the driver is notified that information about the object of their distraction will be provided to the driver, and a warning to stop distraction is given by voice. As a result, the driver will know that they will receive information about the object of their distraction and will stop distracting themselves from that object, and consequently, it is possible to prevent the driver from driving while distracted.

[0041] Furthermore, in this embodiment, the distraction warning device 200 includes an eye-tracking device 217 (an example of "eye-tracking means") which measures the driver's gaze, an in-vehicle camera 218 and a control unit 211 (an example of "driver image acquisition means") which acquire a driver image of the driver, and the control unit 211 which determines whether the driver is distracted based on at least one of the driver image and the results of the eye-tracking measurement. This makes it possible to determine whether the driver is distracted.

[0042] Furthermore, in this embodiment, the distraction warning device 200 has a control unit 211 (an example of an "acquisition means" and a "providing means") that acquires information about the object of the distraction and provides the driver with information about that object. This makes it possible to provide the driver with information about the object of the distraction.

[0043] Furthermore, in this embodiment, the distraction warning device 200 uses an external camera 219 (an example of a "shooting means") to photograph the object the driver is looking at, and the control unit 211 acquires the image data (an example of "image information") captured by the external camera 219 and provides the image data to the driver. This allows the driver to be provided with image data of the object the driver was looking at, captured by the external camera 219 installed in the vehicle.

[0044] Furthermore, in this embodiment, the distraction warning device 200 may have a control unit 211 that acquires image data related to the object of the distraction from a server and provides this image data to the driver. This allows the driver to be provided with image data of the object of the distraction in addition to the image data captured by the external camera 219. Also, for example, even if the external camera 219 is unable to capture the object of the distraction due to the presence of an obstacle or environmental conditions such as the time of day or weather, the driver can still be provided with image data acquired from the server.

[0045] Furthermore, in this embodiment, the distraction warning device 200 may also notify the driver that the control unit 211 will provide image data relating to the object of the distraction, along with the warning. This allows the driver to know that image data relating to the object of the distraction is available and to stop distracting the driver, thereby preventing the driver from driving while distracted.

[0046] Furthermore, in this embodiment, the distraction warning device 200 has a control unit 211 that obtains information about the object of the distraction from a server and provides this information by voice. As a result, the driver will know that they can obtain information about the object of the distraction by voice and will stop distracting the driver, thus preventing the driver from driving while distracted.

[0047] The image data provided to the driver may also be still image data captured by the external camera 219. Alternatively, the external camera 219 may continuously record the area around the vehicle in video format, and the control unit 211 may provide video data (video data in the direction of the object of the distraction) that includes the time before and after the time when the driver's distraction is detected (or the time when the driver's speech is detected, as described later in Modification 1 below) as image data, or it may provide still image data created by extracting from the video at that time.

[0048] [3. Variant] The following describes some variations of this embodiment. Note that this embodiment and its variations may be combined as appropriate. [3.1. Variation 1] In this embodiment, the control unit 211 constantly determines whether the driver is looking away based on at least one of the driver image and the results of eye-tracking measurement. However, this determination of whether the driver is looking away may also be based on speech made by an occupant (not limited to the driver) inside the vehicle. Specifically, the control unit 211 (an example of a "speech detection means") may determine whether the driver is looking away when it detects speech related to an object of distraction inside the vehicle based on the sound input to the in-vehicle microphone 216 (an example of a "speech detection means"). For example, keywords (including key phrases; for example, "What's that?", "Look at that?", "A new store," "A cool building") corresponding to showing interest in an object outside the vehicle may be stored in the memory unit 212 beforehand. The control unit 211 then determines whether the driver is looking away when it detects that an occupant has uttered one of the keywords stored in the memory unit 212. This way, the determination of whether the driver is looking away is only made when there is a high probability that the driver will shift their gaze to an object outside the vehicle, thus eliminating the need to constantly determine whether the driver is looking away.

[0049] [3.2. Variation 2] When acquiring information about the object of the distracted driver, the control unit 211 may use keywords spoken by the driver as a reference to obtain the information. For example, when acquiring information from a server, the appropriate information can be obtained by searching using those keywords as search terms.

[0050] [3.3. Variation 3] In this embodiment, information regarding the object of the driver's distraction is provided to the driver, but this information may also be provided to persons other than the driver (for example, passengers, or family members or acquaintances of the driver who are not in the vehicle).

[0051] [3.4. Modification 4] In this embodiment, if the information provided to the driver concerns an object different from the driver's intention (for example, if information about a different object next to the object the driver was distracted by is provided), the driver may instruct the distraction warning device 200 to correct the object (for example, by giving an instruction such as "the building next to this"), while the control unit 211 of the distraction warning device 200 may acquire information about the corrected object from a server on the Internet or a private network in response to the instruction to correct the object and provide it again. That is, when the control unit 211 (an example of an "instruction acquisition means") acquires an instruction that specifies an object different from the object corresponding to the information provided to the driver, it acquires information about the object specified by the instruction and provides it to the driver.

[0052] When the control unit 211 identifies a distraction target, if it identifies a building that is in the same line of sight as the driver, and if there is a building along the roadside and a tall building further back from the roadside in the same line of sight, it may set a higher priority for one of the buildings and identify the building with the higher priority as the distraction target. The driver may set which building has a higher priority in advance (for example, setting the former to have a higher priority), or it may obtain information about each building from a server and, for example, set a higher priority for a building that is a topic of discussion (for example, a building with a lot of information on the internet). [Explanation of Symbols]

[0053] 100: Audio output device 101: Judgment means 102: Audio output means 200: Distraction warning device 211: Control Unit 211a: CPU 211b :ROM 211c: RAM 212: Storage section 213: Communications Department 214: Display Unit 214a: Graphics controller 214b: Buffer memory 215: Touch panel 216: In-car microphone 217: Eye-tracking devices 218: In-car camera 219: Camera for external vehicle photography 220: Speakers

Claims

1. A means for determining whether the driver of a moving vehicle is distracted, When it is determined that the driver is distracted, the audio output means provides the driver with the following information: that information regarding the object of the distraction will be provided later, and that the driver should stop distracting. An audio output device characterized by having the following features.

2. The audio output device according to claim 1, The system comprises at least one of the following: a driver image acquisition means for acquiring a driver image of the driver, and a gaze measurement means for measuring the driver's gaze. The voice output device is characterized in that the determination means determines whether or not the driver is looking away based on at least one of the driver image and the results of the eye-tracking measurement.

3. The audio output device according to claim 2, The voice output device is characterized in that the determination means determines that the driver is looking away when the driver is looking in a direction other than the direction of travel of the moving object.

4. An audio output device according to any one of claims 1 to 3, An acquisition means for acquiring information about the aforementioned target, A means for providing information about the aforementioned subject, An audio output device characterized by having the following features.

5. The audio output device according to claim 4, The system includes a means for photographing the aforementioned object, The acquisition means acquires the image information captured by the shooting means. The audio output device is characterized in that the providing means provides image information acquired by the acquisition means.

6. The audio output device according to claim 4 or 5, The acquisition means acquires image information relating to the target from the server, The audio output device is characterized in that the providing means provides image information acquired by the acquisition means from the server.

7. The audio output device according to claim 5 or 6, The audio output device is characterized in that the audio output means provides notification of the provision of image information relating to the object, along with the warning.

8. An audio output device according to any one of claims 4 to 7, The acquisition means acquires information about the target from the server, The aforementioned providing means is an audio output device characterized in that it provides the information acquired by the acquisition means from the server in audio format.

9. An audio output device according to any one of claims 4 to 8, The system includes an instruction acquisition means that acquires an instruction to specify an object different from the object corresponding to the information provided by the aforementioned provision means, The acquisition means acquires information relating to the object specified by the command, The audio output device is characterized in that the providing means provides information about the target specified by the command, which has been acquired by the acquiring means.

10. An audio output device according to any one of claims 1 to 9, The system includes a voice detection means for detecting sounds related to the object within the moving body, The voice output device is characterized in that, when the determination means detects a voice related to the target, it determines whether or not the driver is distracted.

11. A method for outputting sound using an audio output device, A determination process to determine whether the driver of the moving vehicle is distracted, If it is determined that the driver is distracted, the system provides an audio output step that informs the driver that information regarding the object of the distraction will be provided later, and issues a warning to stop the distraction. A method for outputting sound, characterized by including [something].

12. The computer included in the audio output device A means for determining whether the driver of a moving vehicle is distracted or not. When it is determined that the driver is distracted, an audio output means provides the driver with the following information: that information regarding the object of the distraction will be provided later, and a warning to stop the distraction. An audio output program characterized by functioning as such.

Images