Voice output device, voice output method and voice output program

Abstract

To provide a voice output device, etc. which can prevent a driver of a mobile object from driving as looking away by being distracted by an object outside the mobile object.SOLUTION: A voice output device performs notification of providing information regarding an object of looking away, and a warning for stopping the looking away to a driver by voice when it is determined that the driver of a mobile object looks away.SELECTED DRAWING: Figure 3

Description

[Technical field]

[0001] The present application relates to the technical field of audio output devices and the like that can prevent a driver of a mobile body from looking away (including looking aside) while driving. [Background technology]

[0002] Patent Document 1 discloses an inattentive driving detection device that includes a main camera that captures the face of the driver in the vehicle's driver's seat, an inattentive driving determination unit that determines whether or not the driver is inattentive driving based on the direction of the driver's line of sight obtained from the image captured by the main camera, and an alarm output unit that outputs an alarm when the inattentive driving determination unit determines that the driver is inattentive driving. This driving detection device determines that the driver is not inattentive driving if the driver's line of sight is directed toward a specific in-vehicle component and the time that the driver keeps directing the line of sight toward the in-vehicle component does not exceed a reference time. This prevents an alarm from being output in cases that do not correspond to inattentive driving, for example, by not outputting an alarm when the driver checks the driving route displayed on the display unit of the car navigation system for only a short time. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2020-024532 A Summary of the Invention [Problem to be solved by the invention]

[0004] On the other hand, for example, it is a safety problem if a driver of a moving object such as a vehicle becomes distracted by objects outside the moving object, such as stores along the road, and continues to drive while looking away.

[0005] Therefore, in consideration of one example of such a problem, the present invention has an objective of providing an audio output device, etc. that can prevent a driver of a mobile body from being distracted by objects outside the mobile body and driving while looking away. [Means for solving the problem]

[0006] The invention described in claim 1 is characterized by comprising a determination means for determining whether a driver of a moving body is looking away, and an audio output means for, when it is determined that the driver is looking away, notifying the driver by voice that information regarding the object of the looking away will be provided and issuing a warning to stop the looking away.

[0007] The invention described in claim 11 is an audio output method using an audio output device, characterized in that it includes a determination step of determining whether or not a driver of a moving body is looking away, and an audio output step of, when it is determined that the driver is looking away, informing the driver by audio of information regarding the object of the looking away and issuing a warning to stop the looking away.

[0008] The invention described in claim 12 is characterized in that a computer included in an audio output device functions as a determination means for determining whether or not a driver of a moving body is looking away, and, when it is determined that the driver is looking away, informs the driver by voice that information regarding the object of the look away will be provided, and issues a warning to stop the driver from looking away. [Brief description of the drawings]

[0009] [Figure 1] 1 is a block diagram showing an example of the configuration of an audio output device 100 according to an embodiment. [Diagram 2] 1 is a block diagram showing an example of the configuration of a looking away warning device 200 according to an embodiment of the present invention. [Diagram 3] 13 is a flowchart showing an example of a warning process performed by a control unit 211 in the present embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] An embodiment of the present invention will be described with reference to FIG.

[0011] As shown in FIG. 1, the audio output device 100 includes a determination unit 101 and an audio output unit 102.

[0012] The determining means 101 determines whether or not the driver of the moving body is looking away.

[0013] When it is determined that the driver is looking away, the audio output means 102 notifies the driver by audio that information regarding the object of the driver's gaze is to be provided, and issues a warning to stop the driver from looking away.

[0014] According to the audio output device 100, when the driver of a moving body is looking away, the driver is notified by audio that information about the object of his or her gaze is provided to the driver, and a warning is issued to stop looking away. Therefore, the driver knows that he or she will be able to obtain information about the object of his or her gaze and stops looking at the object, thereby preventing the driver from driving while looking away. EXAMPLES

[0015] Next, an example corresponding to the above-described embodiment will be described. Note that the example described below is an example in which the present invention is applied to a distraction warning device 200 mounted on an automobile.

[0016] [1. Configuration of the Looking Away Warning Device 200] The configuration of the looking-away warning device 200 in this embodiment will be described with reference to Fig. 2. Fig. 2 is a block diagram showing an example of the configuration of the looking-away warning device 200.

[0017] As shown in FIG. 2, the distraction warning device 200 is configured to include a control unit 211, a memory unit 212 consisting of a HDD (Hard Disk Drive) or SSD (Solid State Drive) or the like, a communication unit 213, a display unit 214, a touch panel 215, an in-vehicle microphone 216, a gaze measurement device 217, a camera 218 for photographing inside the vehicle, a camera 219 for photographing outside the vehicle, and a speaker 220.

[0018] The control unit 211 is composed of a CPU 211a that controls the entire control unit 211, a ROM 211b in which a control program for controlling the control unit 211 and the like are stored in advance, and a RAM 211c that temporarily stores various data. The CPU 211a functions as a "determination means, audio output means" by reading and executing various programs stored in the ROM 211b and the storage unit 212. The control unit 211 or the CPU 211a corresponds to a "computer."

[0019] The storage unit 212 stores various programs such as an OS (Operating System) and application programs, and data and information used in the various programs. The storage unit 212 also stores a voice output program for notifying the driver of the fact that information about the object of the driver's gaze is being provided and for issuing a warning to stop the driver from looking away, when the driver is looking away. The various programs may be acquired, for example, from a server device or the like via a network, or may be read from a recording medium such as a USB memory.

[0020] The communication unit 213 has a wireless communication device or the like, and communicates with an external server or terminal to transmit and receive data to and from each other.

[0021] The display unit 214 is configured to include a graphics controller 214a and a buffer memory 214b consisting of a memory such as a VRAM (Video RAM). In this configuration, the graphics controller 214a controls the display unit 214 and the touch panel 215 based on control information sent from the control unit 211. The buffer memory 214b also temporarily stores image data that can be instantly displayed on each of the touch panels 215. Then, an image is displayed on the touch panel 215 based on the image data output from the graphics controller 214a.

[0022] The touch panel 215 displays an image based on the image data received from the display unit 214. The touch panel 215 also detects a touch operation by the operator, and transmits touch operation data indicating the touched position and the like to the control unit 211. The control unit 211 determines which operation has been performed based on the touch operation data, and performs processing according to the content of the operation.

[0023] An in-car microphone 216 is provided inside the automobile, and converts the voices of the driver and passengers inside the vehicle into electrical signals and outputs them to the control unit 211 .

[0024] The eye tracking device 217 measures the driver's gaze and detects where the driver is looking. Conventional publicly known technology can be used for gaze measurement. For example, the driver is asked to wear a glasses-type wearable device, which irradiates weak infrared light onto the driver's eyeball, captures the reflected light (Purkinje image) and the pupil with a camera, and measures the direction of the gaze based on the positional relationship between the center of the pupil and the Purkinje image.

[0025] The in-vehicle photographing camera 218 is provided inside the automobile and outputs an image of the driver to the control unit 211. The control unit 211 determines whether the driver is looking away based on the image photographed by the in-vehicle photographing camera 218.

[0026] The looking-away warning device 200 may include at least one of the line-of-sight measuring device 217 and the in-vehicle photographing camera 218. The control unit 211 may determine whether the driver is looking away based on at least one of the line of sight of the driver measured by the line-of-sight measuring device 217 and the image of the driver photographed by the in-vehicle photographing camera 218. At this time, the control unit 211 may determine that the driver is looking away when the driver is looking in a direction other than the traveling direction of the vehicle.

[0027] The exterior camera 219 is provided inside or outside the vehicle, and outputs images of the scenery outside the vehicle to the control unit 211. Note that the exterior camera 219 is preferably configured to be capable of capturing images of the scenery outside the vehicle in all directions, and may be configured, for example, with multiple cameras.

[0028] The speaker 220 is provided in the vehicle and outputs sound under the control of the control unit 211 .

[0029] When the driver is looking away, the control unit 211 obtains information about the object the driver is looking away from, and provides the information about the object to the driver.

[0030] Specifically, when the driver is looking away from an object located on the left side of the vehicle, the control unit 211 acquires image data of the left side of the vehicle captured by the exterior camera 219 and provides the image data. As a method of providing the image data, for example, when the vehicle is stopped or in a state where safety is ensured, the image data is displayed on the touch panel 215 or transmitted to the terminal device owned by the driver (any method of transmission can be used, such as transmission using Bluetooth (registered trademark) or e-mail transmission; the same applies below).

[0031] Furthermore, for example, when the driver is looking away from an object located on the left side of the vehicle, the control unit 211 may identify a feature in the direction of the driver's looking away based on the map data, and obtain and provide image data, text data, or voice data related to the feature from a server on the Internet or a private network. For example, the text data may be provided by displaying the text data on the touch panel 215, outputting the text data as voice from the speaker 220, or transmitting the text data to the terminal device owned by the driver. The voice data may be provided in the same manner, but when the voice data is displayed on the touch panel 215, the voice data is converted into text data before displaying it. When the voice data is transmitted to the terminal device owned by the driver, the voice data is converted into text data before transmission, or transmitted as attached data.

[0032] It is preferable that the driver can select the type of information (image data, text data, voice data) to be provided regarding the target, and the control unit 211 provides the type of information selected by the driver. When the driver looks away, he or she has a desire to look at the target of the look, so when the driver does not select the type of information, it is preferable to provide image data as information regarding the target.

[0033] Furthermore, when the driver is looking away, the control unit 211 notifies the driver by voice that information about the object of the look will be provided and issues a warning to stop the driver from looking away. For example, the control unit 211 outputs a message such as "An image of the building will be displayed later, so please stop looking away and concentrate on driving" from the speaker 220. When providing image data to the driver, the control unit 211 may notify the driver that image data related to the object will be provided together with the warning.

[0034] [2. Warning Processing by the Control Unit 211] Next, a warning process performed by the control unit 211 will be described with reference to the flowchart shown in Fig. 3. Fig. 3 is a flowchart showing an example of the warning process performed by the control unit 211.

[0035] First, the control unit 211 judges whether the driver is looking away (step S101). Specifically, the control unit 211 judges whether the driver is looking away based on at least one of the driver's line of sight measured by the line of sight measurement device 217 and the image of the driver captured by the in-vehicle photographing camera 218. It is not necessary to judge whether the driver is looking away when the vehicle is stopped. Also, when the driver is moving his / her line of sight from side to side to check for safety at an intersection or a railroad crossing, it is preferable to judge that the driver is not looking away. This is because the driver should be warned only when looking away is dangerous to safety.

[0036] When the control unit 211 determines that the driver is not looking away (step S102: NO), it ends the warning process. On the other hand, when the control unit 211 determines that the driver is looking away (step S102: YES), it acquires information about the object at which the driver is looking away (step S103). Specifically, the control unit 211 acquires image data of the object captured by the exterior camera 219, and acquires image data, text data, and audio data about the object from the server.

[0037] Next, the control unit 211 notifies the driver of the provision of information about the object on which the driver is looking aside and issues a warning to stop the driver from looking aside by voice (step S104).

[0038] Next, the control unit 211 provides the acquired information on the target to the driver (step S105), and ends the warning process. Note that, when the acquired information on the target is image data and the method of providing is displaying on the touch panel 215, the control unit 211 preferably displays the image on the touch panel 215 in a situation where there is no safety problem even if the driver looks at the image displayed on the touch panel 215, such as when the vehicle is stopped.

[0039] As described above, in the present embodiment, the distraction warning device 200 has the control unit 211 (an example of a "determination means" and "audio output means") determine whether the driver of an automobile (an example of a "moving body") is looking away, and if it determines that the driver is looking away, it notifies the driver by voice that it will provide information about the object of the distraction and issues a warning to stop looking away.

[0040] Therefore, according to the distraction warning device 200 of this embodiment, when the driver of a vehicle is looking away, the driver is notified by voice that information about the object of his or her gaze is provided to the driver, and a warning is issued to stop looking away. Therefore, the driver knows that he or she will be able to obtain information about the object of his or her gaze and stops looking at that object, thereby preventing the driver from driving while looking away.

[0041] In addition, in the distraction warning device 200 of this embodiment, the gaze measurement device 217 (an example of "gaze measurement means") measures the gaze of the driver, the in-vehicle photographing camera 218 and the control unit 211 (an example of "driver image acquisition means") acquire an image of the driver, and the control unit 211 judges whether the driver is looking away based on at least one of the driver image and the result of the gaze measurement. This makes it possible to judge whether the driver is looking away.

[0042] Furthermore, in the distraction warning device 200 in this embodiment, the control unit 211 (an example of an "acquisition means" and "provision means") acquires information about the target of distraction and provides the driver with the information about the target. This makes it possible to provide the driver with information about the target of distraction.

[0043] Furthermore, in the distraction warning device 200 of this embodiment, the vehicle exterior shooting camera 219 (an example of an "imaging means") captures an image of the distracted object, and the control unit 211 acquires image data (an example of "image information") captured by the vehicle exterior shooting camera 219 and provides the image data to the driver. This makes it possible to provide the driver with image data of the object at which the driver is looking away, captured by the vehicle exterior shooting camera 219 installed in the automobile.

[0044] Furthermore, in the distraction warning device 200 of this embodiment, the control unit 211 may acquire image data relating to the distracted object from the server and provide the image data to the driver. This allows the driver to be provided with image data of the distracted object in addition to image data captured by the vehicle exterior shooting camera 219. Even if the vehicle exterior shooting camera 219 cannot capture the distracted object due to environmental conditions such as the presence of an obstacle, the time of day, or weather, the image data acquired from the server can be provided to the driver.

[0045] Furthermore, in the distraction warning device 200 of this embodiment, the control unit 211 may notify the driver that image data related to the object of the distraction will be provided together with the warning. This allows the driver to know that image data related to the object of the distraction will be available and to stop looking at the object, thereby preventing the driver from driving while looking away.

[0046] Furthermore, in the distraction warning device 200 of this embodiment, the control unit 211 acquires information about the object of distraction from the server and provides the information by voice. This allows the driver to know that the information about the object of distraction is available by voice and to stop looking at the object, thereby preventing the driver from driving while looking away.

[0047] The image data provided to the driver may be image data of a still image captured by the exterior camera 219. Alternatively, the exterior camera 219 may constantly record the surroundings of the vehicle in video format, and the control unit 211 may provide as image data video data (video data in the direction in which the driver is looking aside) including data before and after the time when the driver's looking aside is detected (or the time when the driver's speech is detected as described in Modification 1 below), or may provide image data of a still image created by cutting out the video at that time.

[0048] [3. Modifications] Modifications of this embodiment will be described below. Note that this embodiment and the modifications may be combined as appropriate. [3.1. Variation 1] In this embodiment, the control unit 211 always judges whether the driver is looking away based on at least one of the driver image and the result of the gaze measurement, but the judgment of the driver looking away may be made based on the speech of the passenger (not limited to the driver) in the vehicle. Specifically, when the control unit 211 (an example of the "voice detection means") detects a voice related to the object of the driver looking away from the voice input to the in-vehicle microphone 216 (an example of the "voice detection means"), the control unit 211 may judge whether the driver is looking away. For example, the storage unit 212 is pre-stored with keywords (including key phrases, such as "What's that?", "Look at that?", "New store", "Cool building") corresponding to the interest in the object outside the vehicle, and the control unit 211 judges whether the driver is looking away when it detects that the passenger utters the keyword stored in the storage unit 212. As a result, the judgment of the driver looking away is made only when there is a high possibility that the driver will shift his / her gaze to an object outside the vehicle, so that the judgment of the driver looking away does not have to be made all the time.

[0049] [3.2. Variation 2] When acquiring information about the object of looking away, the control unit 211 may acquire the information by referring to a keyword uttered by the occupant. For example, when acquiring information from a server, the control unit 211 can acquire appropriate information by searching using the keyword as a search word.

[0050] [3.3. Variation 3] In this embodiment, the driver is provided with information regarding the object of his or her distraction, but the information may also be provided to someone other than the driver (for example, a passenger, or the driver's family or acquaintances who are not in the vehicle).

[0051] [3.4. Variation 4] In this embodiment, if the information provided to the driver is information about a target different from the driver's intention (for example, when information about another target next to the target about which the driver was looking is provided), the driver instructs the looking-away warning device 200 to correct the target (for example, by instructing "the building next to this"), while the control unit 211 of the looking-away warning device 200 may obtain information about the target that has been corrected in response to the instruction for correction from a server on the Internet or a private network and provide it again. That is, when the control unit 211 (an example of "command acquisition means") obtains an instruction specifying a target different from the target corresponding to the information provided to the driver, it obtains information about the target specified by the instruction and provides it to the driver.

[0052] When identifying a building standing in the same line of sight as the driver, the control unit 211 may set the priority of one of the buildings higher and identify the building with the higher priority as the target of looking away if there is a building along the road (roadside) and a tall building standing further back from the road in the same line of sight. The priority of the building to be set higher may be set in advance by the driver or the like (for example, the priority of the former may be set higher), or information about each building may be obtained from a server, and, for example, if there is a building that is highly topical (for example, a building with a lot of information on the Internet), the priority of that building may be set higher. [Explanation of symbols]

[0053] 100: Audio output device 101: Judgment means 102: Audio output means 200:Looking Away Warning Device 211: Control unit 211a: CPU 211b :ROM 211c:RAM 212: Storage section 213: Communications Department 214: Display unit 214a: Graphics controller 214b: Buffer memory 215: Touch panel 216: Car microphone 217: Eye tracking device 218: Car interior camera 219: Camera for outside the vehicle 220: Speaker

Claims

[Claim 1] A determination means for determining whether or not a driver of the moving body is looking away; a voice output means for, when it is determined that the driver is looking away, notifying the driver of the provision of information regarding the object of the looking away and issuing a warning to the driver to stop looking away by voice; An audio output device comprising:

Images