Output device

Abstract

To provide an output device capable of reducing troublesomeness while keeping a warning function.SOLUTION: When object information of a same object is included in a video acquired by a video acquisition unit and environmental sound acquired by a sound acquisition unit, it is possible to reduce troublesomeness while maintaining a warning function by not outputting broadcast information. Namely, when the same object is detected as both video and sound, it is highly possible that a passenger recognizes the object, and in such a case, it is possible to reduce troublesomeness without bothering to warn.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present invention relates to an output device. [Background technology]

[0002] Generally, a moving body such as a vehicle is provided with an output device that outputs notification information such as an alarm sound when it detects the approach of an object such as another moving body or a pedestrian. As such an output device, a vehicle approach detection device that detects other vehicles and issues a notification has been proposed (see, for example, Patent Document 1). The vehicle approach detection device described in Patent Document 1 uses a camera and a microphone to detect images and environmental sounds around the vehicle, and notifies occupants of the vehicle that another vehicle is approaching by emitting an alarm sound or displaying a warning message. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2017-138766 Summary of the Invention [Problem to be solved by the invention]

[0004] However, the vehicle approach detection device described in Patent Document 1 generates a warning sound or the like even when the occupant is aware of another vehicle, which may cause annoyance to the occupant. Furthermore, for example, if the warning sound or the like is generated only when the proximity of another vehicle is high, the frequency of the warning can be reduced and the annoyance can be reduced, but the occupant may be slow to notice the approach, which reduces the warning function. Thus, it has been difficult to reduce the annoyance while maintaining the warning function.

[0005] Therefore, an object of the present invention is to provide an output device that can reduce annoyance while maintaining a warning function, for example. [Means for solving the problem]

[0006] In order to solve the above-mentioned problems and achieve the object, the output device of the present invention described in claim 1 comprises a sound acquisition unit that acquires environmental sounds outside the vehicle cabin, an image acquisition unit that acquires images outside the vehicle cabin, and an output unit that, when it is determined based on the acquired environmental sounds and images that at least one of the images or the environmental sounds contains object information, outputs alarm information related to the object information to be notified inside the vehicle cabin, and is characterized in that the output unit does not output the alarm information when the image and the environmental sounds contain object information from the same object. [Brief explanation of the drawings]

[0007] [Figure 1] 1 is a block diagram showing an output device according to a first embodiment of the present invention. [Figure 2] 10 is a flowchart illustrating an example of a first notification information output process executed by a control unit of the output device. [Figure 3] FIG. 10 is a block diagram showing an output device according to a second embodiment of the present invention. [Figure 4] 10 is a flowchart illustrating an example of a second notification information output process executed by a control unit of the output device. DETAILED DESCRIPTION OF THE INVENTION

[0008] An embodiment of the present invention will be described below. An output device according to an embodiment of the present invention includes a sound acquisition unit that acquires outdoor environmental sounds, an image acquisition unit that acquires outdoor images, and an output unit that, when it is determined based on the acquired environmental sounds and images that object information is included in the image or the environmental sounds, outputs notification information related to the object information and used to notify the user in the room. When object information for the same object is included in the image and the environmental sounds, the output unit does not output the notification information.

[0009] When the same object is detected as both an image and sound, it is highly likely that the user recognizes the object. Therefore, when the image acquired by the image acquisition unit and the environmental sound acquired by the sound acquisition unit contain object information of the same object, by not outputting the notification information, it is possible to reduce annoyance while maintaining the warning function.

[0010] The output unit may output the notification information using the same recognition method as the environmental sound or video containing the object information. In this case, if the environmental sound contains the object information, the output unit preferably outputs the object information into the room as the notification information. Outputting the notification information in this manner makes it easier for the user to intuitively recognize the type of object. Furthermore, when the object information contained in the environmental sound is output as the notification information, the sound may be output as is, or may be appropriately amplified and then output.

[0011] The output unit may output the alarm information using a different perception method from that of the environmental sound or video containing the object information. That is, if the environmental sound contains the object information, the output unit may output the alarm information using video (light) or vibration, and if the video contains the object information, the output unit may output the alarm information using sound or vibration. This allows the output of the alarm information to complement the object information, thereby improving the warning function.

[0012] For example, if a user is distracted by visual information, the user may have difficulty recognizing the object even if the object is available as an image. In such cases, outputting alarm information using sound or vibration can help the user recognize the object, improving the warning function.

[0013] An output device according to another embodiment of the present invention includes a sound acquisition unit that acquires outdoor environmental sounds, an image acquisition unit that acquires outdoor images, and an output unit that, when it is determined based on the acquired environmental sounds and images that object information is included in the image or the environmental sounds, outputs notification information related to the object information and for issuing a notification to the room. When only one object information item is included in the image and the environmental sounds and the objects are the same, the output unit does not output the notification information.

[0014] In other words, when the video or environmental sound contains information about multiple objects, the output unit outputs the notification information even if the video and environmental sound contain information about the same object. When the video or environmental sound contains information about multiple objects, the user may only recognize some of the objects. By not omitting to output the notification information in such cases, the warning function can be improved. [Example]

[0015] Examples of the present invention will be specifically described below.

[0016] [First Example] 1, the output device 1A of this embodiment includes a sound acquisition unit 2, a video acquisition unit 3, a control unit 4, and an output unit 5, and is mounted on a vehicle as a moving object. The user is a passenger (particularly the driver) of this vehicle.

[0017] The sound acquisition unit 2 is, for example, a microphone, and acquires environmental sounds outside the vehicle cabin by collecting them. The sound acquisition unit 2 may be provided in one location on the vehicle, or in multiple locations (for example, at the four corners when the vehicle is viewed from above). The sound acquisition unit 2 transmits information on the acquired environmental sounds (acquisition results) to the control unit 4.

[0018] The image acquisition unit 3 is, for example, a camera, and acquires images by capturing images of the outside of the vehicle cabin. In this embodiment, the image acquisition unit 3 captures images in front of the vehicle, but it may also capture images of the side, rear, or multiple directions. The image acquisition unit 3 transmits information about the acquired images (acquisition results) to the control unit 4. Note that the image acquisition unit 3 may be any unit that acquires images by detecting light that does not cause diffraction, such as visible light, infrared light, or ultraviolet light. Furthermore, the image acquisition unit 3 may transmit light and receive reflected light from an object, or it may simply receive light without transmitting it.

[0019] The control unit 4 is configured with a CPU (Central Processing Unit) equipped with memories such as RAM (Random Access Memory) and ROM (Read Only Memory), and is responsible for overall control of the output device 1A. Note that the meter ECU provided in the vehicle may be used as the control unit 4, or a dedicated control unit 4 may be provided. The control unit 4 transmits a control signal to the output unit 5 as will be described later.

[0020] The control unit 4 functions as an identification unit, identifying object information contained in the environmental sound and video based on the results of acquisition by the sound acquisition unit 2 and the video acquisition unit 3. For example, the control unit 4 identifies the sound of a bicycle or motorcycle as an object as object information, or the voice of a child as an object as object information, based on the wavelength, waveform, intervals, etc. of the sound contained in the environmental sound. Furthermore, the control unit 4 identifies the image of a bicycle or motorcycle as an object as object information, or the image of a child as object information, based on the size, shape, color, movement, etc. of the image contained in the video.

[0021] The object to be identified may be a human, an animal, or another moving object. When a human is the object, the age of the object may be arbitrary, and the presence of the human may be simply detected, or the age of the human may also be estimated.

[0022] The output unit 5 is composed of a sound generating unit such as a speaker and a display unit such as a display screen provided on an instrument panel, and is configured to be able to output alarm information by sound (alarm sound) and alarm information by video (warning display).

[0023] The first alarm information output process executed by the control unit 4 will be described below with reference to Fig. 2. The control unit 4 executes the first alarm information output process, for example, when the vehicle is running or starting. In the alarm information output process, the control unit 4 first transmits an acquisition command signal to the sound acquisition unit 2 and the video acquisition unit 3 to acquire information (step S1). Next, the control unit 4 receives signals of the acquisition results (environmental sound and video) from the sound acquisition unit 2 and the video acquisition unit 3 (step S2), and identifies object information contained in the environmental sound and video (step S3).

[0024] Next, the control unit 4 determines whether or not the environmental sound includes object information (step S4). If the environmental sound includes object information (Y in step S4), the control unit 4 determines whether or not the video includes object information (step S5). If the video includes object information (Y in step S5), the control unit 4 determines whether or not the object in the object information included in the environmental sound is the same as the object in the object information included in the video (step S6).

[0025] If the object in the object information contained in the environmental sound is the same as the object in the object information contained in the video (Y in step S6), the control unit 4 does not send a control signal to the output unit 5 (step S7) and returns to step S1. If the object in the object information contained in the environmental sound is different from the object in the object information contained in the video (N in step S6), the control unit 4 sends a control signal to the output unit 5 to output notification information (step S8) and returns to step S1. On the other hand, if the video does not contain object information (N in step S5), the control unit 4 sends a control signal to the output unit 5 to output notification information (step S9) and returns to step S1.

[0026] On the other hand, if the environmental sound does not include object information (N in step S4), the control unit 4 determines whether or not the video includes object information (step S10). If the video includes object information (Y in step S10), the control unit 4 sends a control signal to the output unit 5 to output notification information (step S11). If the video does not include object information (N in step S10), the control unit 4 does not send a control signal to the output unit 5 (step S12) and returns to step S1.

[0027] Here, an example of the control signal transmitted in steps S8, S9, and S11 will be described. When the control signal is transmitted in step S9, the environmental sound contains object information, but the video does not contain object information, and the object is detected only by sound. In this case, the control unit 4 transmits a control signal to the output unit 5 to output the object information contained in the environmental sound as notification information into the vehicle cabin. That is, the output unit 5 is caused to output the notification information from the speaker.

[0028] When outputting the object information contained in the environmental sound into the vehicle cabin, the component of the acquired environmental sound corresponding to the object information may be appropriately amplified and output into the vehicle cabin, or audio information corresponding to the object information may be stored in advance and output into the vehicle cabin by playing back this audio information. For example, when a child's voice is detected as the object, the voice may be output into the vehicle cabin as is, or may be amplified and output, or audio information pre-stored as the child's voice may be played back.

[0029] When a control signal is transmitted in step S11, the environmental sound does not contain object information, the video contains object information, and the object has been detected from the video alone. In this case, the control unit 4 transmits a control signal to the output unit 5 to output an alarm sound as notification information into the vehicle cabin. That is, the output unit 5 is caused to output the notification information from its speaker. This alarm sound may be a simple buzzer sound, or may be a voice that notifies the passenger of the approaching object (for example, a voice saying, "A bicycle is approaching from the front left").

[0030] In addition, in steps S9 and S11, a control signal may be sent to the output unit 5 to display a warning on the display unit. That is, the output unit 5 may be caused to output the notification information using the same perception method as that for the environmental sound or video containing object information, or may be caused to output the notification information using a perception method different from that for the environmental sound or video containing object information. In addition, in steps S9 and S11, both an alarm sound and a warning display may be output.

[0031] In step S8, the output unit 5 may be caused to output both an audible warning and a visual warning, or may be caused to output only one of them.

[0032] In addition, the output unit 5 may have a vibration generating unit provided in a position where a passenger can come into contact, such as the steering wheel or seat, and in steps S8, S9, and S11, a warning vibration may be output together with an alarm sound or alarm display, or only a warning vibration may be output.

[0033] With the above configuration, when the image acquired by the image acquisition unit and the environmental sound acquired by the sound acquisition unit contain object information of the same object, notification information is not output, thereby reducing annoyance while maintaining the warning function. In other words, when the same object is detected as both an image and sound, it is highly likely that the occupant recognizes the object, and in such a case, annoyance can be reduced by deliberately not issuing a warning.

[0034] Furthermore, if the environmental sound contains object information, the object information contained in the environmental sound can be output as notification information into the vehicle cabin, making it easier for the passengers to intuitively recognize the type of object.

[0035] Furthermore, if the output unit 5 is made to output alarm information using a different recognition method from that of environmental sounds or images containing object information, the alarm information can be output complementarily to the object information, thereby improving the warning function.

[0036] [Second Example] 3, the output device 1B of this embodiment includes a sound acquisition unit 2, a line of sight acquisition unit 6, a control unit 4, and an output unit 5, and is mounted on a vehicle as a moving object. The user is a passenger (particularly the driver) of this vehicle.

[0037] The gaze acquisition unit 6 acquires the gaze direction of the driver by, for example, detecting the movement of the passenger's eyeballs. The gaze acquisition unit 6 may also acquire the gaze direction based on the position and orientation of the passenger's head. The gaze acquisition unit 6 transmits information on the acquired gaze direction (acquisition result) to the control unit 4.

[0038] The second alarm information output process executed by the control unit 4 will be described below with reference to Fig. 4. The control unit 4 executes the second alarm information output process, for example, when the vehicle is running or starting. In the alarm information output process, the control unit 4 first transmits an acquisition command signal to the sound acquisition unit 2 and the gaze acquisition unit 6 to acquire information (step S13). Next, the control unit 4 receives signals of the acquisition results (environmental sound and gaze direction) from the sound acquisition unit 2 and the gaze acquisition unit 6 (step S14), and identifies object information contained in the environmental sound and the direction in which the object is located (step S15).

[0039] Next, the control unit 4 determines whether or not the environmental sound includes object information (step S16). If the environmental sound includes object information (Y in step S16), the control unit 4 determines whether or not an object is located in the passenger's line of sight (step S17). If an object is located in the passenger's line of sight (Y in step S17), the control unit 4 does not send a control signal to the output unit 5 (step S18) and returns to step S13. If an object is not located in the passenger's line of sight (N in step S17), the control unit 4 sends a control signal to the output unit 5 to output notification information (step S19) and returns to step S13. On the other hand, if the environmental sound does not include object information (N in step S16), the control unit 4 does not send a control signal to the output unit 5 (step S20) and returns to step S13.

[0040] Here, an example of the control signal transmitted in step S19 will be described. When the control signal is transmitted in step S18, although the environmental sound contains object information, the object is not located in the line of sight. In other words, it is highly likely that the passenger does not recognize the object. In this case, the control unit 4 transmits a control signal to the output unit 5 so as to output the object information contained in the environmental sound as notification information into the vehicle cabin. In other words, it causes the speaker of the output unit 5 to output the notification information.

[0041] When outputting the object information contained in the environmental sound into the vehicle cabin, the component of the acquired environmental sound corresponding to the object information may be appropriately amplified and output into the vehicle cabin, or audio information corresponding to the object information may be stored in advance and reproduced to be output into the vehicle cabin. For example, when a child's voice is detected as the object, the voice may be output into the vehicle cabin as is, or may be amplified and output, or audio information pre-stored as the child's voice may be reproduced. Also, an alarm sound may be output from the speaker of the output unit 5 as notification information. This alarm sound may be a simple buzzer sound, or may be a sound that notifies the passenger of the approaching object (for example, a sound saying, "A bicycle is approaching from the front left").

[0042] In addition, in step S19, a control signal may be sent to the output unit 5 to display a warning on the display unit. That is, the output unit 5 may be caused to output notification information using a different recognition method from that used for environmental sound. In addition, in step S19, both an alarm sound and a warning display may be output.

[0043] In addition, the output unit 5 may have a vibration generating unit provided in a position where a passenger can come into contact, such as the steering wheel or the seat, and in step S19, a warning vibration may be output together with an alarm sound or an alarm display, or only a warning vibration may be output.

[0044] When the environmental sound contains information about a plurality of objects, it is sufficient to determine whether or not each of the objects is located in the line of sight.

[0045] With the above configuration, when an object is located in the line of sight acquired by the line of sight acquisition unit of the occupant, notification information about the object is not output, thereby reducing annoyance while maintaining the warning function. In other words, when an object is located in the line of sight of the occupant, there is a high possibility that the occupant is aware of the object, and in such a case, annoyance can be reduced by deliberately not issuing a warning.

[0046] Furthermore, by outputting object information contained in the environmental sound as notification information into the vehicle cabin, it becomes easier for the passengers to intuitively recognize the type of object.

[0047] Furthermore, if the output unit 5 is made to output notification information other than sound, the notification information can be output in a complementary manner to the object information contained in the environmental sound, thereby improving the warning function.

[0048] The present invention is not limited to the above-described embodiment, but includes other configurations that can achieve the object of the present invention, and the following modifications are also included in the present invention.

[0049] For example, in the first embodiment, the number of object information contained in the environmental sound and the video is not determined, and if the object in the object information contained in the video is the same, no alarm information is output (steps S6, S7). However, the number of object information contained in the environmental sound and the video may be determined, and if only one object information is contained in each of the video and the environmental sound and these objects are the same, no alarm information may be output.

[0050] In other words, if the video or environmental sound contains information about multiple objects, notification information may be output even if the video and environmental sound contain information about the same object. If the video or environmental sound contains information about multiple objects, the user may only recognize some of the objects. In such cases, not omitting to output notification information can improve the warning function.

[0051] In addition, in the second embodiment, it is determined whether or not an object is located in the line of sight (steps S17 to S19), but it is also possible to determine the number of objects located in the line of sight, and not output the alarm information if only one object is located in the line of sight.

[0052] That is, notification information may be output when multiple objects are located in the line of sight. When multiple objects are located in the line of sight, the user may only recognize some of the objects. By not omitting to output notification information in such cases, the warning function can be improved.

[0053] In the first embodiment, object information included in the image in front of the vehicle acquired by the image acquisition unit 3 is identified. However, the image in which the object information is identified may be selected depending on the traveling direction of the vehicle. That is, when the vehicle is moving forward (or is about to start moving forward), object information included in the image in front of the vehicle may be identified, and when the vehicle is reversing (or is about to start reversing), object information included in the image behind the vehicle may be identified. The traveling direction of the vehicle may be determined based on, for example, the position of the shift lever. Furthermore, when the vehicle is about to change lanes, object information included in the images to the sides and rear of the vehicle may be identified. Whether the vehicle is about to change lanes may be determined based on, for example, the operation of the turn signal or the steering wheel.

[0054] Furthermore, although the output device 1A of the first embodiment is equipped with the sound acquisition unit 2 and the video acquisition unit 3, and the output device 1B of the second embodiment is equipped with the sound acquisition unit 2 and the line of sight acquisition unit 6, the output means may be equipped with the sound acquisition unit 2, the video acquisition unit 3, and the line of sight acquisition unit 6. In this case, if the video acquired by the video acquisition unit and the environmental sound acquired by the sound acquisition unit contain object information of the same object, and this object is located in the line of sight of the passenger, the notification information may not be output.

[0055] In the first and second embodiments, the output unit 5 is configured to include a sound generating unit and a display unit and is configured to be able to output alarm information by sound and alarm information by video, but the output unit may output the alarm information as a signal to an external device. That is, the output device may not have a sound generating unit or a display unit, but may send a signal to a sound generating unit or a display unit as an external device to generate an alarm sound or an alarm display.

[0056] Although the best configurations and methods for carrying out the present invention have been disclosed above, the present invention is not limited thereto. That is, although the present invention has been particularly illustrated and described mainly with reference to specific embodiments, those skilled in the art can make various modifications to the above-described embodiments in terms of shape, material, quantity, and other detailed configurations without departing from the scope of the technical idea and purpose of the present invention. Therefore, the above-disclosed descriptions limiting the shape, material, etc. are provided as examples to facilitate understanding of the present invention and are not intended to limit the present invention. Therefore, descriptions using names of components that are free from some or all of the limitations on shape, material, etc. are included in the present invention. [Explanation of symbols]

[0057] 1A output device 2 Sound acquisition section 3. Video acquisition unit 4. Control Unit 5 Output section

Claims

1. a sound acquisition unit that acquires environmental sounds outside the vehicle; an image acquisition unit that acquires an image of the outside of the vehicle; an output unit that, when it is determined based on the acquired environmental sound and image that object information is included in at least one of the image and the environmental sound, outputs notification information related to the object information to be notified to an interior of the vehicle; The output device is characterized in that the output unit does not output the notification information when the object information for the same object is included in the video and the environmental sound.

2. The output device according to claim 1 , wherein the output unit outputs the notification information in the same manner as the environmental sound or the video including the object information.

3. The output device according to claim 2 , wherein, when the environmental sound includes the object information, the output unit outputs the object information into the vehicle interior as the notification information.

4. The output device according to claim 1 , wherein the output unit outputs the notification information in a manner different from that of the environmental sound or the image including the object information.

5. a sound acquisition unit that acquires environmental sounds outside the vehicle; an image acquisition unit that acquires an image of the outside of the vehicle; an output unit that, when it is determined based on the acquired environmental sound and image that object information is included in at least one of the image and the environmental sound, outputs notification information related to the object information to be notified to an interior of the vehicle; The output device is characterized in that the output unit does not output the notification information if the video and the environmental sound each contain only one piece of object information and if these objects are the same.

Images