Output device

Abstract

To provide an output device capable of reducing troublesomeness while keeping a warning function.SOLUTION: When object information of a same object is included in a video acquired by a video acquisition unit and environmental sound acquired by a sound acquisition unit, it is possible to reduce troublesomeness while maintaining a warning function by not outputting broadcast information. Namely, when the same object is detected as both video and sound, it is highly possible that a passenger recognizes the object, and in such a case, it is possible to reduce troublesomeness without bothering to warn.SELECTED DRAWING: Figure 2

Description

Technical Field

[0001] The present invention relates to an output device. Background Art

[0002] Generally, a moving body such as a vehicle is provided with an output device that outputs notification information such as an alarm sound when an approach of an object such as another moving body or a pedestrian is detected. As such an output device, a vehicle approach detection device that detects and notifies another vehicle has been proposed (see, for example, Patent Document 1). In the vehicle approach detection device described in Patent Document 1, a camera and a microphone detect images around the vehicle and environmental sounds around the vehicle, and generate a warning sound or display a warning to notify an occupant of the own vehicle that another vehicle is approaching. Prior Art Literature Patent Literature

[0003] Patent Document 1 Japanese Unexamined Patent Application Publication No. 2017-138766 Summary of the Invention Problems to be Solved by the Invention

[0004] However, the vehicle approach detection device described in Patent Document 1 generates a warning sound or the like even when the occupant is already aware of the other vehicle, which may cause annoyance to the occupant. Further, for example, if a warning sound or the like is generated only when the approach degree of another vehicle is high, the frequency of warnings can be lowered to reduce annoyance, but there is a possibility that the occupant may be delayed in noticing the approach, resulting in a degraded warning function. As described above, it has been difficult to reduce annoyance while maintaining the warning function.

[0005] Accordingly, an example of the object of the present invention is to provide an output device capable of reducing annoyance while maintaining the warning function. [Means for solving the problem]

[0006] To solve the aforementioned problems and achieve the objective, the output device of the present invention as described in claim 1 comprises: a sound acquisition unit for acquiring ambient sounds from outside the vehicle; a video acquisition unit for acquiring video from outside the vehicle; and an output unit that, when it is determined that object information is included in at least one of the video or the ambient sound based on the acquired ambient sounds and video, outputs notification information for notification inside the vehicle in relation to the object information, wherein the output unit does not output the notification information if the video and the ambient sounds include object information from the same object. [Brief explanation of the drawing]

[0007] [Figure 1] This is a block diagram showing an output device according to the first embodiment of the present invention. [Figure 2] This flowchart shows an example of the first broadcast information output process performed by the control unit of the output device. [Figure 3] This is a block diagram showing an output device according to a second embodiment of the present invention. [Figure 4] This flowchart shows an example of the second notification information output processing performed by the control unit of the output device. [Modes for carrying out the invention]

[0008] Embodiments of the present invention will be described below. An output device according to an embodiment of the present invention comprises: a sound acquisition unit that acquires ambient sounds from outside; a video acquisition unit that acquires video images from outside; and an output unit that, when it is determined that the video or ambient sounds contain object information based on the acquired ambient sounds and video, outputs notification information related to the object information for notification inside the room. The output unit does not output notification information if the video and ambient sounds contain object information from the same object.

[0009] If the same object is detected in both video and audio, it is highly likely that the user is aware of the object. Therefore, if the video acquired by the video acquisition unit and the ambient sound acquired by the audio acquisition unit both contain object information from the same object, the system can reduce annoyance while maintaining its warning function by not outputting notification information.

[0010] The output unit may output notification information using the same recognition method as ambient sound or video containing object information. In this case, if the ambient sound contains object information, it is preferable for the output unit to output this object information as notification information into the room. Outputting notification information in this way makes it easier for the user to intuitively recognize the type of object. Furthermore, when outputting object information contained in ambient sound as notification information, this sound may be output as is, or it may be amplified as appropriate before output.

[0011] The output unit may output notification information using a recognition method different from that of ambient sound or video containing object information. That is, if object information is included in the ambient sound, notification information may be output as video (light) or vibration, and if object information is included in the video, notification information may be output as sound or vibration. This allows notification information to be output complementary to object information, thereby improving the warning function.

[0012] For example, if a user is distracted by visual information, even if an object can be captured as an image, it may be difficult for the user to recognize it. In this case, outputting notification information through sound or vibration can make it easier for the user to recognize the object and improve the warning function.

[0013] Another embodiment of the present invention provides an output device comprising: a sound acquisition unit for acquiring ambient sounds from outside; a video acquisition unit for acquiring images from outside; and an output unit that, when it is determined from the acquired ambient sounds and images that the images or ambient sounds contain object information, outputs notification information related to the object information for notification inside the room. The output unit does not output notification information if the images and ambient sounds each contain only one piece of object information and these objects are the same.

[0014] In other words, the output unit outputs notification information even if the same object information is included in both the video and the ambient sound when the video or ambient sound contains information about multiple objects. When the video or ambient sound contains information about multiple objects, the user may only recognize some of the objects. By not omitting the output of notification information in such cases, the warning function can be improved. [Examples]

[0015] The following describes specific embodiments of the present invention.

[0016] [First Embodiment] As shown in Figure 1, the output device 1A of this embodiment comprises a sound acquisition unit 2, a video acquisition unit 3, a control unit 4, and an output unit 5, and is mounted on a vehicle as a mobile device. The users are the passengers (especially the driver) of this vehicle.

[0017] The sound acquisition unit 2 is, for example, a microphone, which acquires ambient sounds from outside the vehicle. The sound acquisition unit 2 may be installed in one location on the vehicle, or in multiple locations (for example, at the four corners when viewing the vehicle from above). The sound acquisition unit 2 transmits the acquired ambient sound (acquisition result) information to the control unit 4.

[0018] The image acquisition unit 3 is, for example, a camera, and acquires an image by capturing an image of the outside of the vehicle compartment. In the present embodiment, the image acquisition unit 3 is assumed to capture an image of the front of the vehicle, but it may capture an image of a side, may capture an image of the rear, or may capture images in a plurality of directions. The image acquisition unit 3 transmits information of the acquired image (acquisition result) to the control unit 4. It should be noted that the image acquisition unit 3 only needs to acquire an image by detecting light that does not cause diffraction such as visible light, infrared light, and ultraviolet light. Further, the image acquisition unit 3 may transmit light and receive reflected light from an object, or may only receive light without transmitting light.

[0019] The control unit 4 is constituted by a CPU (Central Processing Unit) including memories such as a RAM (Random Access Memory) and a ROM (Read Only Memory), and is responsible for overall control of the output device 1A. It should be noted that a meter ECU provided in the vehicle may be used as the control unit 4, or a dedicated control unit 4 may be provided. The control unit 4 transmits a control signal to an output unit 5 as described later.

[0020] The control unit 4 functions as a specifying unit that specifies object information included in environmental sound and an image based on acquisition results from the sound acquisition unit 2 and the image acquisition unit 3. For example, based on the wavelength, waveform, sound interval, etc. of the sound included in the environmental sound, the control unit 4 specifies traveling sounds of a bicycle or a motorcycle as an object as the object information, or specifies a child's voice as an object as the object information. Further, based on the size, shape, color, motion, etc. of an image included in the image, the control unit 4 specifies an image of a bicycle or a motorcycle as an object as the object information, or specifies an image of a child as an object as the object information.

[0021] It should be noted that the specified object may be a human being, an animal, or another moving body. When a human being is taken as an object, the target age is arbitrary, the control unit 4 may only detect the presence of the human being, or may also estimate the age of the human being.

[0022] An output unit 5 includes a sound generation unit such as a speaker and a display unit that is a display screen provided on, for example, an instrument panel, and is configured to be capable of outputting sound-based notification information (alarm sound) and video-based notification information (warning display).

[0023] Hereinafter, a first notification information output process executed by a control unit 4 will be described with reference to FIG. 2. The control unit 4 executes the first notification information output process, for example, when the vehicle is traveling or starting. In the notification information output process, the control unit 4 first transmits an acquisition command signal to the sound acquisition unit 2 and the video acquisition unit 3 so as to acquire information (step S1). Next, the control unit 4 receives signals of acquisition results (environmental sound and video) from the sound acquisition unit 2 and the video acquisition unit 3 (step S2), and specifies object information included in the environmental sound and the video (step S3).

[0024] Next, the control unit 4 determines whether or not the object information is included in the environmental sound (step S4). If the object information is included in the environmental sound (Y in step S4), the control unit 4 determines whether or not the object information is included in the video (step S5). If the object information is included in the video (Y in step S5), the control unit 4 determines whether or not the object corresponding to the object information included in the environmental sound and the object corresponding to the object information included in the video are the same (step S6).

[0025] If the object corresponding to the object information included in the environmental sound and the object corresponding to the object information included in the video are the same (Y in step S6), the control unit 4 does not transmit a control signal to the output unit 5 (step S7), and returns to step S1. If the object corresponding to the object information included in the environmental sound and the object corresponding to the object information included in the video are different (N in step S6), the control unit 4 transmits a control signal to the output unit 5 to output notification information (step S8), and returns to step S1. In addition, if the object information is not included in the video (N in step S5), the control unit 4 transmits a control signal to the output unit 5 to output notification information (step S9), and returns to step S1.

[0026] On the other hand, if the ambient sound does not contain object information (N in step S4), the control unit 4 determines whether or not the video contains object information (step S10). If the video contains object information (Y in step S10), the control unit 4 sends a control signal to the output unit 5 to output notification information (step S11). Also, if the video does not contain object information (N in step S10), the control unit 4 does not send a control signal to the output unit 5 (step S12) and returns to step S1.

[0027] Here, we will describe an example of a control signal transmitted in steps S8, S9, and S11. When a control signal is transmitted in step S9, the ambient sound contains object information, but the video does not, and the object is detected only by sound. In this case, the control unit 4 transmits a control signal to the output unit 5 to output the object information contained in the ambient sound into the vehicle interior as notification information. That is, it causes the speaker of the output unit 5 to output notification information.

[0028] Furthermore, when outputting object information contained in ambient sound into the vehicle interior, the component of the acquired ambient sound corresponding to the object information may be appropriately amplified and output into the vehicle interior, or audio information corresponding to the object information may be stored in advance and this audio information may be played back and output into the vehicle interior. For example, if a child's voice is detected as the object, the voice may be output directly into the vehicle interior, amplified and output, or audio information stored in advance as a child's voice may be played back.

[0029] If a control signal is transmitted in step S11, the ambient sound does not contain object information, but the video does, and the object is detected solely from the video. In this case, the control unit 4 transmits a control signal to the output unit 5 to output an alarm sound as notification information into the vehicle cabin. That is, it causes the speaker of the output unit 5 to output notification information. This alarm sound may be a simple buzzer sound, or it may be an audio message informing the occupants of the approaching object (for example, "A bicycle is approaching from the left front").

[0030] Furthermore, in steps S9 and S11, a control signal may be sent to the output unit 5 to display a warning on the display unit. That is, the output unit 5 may be instructed to output notification information using the same recognition method as the ambient sound or video containing object information, or it may be instructed to output notification information using a different recognition method than the ambient sound or video containing object information. Also, in steps S9 and S11, both an alarm sound and a warning display may be output.

[0031] In step S8, the output unit 5 may output both an alarm sound and a warning display, or it may output only one of them.

[0032] Furthermore, the output unit 5 may have a vibration generating unit located at a position where an occupant may come into contact with it, such as the steering wheel or seat. In steps S8, S9, and S11, a warning vibration may be output in conjunction with an alarm sound or alarm display, or only a warning vibration may be output.

[0033] With the above configuration, if the video acquired by the video acquisition unit and the ambient sound acquired by the sound acquisition unit contain object information from the same object, the system will not output notification information, thereby reducing annoyance while maintaining the warning function. In other words, if the same object is detected in both the video and the sound, there is a high probability that the passenger is aware of the object, and in such cases, the system can reduce annoyance by deliberately not issuing a warning.

[0034] Furthermore, if ambient sounds contain information about an object, outputting that information as notification into the vehicle cabin makes it easier for passengers to intuitively recognize the type of object.

[0035] Furthermore, by having the output unit 5 output notification information using a recognition method different from that used for ambient sound or video containing object information, the notification information can be output complementary to the object information, thereby improving the warning function.

[0036] [Second Example] As shown in Figure 3, the output device 1B of this embodiment comprises a sound acquisition unit 2, a gaze acquisition unit 6, a control unit 4, and an output unit 5, and is mounted on a vehicle as a mobile device. The users are the occupants of this vehicle (especially the driver).

[0037] The gaze acquisition unit 6 acquires the driver's gaze direction, for example, by detecting the movement of the passenger's eyeballs. The gaze acquisition unit 6 may also acquire the gaze direction based on the position and orientation of the passenger's head. The gaze acquisition unit 6 transmits the acquired gaze direction (acquisition result) information to the control unit 4.

[0038] The second notification information output process performed by the control unit 4 will be described below with reference to Figure 4. The control unit 4 performs the second notification information output process, for example, when the vehicle is moving or starting. In the notification information output process, the control unit 4 first sends an acquisition command signal to the sound acquisition unit 2 and the gaze acquisition unit 6 to acquire information (step S13). Next, the control unit 4 receives signals of the acquisition results (ambient sound and gaze direction) from the sound acquisition unit 2 and the gaze acquisition unit 6 (step S14), and identifies the object information contained in the ambient sound and the direction in which the object is located (step S15).

[0039] Next, the control unit 4 determines whether or not object information is included in the ambient sound (step S16). If object information is included in the ambient sound (Y in step S16), the control unit 4 determines whether or not the object is located in the direction of the passenger's line of sight (step S17). If the object is located in the direction of the passenger's line of sight (Y in step S17), the control unit 4 does not send a control signal to the output unit 5 (step S18) and returns to step S13. If the object is not located in the direction of the passenger's line of sight (N in step S17), the control unit 4 sends a control signal to the output unit 5 to output notification information (step S19) and returns to step S13. On the other hand, if object information is not included in the ambient sound (N in step S16), the control unit 4 does not send a control signal to the output unit 5 (step S20) and returns to step S13.

[0040] Here, an example of a control signal transmitted in step S19 will be described. When a control signal is transmitted in step S18, the ambient sound contains object information, but the object is not located in the line of sight. In other words, there is a high probability that the occupant has not recognized the object. In this case, the control unit 4 transmits a control signal to the output unit 5 so that the object information contained in the ambient sound is output into the vehicle as notification information. That is, the speaker of the output unit 5 is made to output notification information.

[0041] Furthermore, when outputting object information contained in ambient sound into the vehicle cabin, the component of the acquired ambient sound corresponding to the object information may be appropriately amplified and output into the vehicle cabin, or audio information corresponding to the object information may be stored in advance and output by playing this audio information into the vehicle cabin. For example, if a child's voice is detected as the object, the voice may be output directly into the vehicle cabin, amplified and output, or audio information stored in advance as a child's voice may be played. In addition, an alarm sound may be output as notification information from the speaker of the output unit 5. This alarm sound may be a simple buzzer sound, or it may be an audio message informing the occupants of the approaching object (for example, "A bicycle is approaching from the front left").

[0042] Furthermore, in step S19, a control signal may be sent to the output unit 5 to display a warning on the display unit. That is, the output unit 5 may be made to output notification information using a recognition method different from that of ambient sound. Also, in step S19, both an alarm sound and a warning display may be output.

[0043] Furthermore, the output unit 5 may have a vibration generating unit located at a position where an occupant may come into contact with it, such as the steering wheel or seat. In step S19, a warning vibration may be output in conjunction with an alarm sound or alarm display, or only a warning vibration may be output.

[0044] Furthermore, if the ambient sound contains information about multiple objects, it is sufficient to determine whether or not each object is located in the line of sight.

[0045] With the above configuration, if an object is located in the direction of the passenger's gaze acquired by the gaze acquisition unit, the system will not output notification information about that object, thereby reducing annoyance while maintaining the warning function. In other words, if an object is located in the direction of the passenger's gaze, there is a high probability that the passenger is aware of the object, and in such cases, the system can reduce annoyance by deliberately not issuing a warning.

[0046] Furthermore, by outputting information about objects contained in ambient sounds into the vehicle as notification information, passengers can more easily intuitively recognize the type of object.

[0047] Furthermore, by having the output unit 5 output notification information other than sound, it is possible to output notification information in a complementary manner to the object information contained in the ambient sound, thereby improving the warning function.

[0048] Furthermore, the present invention is not limited to the embodiments described above, and includes other configurations that can achieve the objectives of the present invention, as well as the following modifications.

[0049] For example, in the first embodiment described above, without determining the number of object information items included in the ambient sound and the video, if the object information item included in the ambient sound and the object information item included in the video are the same, the notification information is not output (steps S6, S7). However, the number of object information items included in the ambient sound and the video may be determined, and if only one object information item is included in the video and the ambient sound respectively, and these objects are the same, the notification information may not be output.

[0050] In other words, if the video or ambient sound contains information about multiple objects, the notification information may be output even if the video and ambient sound contain information about the same object. When the video or ambient sound contains information about multiple objects, the user may only recognize some of the objects. By not omitting the output of the notification information in such cases, the warning function can be improved.

[0051] Furthermore, in the second embodiment described above, it was determined whether or not an object is located in the line of sight (steps S17 to S19). However, it is also possible to determine the number of objects located in the line of sight and not output notification information if only one object is located in the line of sight.

[0052] In other words, notification information may be output when multiple objects are located in the line of sight. When multiple objects are located in the line of sight, the user may only recognize some of the objects. By not omitting the output of notification information in such cases, the warning function can be improved.

[0053] Furthermore, in the first embodiment described above, object information contained in the image of the front of the vehicle acquired by the image acquisition unit 3 was identified. However, depending on the direction of the vehicle's movement, it may be selected which direction of the image to identify object information from. That is, when the vehicle is moving forward (or about to start moving forward), object information contained in the image of the front of the vehicle may be identified, and when the vehicle is moving backward (or about to start moving backward), object information contained in the image of the rear of the vehicle may be identified. The direction of the vehicle's movement can be determined, for example, based on the position of the shift lever. Also, when the vehicle is about to change lanes, object information contained in the images of the sides and rear of the vehicle may be identified. Whether or not the vehicle is about to change lanes can be determined, for example, based on the operation of the turn signals or the steering wheel.

[0054] Furthermore, although the output device 1A of the first embodiment is equipped with a sound acquisition unit 2 and a video acquisition unit 3, and the output device 1B of the second embodiment is equipped with a sound acquisition unit 2 and a gaze acquisition unit 6, the output means may be equipped with a sound acquisition unit 2, a video acquisition unit 3, and a gaze acquisition unit 6. In this case, if the video acquired by the video acquisition unit and the ambient sound acquired by the sound acquisition unit contain object information of the same object, and this object is located in the direction of the passenger's line of sight, the notification information may not be output.

[0055] Furthermore, in the first and second embodiments, the output unit 5 is composed of a sound generation unit and a display unit, and is configured to output audible notification information and visual notification information. However, the output unit may output notification information as a signal to an external device. That is, the output device may not have a sound generation unit or a display unit, but may transmit signals to an external sound generation unit or display unit to generate an alarm sound or alarm display.

[0056] Furthermore, while the best configurations and methods for carrying out the present invention are disclosed in the above description, the present invention is not limited thereto. That is, although the present invention is particularly illustrated and described with respect to specific embodiments, those skilled in the art can make various modifications to the embodiments described above in terms of shape, material, quantity, and other detailed configurations without departing from the scope of the technical idea and objectives of the present invention. Therefore, the descriptions of shapes, materials, etc. disclosed above are illustrative to facilitate understanding of the present invention and do not limit the present invention. Accordingly, descriptions of components with some or all of these limitations removed are included in the present invention. [Explanation of symbols]

[0057] 1A output device 2 Sound acquisition section 3. Video Acquisition Unit 4. Control Unit 5 Output section

Claims

1. A sound acquisition unit that acquires ambient sounds from outside the vehicle, A video acquisition unit that acquires images from outside the vehicle, The vehicle includes an output unit that outputs notification information for notification inside the vehicle when it is determined that object information is included in at least one of the acquired ambient sound and video, based on the ambient sound and video, in relation to the object information. The output unit is characterized in that it does not output the notification information if the video and the ambient sound contain object information from the same object.

2. The output device according to claim 1, characterized in that the output unit outputs the notification information using the same recognition method as the ambient sound or video containing the object information.

3. The output device according to claim 2, characterized in that, when the ambient sound includes the object information, the output unit outputs the object information as notification information into the vehicle interior.

4. The output device according to claim 1, characterized in that the output unit outputs the notification information using a recognition method different from that of the ambient sound or video containing the object information.

5. A sound acquisition unit that acquires ambient sounds from outside the vehicle, A video acquisition unit that acquires images from outside the vehicle, The vehicle includes an output unit that outputs notification information for notification inside the vehicle when it is determined that object information is included in at least one of the acquired ambient sound and video, based on the ambient sound and video, in relation to the object information. The output unit is characterized in that the video and ambient sound each contain only one piece of object information, and does not output the notification information if these objects are the same.

Images