Notification control device and notification method

Abstract

To notify that a vehicle door may open during travelling when passing alongside a stationary vehicle.SOLUTION: A control unit 20 includes: an image acquisition unit 21 that acquires videos obtained by capturing the front of a moving vehicle; a recognition unit 23 that recognizes door opening actions of vehicle occupants inside a front vehicle located ahead of the moving vehicle, based on videos obtained by the image acquisition unit 21; a determination unit 25 that determines whether door opening actions of the vehicle occupants inside the front vehicle have been recognized, based on the recognition result from the recognition unit 23; and a notification control unit 29 that issues a notification when the determination unit 25 determines that the door opening actions of the vehicle occupants inside the front vehicle has been recognized.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a notification control device and a notification method. [Background technology]

[0002] There is known a technique that uses vehicle-to-vehicle communication to inform a driver of a parked vehicle whether or not the driver should pay special attention to the opening of the door of the parked vehicle when passing the parked vehicle (see, for example, Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2006-268081 Summary of the Invention [Problem to be solved by the invention]

[0004] When a moving object such as a bicycle or a kick scooter passes by the side of a stopped vehicle, it is difficult for a passenger on the moving object to detect that the vehicle door is opening. When a moving object such as a bicycle or a kick scooter passes by the side of a stopped vehicle, it is desirable to be notified that the vehicle door is opening.

[0005] The present disclosure has been made in view of the above, and aims to provide a mobile object that notifies a vehicle door that the vehicle will open when the mobile object passes by the side of a stopped vehicle. [Means for solving the problem]

[0006] In order to solve the above-mentioned problems and achieve the objectives, the notification control device of the present disclosure comprises an image acquisition unit that acquires an image of the area in front of a moving body, a recognition unit that recognizes, from the image acquired by the image acquisition unit, the door opening action of a vehicle occupant in a forward vehicle located in front of the moving body, a determination unit that determines whether the door opening action of the vehicle occupant in the forward vehicle has been recognized based on the recognition result of the recognition unit, and a notification control unit that issues a notification when the determination unit determines that the door opening action of the vehicle occupant in the forward vehicle has been recognized.

[0007] The notification method according to the present disclosure includes an image acquisition step of acquiring an image of the area ahead of a moving body, a recognition step of recognizing, from the image acquired by the image acquisition step, a door-opening action of a vehicle occupant in a forward vehicle located in front of the moving body, a determination step of determining, based on the recognition result of the recognition step, whether or not the door-opening action of the vehicle occupant in the forward vehicle has been recognized, and a notification control step of issuing a notification if it is determined by the determination step that the door-opening action of the vehicle occupant in the forward vehicle has been recognized. [Effects of the Invention]

[0008] According to the present disclosure, it is possible to provide an effect that when a moving body passes the side of a stopped vehicle, it is possible to notify that the vehicle door will open. [Brief explanation of the drawings]

[0009] [Figure 1] FIG. 1 is a block diagram showing an example of the configuration of a notification device according to the first embodiment. [Figure 2] FIG. 2 is a flowchart showing the flow of processing in the notification control device according to the first embodiment. [Figure 3] FIG. 3 is a block diagram showing an example of the configuration of a notification device according to the second embodiment. [Figure 4] FIG. 4 is a flowchart showing the flow of processing in the notification control device according to the second embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0010] Hereinafter, embodiments of a notification control device and a notification method according to the present disclosure will be described in detail with reference to the accompanying drawings. However, the present invention is not limited to the following embodiments.

[0011] [First embodiment] <Notification device> FIG. 1 is a block diagram showing an example of the configuration of a notification device according to an embodiment. The notification device 10 issues a notification when it is determined that a door opening action by a vehicle occupant in a vehicle ahead has been recognized. In this embodiment, the notification device 10 may be implemented as one of the functions of a so-called drive recorder for a mobile object such as a bicycle or a kick scooter (a specific small motorized bicycle). The notification device 10 may be a device mounted on the mobile object, or may be a portable device that can be used on the mobile object. The notification device 10 includes a camera 11, an operation data storage unit 12, a notification unit 19, and a control device (notification control device) 20.

[0012] Camera 11 is a camera that captures images ahead of the moving object. Camera 11 constantly captures images while the moving object is traveling. Camera 11 outputs the captured image data to image acquisition unit 21 of control device 20. The image data is a moving image composed of, for example, 30 frames of images per second.

[0013] The action data storage unit 12 stores dictionary data for recognizing, from video, the door-opening action of a vehicle occupant in a vehicle ahead of the moving body. The action data storage unit 12 is, for example, a semiconductor memory element such as a ROM (Read Only Memory) or a flash memory, or a recording unit such as a memory card. Alternatively, the action data storage unit 12 may be an external recording unit wirelessly connected via a communication device (not shown).

[0014] The action data includes data recognizing an opening action of a driver's door by a driver of a forward vehicle. The action data includes data recognizing an opening action of a passenger seat occupant opening the passenger seat door. The action data includes data recognizing an opening action of a rear seat occupant opening the left rear door. The action data includes data recognizing an opening action of a rear seat occupant opening the right rear door.

[0015] The motion data includes, for example, data recognizing a motion of the driver or a vehicle occupant in the front passenger seat of the vehicle looking toward a side mirror. The motion data includes, for example, data recognizing a motion of the driver or a vehicle occupant in the front passenger seat of the vehicle looking toward a side mirror. The motion data includes, for example, data recognizing a movement in the door opening direction corresponding to the driver or a vehicle occupant in the front passenger seat of the vehicle.

[0016] The operation data includes, for example, data for recognizing the movement of the door in the opening direction corresponding to a vehicle occupant in the rear seat. In the case of a vehicle with a sliding door, the data for recognizing the movement of the door in the opening direction is data for recognizing the protrusion of the door from the side of the vehicle.

[0017] The notification unit 19 issues a notification to a vehicle occupant of the vehicle or a vehicle occupant of a preceding vehicle. The notification unit 19 includes a first notification unit 191 and a second notification unit 192. The notification unit 19 issues a notification based on a control signal from the notification control unit 29.

[0018] The first notification unit 191 is a notification device that notifies a passenger of the moving body. The first notification unit 191 includes at least one of a display unit that displays a notification image, an audio output unit that outputs a notification sound, and a notification lamp.

[0019] The function of first notification unit 191 as a display unit will be described. As an example, first notification unit 191 is a display device arranged around the handle of the vehicle facing the vehicle occupant. Based on a video signal output from notification control unit 29, first notification unit 191 displays a video notifying that the door of the vehicle ahead is about to be opened.

[0020] The function of first notification unit 191 as an audio output unit will be described. As an example, first notification unit 191 is an audio output unit arranged around the handle of the vehicle toward a vehicle occupant. When first notification unit 191 is an audio output unit, it outputs an audio signal output from notification control unit 29 to notify that the door of the vehicle ahead is about to be opened.

[0021] The function of first notification unit 191 as a notification lamp will be described. As an example, first notification unit 191 is a lamp arranged around the handle of the vehicle facing the vehicle occupant. When first notification unit 191 is a notification lamp, it notifies the driver that the door of the vehicle ahead is about to be opened by lighting up the lamp based on a control signal output from notification control unit 29.

[0022] The second notification unit 192 is a notification device that notifies the vehicle occupants of the vehicle ahead. The second notification unit 192 includes at least one of a speaker that outputs a notification sound and a notification lamp.

[0023] The function of the second notification unit 192 as a speaker will be described. As an example, the second notification unit 192 is a speaker that outputs sound forward. When the second notification unit 192 is a speaker, it outputs sound that notifies the vehicle ahead that a moving object is approaching from behind the vehicle ahead, based on the audio signal output from the notification control unit 29.

[0024] The function of second notification unit 192 as a notification lamp will be described. As an example, second notification unit 192 is a lamp arranged facing forward. When second notification unit 192 is a notification lamp, it notifies the vehicle ahead that a moving object is approaching from behind the vehicle ahead by lighting up the lamp based on a control signal output from notification control unit 29.

[0025] <Notification Control Device> The control device 20 issues a notification when it is determined that it has recognized the door opening action of a vehicle occupant in the vehicle ahead. The control device 20 is an arithmetic processing device (control device) configured with, for example, a CPU (Central Processing Unit) and the like. The control device 20 loads a stored program into memory and executes instructions included in the program. The control device 20 includes an internal memory (not shown), which is used for temporary storage of data in the control device 20, etc. The control device 20 includes an image acquisition unit 21, a recognition unit 23, a determination unit 25, and a notification control unit 29.

[0026] The image acquisition unit 21 acquires an image of the area ahead of the moving object. More specifically, the image acquisition unit 21 acquires an image output by the camera 11.

[0027] The recognition unit 23 uses the action data storage unit 12 to recognize, from the video acquired by the video acquisition unit 21, the door opening action of a vehicle occupant who is a passenger in a front vehicle located in front of the moving object.

[0028] The recognition unit 23 uses the action data storage unit 12 to recognize, from the video, at least one of the action of the vehicle occupant reflected in the side mirror of the vehicle ahead and the movement of the door of the vehicle ahead in the opening direction.

[0029] The recognition unit 23 recognizes from the video, for example, an action of the driver of the vehicle ahead or a vehicle occupant sitting in the passenger seat directing their gaze toward a side mirror. The recognition unit 23 recognizes from the video, for example, an action of the driver of the vehicle ahead or a vehicle occupant sitting in the passenger seat directing their face toward a side mirror. The recognition unit 23 recognizes from the video, for example, a movement in the door opening direction corresponding to the driver of the vehicle ahead or a vehicle occupant sitting in the passenger seat.

[0030] The recognition unit 23 recognizes, from the video, for example, a movement in the door opening direction corresponding to a vehicle occupant positioned in the rear seat.

[0031] The determination unit 25 determines whether or not the door opening action of a vehicle occupant in the vehicle ahead has been recognized based on the recognition result of the recognition unit 23. When the recognition unit 23 recognizes an action included in the action data stored in the action data storage unit 12 from the captured video acquired by the video acquisition unit 21, the determination unit 25 determines that the door opening action of a vehicle occupant in the vehicle ahead has been recognized.

[0032] The notification control unit 29 issues a notification when the determination unit 25 determines that the door opening operation of a vehicle occupant in the vehicle ahead has been recognized.

[0033] When the determination unit 25 determines that the door opening action of a vehicle occupant in the forward vehicle has been recognized, the notification control unit 29 notifies the vehicle occupant who is a passenger of the moving body and the forward vehicle. When the determination unit 25 determines that the door opening action of a vehicle occupant in the forward vehicle has been recognized, the notification control unit 29 controls to notify the vehicle occupant who is a passenger of the moving body that the door of the forward vehicle will be opened. When the determination unit 25 determines that the door opening action of a vehicle occupant in the forward vehicle has been recognized, the notification control unit 29 controls to notify the forward vehicle that a moving body is approaching from behind the forward vehicle.

[0034] <Processing in the notification control device> Next, information processing in the control device 20 will be described with reference to Fig. 2. Fig. 2 is a flowchart showing the flow of processing in the control device according to the first embodiment. While the moving object is traveling, the notification device 10 is activated and the camera 11 constantly captures images of the area ahead. The control device 20 acquires the images captured by the camera 11 using the image acquisition unit 21.

[0035] The control device 20 starts recognizing the door opening operation of the vehicle ahead (step S101). The control device 20 recognizes, from the video, at least one of the operation of a vehicle occupant reflected in the side mirror of the vehicle ahead and the movement of the door of the vehicle ahead in the opening direction by the recognition unit 23. The control device 20 proceeds to step S102.

[0036] The control device 20 determines whether or not a door-opening operation has been performed (step S102). More specifically, the control device 20 determines whether or not a door-opening operation has been recognized based on the recognition result of the recognition unit 23. If the control device 20 determines that a door-opening operation has been performed (Yes in step S102), the process proceeds to step S103. If the control device 20 does not determine that a door-opening operation has been performed (No in step S102), the process proceeds to step S104.

[0037] When it is determined that a door opening operation has occurred (Yes in step S102), the control device 20 notifies the vehicle occupant that the door of the vehicle ahead will be opened (step S103). More specifically, the control device 20 notifies the vehicle occupant that the door of the vehicle ahead will be opened by the notification control unit 29 via the first notification unit 191. The control device 20 notifies the vehicle occupant of the approach of the vehicle by the notification control unit 29 via the second notification unit 192. The control device 20 proceeds to step S104.

[0038] The control device 20 determines whether to end this process (step S104). More specifically, the control device 20 determines to end this process when, for example, a user operation to end this process is received, or when the traveling of the mobile object is finished. When the control device 20 determines to end this process (Yes in step S104), it ends the processing of this flowchart. When the control device 20 does not determine to end this process (No in step S104), it executes the processing of step S102 again.

[0039] <Effects> As described above, in this embodiment, when it is determined from the video that a door opening action of a vehicle occupant in a vehicle ahead has been recognized, a notification can be made. According to this embodiment, when a moving object passes by the side of a stopped vehicle, a notification can be made that the door of the vehicle ahead is opening.

[0040] In this embodiment, when it is determined that a door opening action by a vehicle occupant in a forward vehicle has been recognized, a notification can be sent to the vehicle occupant and the forward vehicle.

[0041] In this embodiment, the movement of a vehicle occupant reflected in the side mirror of the vehicle ahead and / or the movement of the door of the vehicle ahead in the opening direction can be recognized from the video. According to this embodiment, the opening movement of the door of the vehicle ahead can be appropriately recognized.

[0042] [Second embodiment] A notification device 10A according to the second embodiment will be described using Figures 3 and 4. Figure 3 is a block diagram showing an example of the configuration of the notification device according to the second embodiment. Figure 4 is a flowchart showing the processing flow in the notification control device according to the second embodiment. In the following description, components similar to those in the notification device 10 of the first embodiment are given the same or corresponding reference numerals, and detailed description thereof will be omitted. This also applies to subsequent embodiments. The notification device 10A differs from the first embodiment in that the notification device 10A includes a face recognition data storage unit 14A, the control device 20A includes a face recognition unit 24A and a face determination unit 26A, and in the processing in the control device 20A.

[0043] The face recognition data storage unit 14A stores dictionary data for recognizing the faces of vehicle occupants reflected in the side mirrors of the vehicle ahead from video. The face recognition data storage unit 14A is, for example, a semiconductor memory element such as a ROM or flash memory, or a recording unit such as a memory card. Alternatively, it may be an external recording unit wirelessly connected via a communication device (not shown).

[0044] The first notification unit 191A notifies a passenger of the moving object to guide the moving position.

[0045] The face recognition unit 24A uses the face recognition data storage unit 14A to recognize the faces of vehicle occupants reflected in the side mirrors of the vehicle ahead from the video captured by the video acquisition unit 21.

[0046] The face determination unit 26A determines whether or not the face of a vehicle occupant in the forward vehicle has been recognized based on the recognition result of the face recognition unit 24A.

[0047] When the face determination unit 26A determines that it has not recognized the face of the vehicle occupant in the forward vehicle, the notification control unit 29A notifies the vehicle occupant who is a passenger of the moving body to guide the vehicle occupant to change the traveling position so that the face of the vehicle occupant in the forward vehicle is reflected in the video. When the face determination unit 26A determines that it has not recognized the face of the vehicle occupant in the forward vehicle, the notification control unit 29A notifies the vehicle occupant who is a passenger of the moving body to guide the vehicle occupant to, for example, shift the traveling position to the left or right.

[0048] <Processing in the notification control device> Next, the information processing in the control device 20A will be described with reference to Fig. 4. The processing in steps S114 to S116 in Fig. 4 is the same as the processing in steps S102 to S104 in the flowchart shown in Fig. 2.

[0049] The control device 20A starts recognizing the door opening operation of the vehicle ahead (step S111). More specifically, the control device 20A starts recognizing the face of the vehicle occupant reflected in the side mirror of the vehicle ahead from the video using the face recognition unit 24A. The control device 20A starts recognizing at least one of the action of the vehicle occupant reflected in the side mirror of the vehicle ahead from the video and the movement of the door of the vehicle ahead in the opening direction using the recognition unit 23. The control device 20A proceeds to step S112.

[0050] The control device 20A determines whether the face of the vehicle occupant is reflected in the side mirror (step S112). More specifically, the control device 20A determines whether the face of the vehicle occupant reflected in the side mirror has been recognized from the video based on the recognition result of the face recognition unit 24A. If the control device 20A determines that the face of the vehicle occupant is reflected in the side mirror (Yes in step S112), the control device 20A proceeds to step S114. If the control device 20A does not determine that the face of the vehicle occupant is reflected in the side mirror (No in step S112), the control device 20A proceeds to step S113.

[0051] <Effects> As described above, in this embodiment, when it is determined from the video that the face of the vehicle occupant reflected in the side mirror of the vehicle ahead is not recognized, a notification can be given to the vehicle occupant to guide the vehicle occupant to a traveling position so that the face of the vehicle occupant in the vehicle ahead is reflected in the video. According to this embodiment, the opening operation of the door of the vehicle ahead can be appropriately recognized.

[0052] The notification control device according to the present invention may be implemented in various different forms other than the above-described embodiment.

[0053] The components of the illustrated notification control device are conceptual and functional, and do not necessarily have to be physically configured as shown. In other words, the specific form of each device is not limited to that shown, and all or part of each device may be functionally or physically distributed or integrated in any unit depending on the processing load and usage status of each device.

[0054] The configuration of the notification control device is realized, for example, as software, by a program loaded into memory. In the above embodiment, the configuration is described as a functional block realized by the cooperation of these hardware and software. In other words, these functional blocks can be realized in various forms using only hardware, only software, or a combination of both.

[0055] The above-described components include those that can be easily imagined by a person skilled in the art and those that are substantially the same. Furthermore, the above-described configurations can be appropriately combined. Furthermore, various omissions, substitutions, or modifications of the configurations are possible within the scope of the gist of the present invention.

[0056] [Variations] In the above, when it is determined that the notification device 10 has recognized the door opening action of a vehicle occupant in the vehicle ahead, it may notify the user and may also be equipped with a driving control unit that outputs a control signal to the moving body to slow down or stop the moving body, for example by activating the brakes.

[0057] In the above, the recognition unit 23 may further recognize the opening operation of the door of the vehicle ahead from a sound picked up by a microphone (not shown). The recognition unit 23 may store operation sound data indicating operation sounds such as the door unlocking sound and the door knob unlocking sound in the operation data storage unit 12, and may recognize that the door of the vehicle ahead is opening by using the operation sound data.

[0058] In the above, the recognition unit 23 may further determine whether the side mirrors are folded or not, and if the side mirrors are not folded, start information processing in the control device 20. This is because if the side mirrors are folded, the passengers have disembarked and the vehicle is in a parked state, so there is no need to worry about the doors opening. [Explanation of symbols]

[0059] 10 Notification device 11 Camera 12 Operation data storage unit 19 Notification Department 191 First Notification Department 192 Second Notification Department 20 Control device (notification control device) 21 Video acquisition unit 23 Recognition part 25 Judgment section 29 Notification control section

Claims

1. an image acquisition unit that acquires an image of a scene ahead of the moving object; a recognition unit that recognizes, from the image captured by the image acquisition unit, a door opening operation of a vehicle occupant who is an occupant in a forward vehicle located in front of the moving object; a determination unit that determines whether or not a door opening operation by the vehicle occupant in the forward vehicle has been recognized based on the recognition result of the recognition unit; a notification control unit that issues a notification when the determination unit determines that a door opening operation by the vehicle occupant in the forward vehicle has been recognized; A notification control device comprising:

2. a face recognition unit that recognizes a face of the vehicle occupant reflected in a side mirror of the forward vehicle from the image captured by the image acquisition unit; a face determination unit that determines whether or not the face of the vehicle occupant in the forward vehicle has been recognized based on the recognition result of the face recognition unit; Equipped with When it is determined by the face determination unit that the face of the vehicle occupant in the forward vehicle has not been recognized, the notification control unit notifies the vehicle occupant who is a passenger in the moving body to guide the vehicle occupant to a traveling position so that the face of the vehicle occupant in the forward vehicle is reflected in the image. The notification control device according to claim 1 .

3. When the determination unit determines that the door opening operation of the vehicle occupant in the forward vehicle has been recognized, the notification control unit notifies the vehicle occupant who is an occupant of the moving body and the forward vehicle. The notification control device according to claim 1 or 2.

4. the recognition unit recognizes, from the image, at least one of a movement of the vehicle occupant reflected in a side mirror of the forward vehicle and a movement of a door of the forward vehicle in an opening direction. The notification control device according to claim 1 or 2.

5. an image acquisition step of acquiring an image of a scene ahead of the moving object; a recognition step of recognizing, from the image captured by the image acquisition step, a door opening action of a vehicle occupant who is an occupant in a front vehicle located in front of the moving body; a determination step of determining whether or not a door opening operation of the vehicle occupant in the forward vehicle has been recognized based on the recognition result of the recognition step; a notification control step of issuing a notification when it is determined in the determination step that a door opening operation of the vehicle occupant in the forward vehicle has been recognized; A notification method executed by a notification control device, including:

Images