Vehicle occupant abandonment alarm device

Abstract

To provide an occupant-left-in-vehicle-alarming device which improves alarming of one or more occupants being left behind in a vehicle.SOLUTION: Provided is an occupant-left-in-vehicle-alarming device 20 having an ECU (alarm control unit) 43 that outputs a left-behind alarm when a child or children including an infant are left behind in a vehicle, said device comprising an exterior camera 26 installed so as to be capable of capturing an image of one or more occupants present outside the vehicle, and an ECU (determination unit) 43 for determining whether or not any child or children may be left behind in the vehicle. The determination unit determines whether or not any child or children may be left behind in the vehicle on the basis of change in existence of one or more persons or objects outside the vehicle in the captured image of the exterior camera 26.SELECTED DRAWING: Figure 3

Description

[Technical Field]

[0001] The present invention relates to an occupant abandonment warning device for a vehicle. [Background technology]

[0002] In vehicles such as automobiles, there is a possibility that children, including infants, may be left behind in the vehicle. As a countermeasure for this, it is conceivable to record the opening and closing of the vehicle doors, and if the door opening and closing history is recorded, to output an abandoned warning when the occupant gets off the vehicle. In Patent Document 1, an image of the interior of the vehicle before the door is opened is compared with an image of the interior of the vehicle after the door is closed, and an occupant is detected based on the difference between the images of the interior of the vehicle. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2009-107527 [Patent Document 2] Japanese Patent Application Laid-Open No. 2018-142130 Summary of the Invention [Problem to be solved by the invention]

[0004] However, as described in Patent Document 2, luggage may be carried on a vehicle. Furthermore, if an abandoned cargo alarm is output based only on the door opening / closing history as described above, an abandoned cargo alarm will be output even when the door is opened and closed and cargo is placed on the vehicle. While this is considered meaningful in the sense that it issues a warning about children being left behind in consideration of their safety when there is a possibility that a child, including an infant, may be left behind in the vehicle, some occupants of the vehicle may perceive it as an excessive warning.

[0005] Furthermore, children, including infants, may be small in physique, and when such a small child gets in the car, the child may be in a blind spot behind or between the seats in the image of the interior of the car after the doors are closed, and may not be able to be properly photographed. When detecting that a child, including an infant, has been left in a vehicle based on an image captured inside the car, as in Patent Document 1, there are likely to be cases where it is not possible to properly detect that a child, including an infant, has been left in the vehicle.

[0006] Thus, there is a need to improve warnings regarding abandoned occupants in vehicles. [Means for solving the problem]

[0007] One embodiment of the present invention provides an occupant abandonment warning device for a vehicle, which includes an alarm control unit that outputs an abandonment warning when a child, including an infant, is left in the vehicle, and further includes an exterior camera that is capable of capturing images of occupants outside the vehicle, and a judgment unit that determines the possibility that a child has been left in the vehicle, and the judgment unit determines the possibility that a child has been left in the vehicle based on changes in the presence or absence of people or objects outside the vehicle in the image captured by the exterior camera. [Effects of the Invention]

[0008] In the present invention, rather than estimating whether a passenger is getting on or off the vehicle based on the door opening / closing history or the like, an exterior camera captures images of passengers outside the vehicle to get on or off the vehicle, and the possibility that a child, including an infant, has been left in the vehicle is determined based on a change in the presence or absence of a person or object outside the vehicle in the image captured by the exterior camera. For example, a person who gets on the vehicle may change from being present in the image captured by the exterior camera to not being present in the image captured by the exterior camera. Conversely, a person who gets off the vehicle may change from not being present in the image captured by the exterior camera to being present in the image captured by the exterior camera. In the present invention, the possibility that a child who has not gotten off the vehicle has been left in the vehicle is determined based on such a change in the presence or absence of a person or object outside the vehicle in the image captured by the exterior camera. Then, in the present invention, if a child has been left in the vehicle, a child leaving warning can be output. In this way, in the present invention, if there is a possibility that a child has been left in the vehicle, a child abandonment alarm can be output. On the other hand, if an occupant opens and closes the door and places something other than a child, such as luggage, in the vehicle, it is possible to prevent this from being determined as a child abandonment. In this case, the present invention can suppress the output of the child abandonment alarm. Furthermore, because the present invention uses images taken outside the vehicle by an exterior camera, it is unlikely that the image of the child will be continuously blocked by a seat or the like, as occurs with images taken inside the vehicle, and the exterior camera can capture images of small children, including infants. Even when small children, including infants, get in and out of the vehicle, the exterior camera can reliably capture images of them and output a warning that they have been left behind. In this way, the present invention directly captures images of people actually getting in and out of a vehicle and determines whether a change in presence or absence is detected based on the images, so that, compared to determinations based solely on the door opening and closing history, it is possible to distinguish between getting in and out of or leaving behind a child and getting in and out of or leaving behind other items, such as luggage, and improve the accuracy of the child leaving determination.Furthermore, by obtaining the accuracy of the child leaving determination, the present invention can also improve the output of a child leaving warning so that it is limited to cases when it is generally necessary. [Brief explanation of the drawings]

[0009] [Figure 1] FIG. 1 is a schematic plan view of an automobile to which an occupant abandonment warning device according to a first embodiment of the present invention is applied. [Figure 2] FIG. 2 is an explanatory diagram showing a state in which a plurality of passengers are riding in the automobile of FIG. [Figure 3] FIG. 3 is an explanatory diagram of the occupant abandonment alarm device of the automobile of FIG. [Figure 4] FIG. 4 is an explanatory diagram of an example of a plurality of exterior cameras provided in the automobile of FIG. 1 for capturing images of the outside of the automobile. [Figure 5] FIG. 5 is a flowchart of the boarding / alighting monitoring control by the occupant abandonment alarm device. [Figure 6]FIG. 6 is a flowchart of a child leaving-behind determination control by the occupant leaving-behind alarm device according to the first embodiment of the present invention. [Figure 7] FIG. 7 is a flowchart showing abandoned passenger alarm output control by the abandoned passenger alarm device. [Figure 8] FIG. 8 is an explanatory diagram of an example of the angle of view control in a planar direction of the vehicle exterior right rear camera serving as the vehicle exterior camera provided on the right front door in FIG. [Figure 9] FIG. 9 is an explanatory diagram of an example of vertical angle control of the vehicle exterior left rear camera as the vehicle exterior camera provided on the left front door in FIG. [Figure 10] FIG. 10 is an explanatory diagram of an example of vertical angle control of the vehicle exterior left rear camera as the vehicle exterior camera provided on the left front door in FIG. [Figure 11] FIG. 11 is an explanatory diagram of an example of vertical angle control of the vehicle exterior rear camera serving as the vehicle exterior camera provided on the rear gate in FIG. [Figure 12] FIG. 12 is an explanatory diagram of a method for determining the physique of an occupant when another exterior rear camera is provided below the rear gate. [Figure 13] FIG. 13 is a flowchart of a child leaving-behind determination control by an occupant leaving-behind alarm device according to the second embodiment of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0010] Hereinafter, an embodiment of the present invention will be described with reference to the drawings.

[0011] [First embodiment] FIG. 1 is a schematic plan view of an automobile 1 to which an occupant abandonment warning device 20 according to a first embodiment of the present invention is applied. An automobile 1 in FIG. 1 is an example of a vehicle. 1 has a vehicle body 2 and a passenger compartment 3. The passenger compartment 3 is provided with a driver's seat 4, a passenger seat 5, and a rear seat 6. The rear seat 6 is fitted with a child seat 7 for carrying an infant. The vehicle body 2 is also provided with a right front door 8 that opens and closes when getting in and out of the vehicle so that the driver can sit in the driver's seat 4, a left front door 9 that opens and closes when getting in and out of the vehicle so that a passenger can sit in the passenger seat 5, a right rear door 10 and a left rear door 11 that open and close when getting in and out of the vehicle so that a child or other person can sit in the rear seat 6, and a rear gate 12 that opens and closes when loading luggage or the like into the vehicle interior 3. The multiple doors 8 to 12, including the rear gate 12, are indicated by dashed lines in the drawing along with the ranges within which they can be opened and closed. FIG. 1 also shows a number of occupants, such as adults, infants, and children, who are about to get into the automobile 1. The driver or a passenger opens one of the doors mentioned above, gets into the automobile 1, and closes the door. Additionally, an adult such as a driver or a passenger can open, for example, the right rear door 10, let a child, including an infant, into the automobile 1, and then close the door.

[0012] FIG. 2 is an explanatory diagram showing a state in which a plurality of passengers are riding in the automobile 1 of FIG. An adult driver sits in the driver's seat 4. The child is riding in rear seat 6. The infant is placed in a child seat 7 attached to the rear seat 6. In this way, the automobile 1 can be driven with multiple passengers on board. After driving, the adult driver opens the right front door 8, gets out of the vehicle, and closes the right front door 8. If the driver gets out of the vehicle alone, children, including infants, will be left behind in the vehicle 1.

[0013] As a countermeasure against such abandonment, it is conceivable to record the opening and closing of the doors of the automobile 1, and if the door opening and closing history is recorded, to output an abandonment alarm when the person gets out of the automobile. However, automobile 1 may carry luggage, and if an abandoned vehicle alarm is output based solely on the door opening / closing history, an abandoned vehicle alarm will be output even if luggage is loaded by opening and closing a door such as rear gate 12. Although this is considered meaningful in the sense that it issues a warning about children being left behind in consideration of their safety when there is a possibility that a child, including an infant, may be left behind while riding in the vehicle, it may be perceived as an excessive warning by the occupants of the vehicle 1. Therefore, instead of recording the door opening and closing history of the automobile 1, it is conceivable to use the in-vehicle camera 22 to capture images of the interior of the vehicle before and after the door is opened and closed, and compare the captured images of the interior of the vehicle before and after the door is opened and closed to detect children, including infants, who have been left behind and output an alarm about their leaving behind. Furthermore, children, including infants, may be small in size. When such a small child gets in the vehicle, the child may be in a blind spot behind or between the seats in the image of the interior of the vehicle after the doors are closed, and may not be properly captured. When detecting that a child, including an infant, has been left in a vehicle based on the captured image of the interior of the vehicle, there may be cases where it is not possible to properly detect that a child, including an infant, has been left. Thus, there is a need to improve the warning system for warning occupants left behind in the automobile 1.

[0014] Fig. 3 is an explanatory diagram of the occupant abandoned warning device 20 of the automobile 1 of Fig. 1. The occupant abandoned warning device 20 may be provided in the automobile 1 as a part of the control system of the automobile 1. The occupant abandonment warning device 20 in Figure 3 includes a millimeter-wave in-vehicle sensor 21, an in-vehicle camera 22, a door opening / closing sensor 23, a door lock sensor 24, an ignition switch (IG switch) 25, an outside-vehicle camera 26, an actuator 27, an occupant monitoring device 28, a meter panel 29, a horn 30, a wireless communication device 31, and an in-vehicle network 32 to which these are connected.

[0015] The in-vehicle network 32 may be a wired communication network for the automobile 1 that complies with, for example, a Controller Area Network (CAN) or a Local Interconnect Network (LIN). The in-vehicle network 32 may be a communication network such as a LAN, or a combination of these. A wireless communication network may be included as part of the in-vehicle network 32. The above-mentioned devices connected to the in-vehicle network 32 can transmit and receive detection data and control data to and from each other via the in-vehicle network 32.

[0016] The millimeter-wave interior sensor 21 is mounted, for example, in the front center portion of the roof of the vehicle interior 3, as shown in FIG. 1 . The millimeter-wave interior sensor 21 outputs millimeter waves of a predetermined frequency, for example, in the range of 20 to 80 GHz, toward the vehicle interior 3, and receives the reflected waves. The period from the output of the millimeter waves to the input of the reflected waves differs depending on whether an occupant is present in the vehicle interior 3 or not. Furthermore, by using high-frequency millimeter waves, for example, 40 GHz or higher, as detection waves, it is possible to detect the size (physique) and chest movement due to breathing of occupants, including children, including small infants, in the vehicle interior 3. For this reason, the millimeter-wave interior sensor 21 preferably uses millimeter waves of 60 GHz or higher, for example, 75 GHz, for occupant monitoring under normal conditions. In contrast, by using low-frequency millimeter waves, for example, 20 GHz, as detection waves, the sensor is less susceptible to shielding by seats or metal partitions in the vehicle interior 3. To reduce blind spots inside the vehicle, it is preferable to use low-frequency millimeter waves, for example, 20 GHz. As a result, the millimeter-wave interior sensor 21 outputs millimeter waves toward the passengers inside the automobile 1 and detects the millimeter waves reflected by the passengers inside the automobile 1. The millimeter-wave interior sensor 21, as an interior sensor, can detect the passengers inside the automobile 1.

[0017] The interior camera 22 is an example of an interior sensor and may be, for example, a wide-angle camera. In this case, the interior camera 22 can capture an overall image of the vehicle interior 3 from the front, capturing images of multiple occupants such as the driver and passengers. The interior camera 22 may be provided, for example, in the front center portion of the roof of the vehicle interior 3 as shown in FIG. 1 , or in a dashboard or center console (not shown) provided in front of the front seats and passenger seat 5. This allows the in-vehicle camera 22 to detect passengers inside the automobile 1 as an in-vehicle sensor.

[0018] The door opening / closing sensor 23 detects the opening and closing of multiple doors, including the rear gate 12, provided on the automobile 1. The door opening / closing sensor 23 may be provided for each door provided on the automobile 1. The door opening / closing sensor 23 may detect and output a signal that a closed door has been opened and that an open door has been closed. The door opening / closing sensor 23 may also be configured to detect the amount of opening and closing of the door, such as the opening and closing angle. The door opening / closing sensor 23 is a vehicle sensor that detects the opening and closing of the doors of the automobile 1, and can detect the movement of the automobile 1 when an occupant gets in or out of the automobile. The door opening / closing sensor 23 can detect multiple movements of the automobile 1 during one entry or exit of an occupant.

[0019] The door lock sensor 24 detects the locking and unlocking of a door lock (not shown) provided in the automobile 1. A door lock may be provided for each door provided in the automobile 1, or one door lock may be provided for all of the doors provided in the automobile 1. The door lock sensor 24 may detect and output a signal when a locked door lock is unlocked, or when an unlocked door lock is locked. The door lock sensor 24 is a vehicle sensor that detects the locking and unlocking of the doors of the automobile 1, and can detect the movement of the automobile 1 when an occupant gets in or out of the automobile. The door lock sensor 24 can detect multiple movements of the automobile 1 during a single entry or exit by an occupant.

[0020] The ignition switch 25 is a switch for starting and stopping the engine or motor, which is the power source for driving the automobile 1. When the power source is started, the automobile 1 is put into a state where it can run. When the power source is stopped, the automobile 1 is put into a state where it cannot run on its own. When operated by a passenger such as a driver, the ignition switch 25 may output a start signal or a stop signal in response to the operation.

[0021] The exterior camera 26 is a camera that captures images outside the automobile 1. Basically, the automobile 1 may be provided with multiple exterior cameras 26 as shown in FIG. 4, which will be described later, in order to capture images of the environment surrounding the automobile 1 while it is traveling. By capturing images of the surroundings of the automobile 1 at 360 degrees using the multiple exterior cameras 26, it is possible to obtain information about the surroundings of the automobile 1 while it is traveling. The exterior camera 26 can capture images of passengers outside the automobile 1.

[0022] The actuator 27 is for adjusting the orientation of the exterior camera 26. When multiple exterior cameras 26 are provided on the automobile 1, multiple actuators 27 may be provided so as to correspond one-to-one to the multiple exterior cameras 26. An actuator 27 is provided on the vehicle 1 to enable the exterior camera 26 to be turned.

[0023] The meter panel 29 is provided in front of the driver's seat 4 in the passenger compartment 3 so that it can be easily seen by the driver while driving the automobile 1. The meter panel 29 may be, for example, a liquid crystal panel. The meter panel 29 may switch between displaying a screen for when the vehicle is moving and a screen for when the vehicle is parked or stopped. The screen for when the vehicle is parked may include, for example, a warning screen about leaving passengers behind. The meter panel 29 may serve as an output unit provided in the automobile 1 and output an alarm.

[0024] The horn 30 may output a warning sound or alarm to other automobiles 1 or people around the automobile 1. The horn 30 may be an output unit provided in the automobile 1 and may output a warning.

[0025] The wireless communication device 31 establishes a wireless communication path with an occupant terminal 60 carried by an occupant such as a driver using the automobile 1, and transmits and receives communication data to and from the occupant terminal 60. The wireless communication device 31 may output an alert as status information of the automobile 1 to the occupant terminal 60. When the occupant terminal 60 receives an alert, it may output a sound or a screen image to the occupant. The wireless communication device 31 may output a warning to the passenger terminal 60 of the passenger who has exited the automobile 1.

[0026] The occupant monitoring device 28 monitors occupants aboard the automobile 1. The occupant monitoring device 28 has an input / output unit 41, a timer 42, an ECU 43, a memory 44, and an internal bus 45 to which these are connected.

[0027] The input / output unit 41 is connected to the in-vehicle network 32. The input / output unit 41 transmits and receives data to and from each part of the automobile 1 through the in-vehicle network 32.

[0028] The timer 42 measures time and hour. For example, the timer 42 may measure the timing at which millimeter waves for detection are output and the time elapsed since the millimeter waves were output.

[0029] The memory 44 stores programs and data executed by the ECU 43. The memory 44 may be composed of a volatile memory such as a RAM and a non-volatile memory such as an SSD or an HDD. The memory 44 in FIG. 3 shows a child getting on flag 51 and a child getting off flag 52 for recording the history of the getting on and off states of children as passengers. The child riding flag 51 may record a value that is counted up when it is determined that a child is riding in the automobile 1 based on an image captured by the exterior camera 26, for example, based on detection of the door opening or closing. The child getting off flag 52 may record a value that is counted up when it is determined that a child has gotten off the automobile 1 based on the image captured by the exterior camera 26, for example, based on detection of the door opening or closing. In this case, the value of the child boarding flag 51 and the value of the child disembarking flag 52 will be the same when all children who boarded the automobile 1 have disembarked. If the value of the child boarding flag 51 and the value of the child disembarking flag 52 are different, this indicates that there are children who have not disembarked from the automobile 1. The value of the child boarding flag 51 and the value of the child disembarking flag 52 may be initialized to 0, for example, by a flag reset operation by a passenger who has confirmed that no child has been left behind. The child boarding flag 51 and the child dismounting flag 52 may be changed so that the value is counted up when it is determined that the child has dismounted, for example.

[0030] The ECU 43 may be a CPU used in a computer device, etc. The ECU 43 reads and executes a program from the memory 44. In this way, the ECU 43 functions as a control unit that controls the overall operation of the occupant monitoring device 28. The ECU 43, which serves as a control unit of the occupant monitoring device 28, normally identifies occupants in the automobile 1 and monitors their states such as drowsiness based on images captured by the in-vehicle camera 22 and movements of the occupants' chests detected by the millimeter-wave in-vehicle sensor 21. In this case, the ECU 43 may instruct the millimeter-wave in-vehicle sensor 21 to perform detection using 60 GHz millimeter waves. In this embodiment, the ECU 43 as a control unit of the occupant monitoring device 28 determines the possibility that a child, including an infant, has been left behind in the automobile 1 based on the detection of an occupant inside the vehicle by the in-vehicle camera 22 and the millimeter-wave in-vehicle sensor 21, and outputs a child abandonment warning if a child has been left behind. If a child has not been left behind, the ECU 43 does not output excessive child abandonment warnings. This allows the ECU 43 to function as a determination unit and a warning control unit. In addition to the above, the ECU 43 may, as a judgment unit, estimate and judge the possibility that a child has been left behind in the vehicle 1 based on changes in the presence or absence of people or objects outside the vehicle in images captured by the exterior camera 26.

[0031] FIG. 4 is an explanatory diagram of an example of a plurality of exterior cameras 26 provided on automobile 1 of FIG. 1 for capturing images of the outside of the vehicle. FIG. 4 shows the multiple exterior cameras 26, including an exterior front camera 71, an exterior right front camera 72, an exterior right rear camera 73, an exterior left front camera 74, an exterior left rear camera 75, and an exterior rear camera 76. The automobile 1 is provided with a plurality of exterior cameras 26 to capture images of the environment around the automobile 1 while the automobile 1 is traveling in an automatic driving or driving assistance mode, for example, and to ensure collision safety while the driver is manually driving the automobile 1. In the automobile 1, a 360-degree camera provided on the exterior of the roof of the automobile 1 may be used as the exterior camera 26.

[0032] The exterior front camera 71 is provided facing forward at the front of the automobile 1. In Fig. 4, the exterior front camera 71 is provided in the passenger compartment 3 of the automobile 1, for example, at the center of the front edge of the roof of the automobile 1. The exterior front camera 71 may capture images of the range in front of the automobile 1. The exterior right front camera 72 is provided facing forward on the right side of the automobile 1. In FIG. 4, the exterior right front camera 72 is provided, for example, at the right end of the dashboard in the passenger compartment 3 of the automobile 1. The exterior right front camera 72 may capture images of the range extending from the front right side of the automobile 1 to the right. The exterior right front camera 72 may also be provided on the right front door 8 or on the door mirror. The exterior right rear camera 73 is provided facing rearward on the right side of the automobile 1. In Fig. 4, the exterior right rear camera 73 is provided, for example, on the right front door 8 or door mirror, which is an exterior member that can be opened and closed, of the automobile 1. The exterior right rear camera 73 may capture images in the range from the right side of the automobile 1 to the rear right side. The exterior left front camera 74 is provided facing forward on the left side of the automobile 1. In FIG. 4, the exterior left front camera 74 is provided, for example, at the left end of the dashboard in the passenger compartment 3 of the automobile 1. The exterior left front camera 74 may capture images of the range extending from the front left side of the automobile 1 to the left. The exterior left front camera 74 may also be provided on the left front door 9 or on the door mirror. The exterior left rear camera 75 is provided facing rearward on the left side of the automobile 1. In Fig. 4, the exterior left rear camera 75 is provided, for example, on the left front door 9 or a door mirror, which is an exterior member that can be opened and closed, of the automobile 1. The exterior left rear camera 75 may capture images of the range from the left side of the automobile 1 to the rear left side. The exterior rear camera 76 is provided facing rearward at the rear of the automobile 1. In Fig. 4, the exterior rear camera 76 is provided, for example, in the upper center portion of the rear gate 12, which is an exterior member that can be opened and closed, in the automobile 1. The exterior rear camera 76 may capture images of the area behind the automobile 1. 4 can capture images of the 360-degree surroundings outside the automobile 1 in divided views. If the control system of the automobile 1 predicts, for example, a collision or lane departure based on the images captured by the multiple exterior cameras 26, it can execute driving control to avoid such events.

[0033] FIG. 5 is a flowchart of the boarding / alighting monitoring control by the occupant abandonment warning device 20. In FIG. The ECU 43 of the occupant monitoring device 28 of the automobile 1 may repeatedly execute the getting-in and getting-out monitoring control of FIG. Vehicle sensors such as the door open / close sensor 23 and the door lock sensor 24 of this embodiment are capable of detecting multiple operations of the automobile 1 during one entry or exit of an occupant.

[0034] In step ST1, the ECU 43 determines whether the locked door lock sensor 24 has been unlocked based on the detection of the door lock sensor 24. If the door lock sensor 24 has not been unlocked, the ECU 43 repeats this process. If the door lock sensor 24 has been unlocked, the ECU 43 proceeds to step ST2.

[0035] In step ST2, the ECU 43 determines whether or not a closed door of the automobile 1 has been opened based on the detection of the door opening / closing sensor 23. In this determination, the rear gate 12 is also included in the doors. If any door of the automobile 1 has not been opened, the ECU 43 returns the process to step ST1. If any door of the automobile 1 has been opened, the ECU 43 advances the process to step ST3.

[0036] In step ST3, the ECU 43 uses the actuator 27 to control the angle of view of the exterior camera 26 that can capture an image of an occupant getting in through the open door. The ECU 43 controls the angle of view of the exterior camera 26 so that an image including the road surface around the body 2 of the automobile 1 can be captured, so that an image of a child, including an infant, trying to get in through the open door can be captured. A specific example of controlling the angle of view of the exterior camera 26 will be described later. Here, the ECU 43 may select one exterior camera 26 from the multiple exterior cameras 26 shown in FIG. 4 that can capture an image of a child, including an infant, trying to get in through the open door, and control the angle of view of the selected exterior camera 26, based on the positional relationship with the open door. In addition, if the exterior camera 26 can capture an image of a child, including an infant, trying to get into the vehicle through an open door at the current angle of view for monitoring the surroundings, the ECU 43 does not need to actually control the angle of view of the exterior camera 26 in this process.

[0037] In step ST4, the ECU 43 acquires a captured image of the outside of the vehicle from the outside camera 26 that has undergone the process of step ST3. The ECU 43 may record the acquired captured image of the outside of the vehicle in the memory 44.

[0038] In step ST5, the ECU 43 performs image analysis on the captured image of the exterior of the vehicle acquired in step ST4 to determine people and objects included in the captured image of the exterior of the vehicle. The ECU 43 may determine multiple people included in the captured image of the exterior of the vehicle. The ECU 43 may record information on the determined children, including infants, in the memory 44.

[0039] In step ST6, the ECU 43 determines whether or not the opened door has been closed based on the detection of the door opening / closing sensor 23. In this determination, the rear gate 12 is also included in the doors. If the opened door has not been closed, the ECU 43 repeats this process. If the opened door has been closed, the ECU 43 proceeds to step ST7.

[0040] In step ST7, the ECU 43 uses the actuator 27 to control the angle of view of the exterior camera 26, which can capture an image of an occupant who may have entered the vehicle through the closed door, so that the image capture includes the road surface around the body 2 of the vehicle 1. The ECU 43 controls the angle of view of the exterior camera 26 so that the image capture can capture the outside of the vehicle after a child, including an infant, has entered the vehicle through the closed door. A specific example of controlling the angle of view of the exterior camera 26 will be described later. Here, the ECU 43 may select one exterior camera 26 from the multiple exterior cameras 26 shown in FIG. 4 that can capture the outside of the vehicle after a child has entered the vehicle through the closed door, based on its positional relationship with the closed door, and control the angle of view of the selected exterior camera 26. In addition, if the exterior camera 26 is capable of capturing images of the outside of the vehicle after the occupant has entered the vehicle through a door that has already been closed, for example, by processing step ST3, the ECU 43 does not need to actually control the angle of view of the exterior camera 26 in this processing.

[0041] In step ST8, the ECU 43 acquires a captured image of the outside of the vehicle from the outside camera 26 that has performed the process in step ST7. The ECU 43 may record the acquired captured image of the outside of the vehicle in the memory 44.

[0042] In step ST9, the ECU 43 performs image analysis on the captured image of the exterior of the vehicle acquired in step ST8, and determines people and objects included in the captured image of the exterior of the vehicle. The ECU 43 may determine multiple people included in the captured image of the exterior of the vehicle. The ECU 43 may record information on the determined children, including infants, in the memory 44.

[0043] In step ST10, ECU 43 uses actuator 27 to return the angle of view of outside camera 26, which was actually adjusted in step ST3 or step ST7, to the angle of view before the start of this control. This enables outside camera 26 to capture images of the surroundings of automobile 1 while it is moving.

[0044] In step ST11, the ECU 43 compares the occupant determined in step ST5 with the occupant determined in step ST9 to determine whether the presence or absence of a person or object outside the vehicle has changed. If the presence or absence of a person or object outside the vehicle shown in the multiple captured images of the vehicle exterior has not changed, the ECU 43 determines that no person or object has entered the vehicle even if the door has been opened or closed, and ends this control. If the presence or absence of a person or object outside the vehicle has changed, the ECU 43 proceeds to step ST12. If the angle of view of the image of the exterior of the vehicle captured in step ST4 is the same as the angle of view of the image of the exterior of the vehicle captured in step ST8, the ECU 43 can also determine whether there is a change in the presence or absence of a person or object outside the vehicle based on the presence or absence of a difference in image components, based on a direct comparison of these images of the exterior of the vehicle. However, in this embodiment, the angle of view of the exterior camera for capturing images of the exterior of the vehicle may be controlled in steps ST3 and ST7 before capturing each image. For this reason, it is preferable to determine whether there is a change in the presence or absence of a person or object in each captured image, rather than directly comparing the images of the exterior of the vehicle.

[0045] In step ST12, the ECU 43 determines whether the change in the presence or absence of a person or object outside the vehicle includes a change in the presence or absence due to a child getting in. If there is a change in the presence or absence due to a child getting in, the ECU 43 proceeds to step ST13.

[0046] In step ST13, the ECU 43 counts up the child riding flag 51 stored in the memory 44 based on the number of children that has changed as a result of riding. As a result, the value of the child riding flag 51 is changed when the ECU 43 determines that children are riding in the automobile 1 based on the image captured by the exterior camera 26. Thereafter, the ECU 43 ends this control.

[0047] In step ST14, the ECU 43 determines whether the change in the presence or absence of a person or object outside the vehicle includes a change due to a child getting off. If there is a change in the presence or absence due to a child getting off, the ECU 43 proceeds to step ST14. If there is no change in the presence or absence due to a child getting off, the ECU 43 ends this control.

[0048] In step ST15, the ECU 43 counts up the child getting-out flag 52 stored in the memory 44 based on the number of children who have gotten out of the vehicle. As a result, the value of the child getting-out flag 52 is changed when the ECU 43 determines that children have gotten out of the vehicle 1 based on the image captured by the exterior camera 26. Thereafter, the ECU 43 ends this control. The ECU 43 may count down the child riding flag 51 stored in the memory 44 depending on the number of children who have dismounted.

[0049] In this way, the ECU 43, as a determination unit, can determine the presence or absence of a child outside the vehicle based on the images of the exterior captured by the exterior camera 26 each time the vehicle sensor, such as the door lock sensor 24 or the door open / close sensor 23, detects the movement of the vehicle 1. The ECU 43 can then infer that a child has boarded the vehicle 1 when, in multiple images of the exterior captured by the exterior camera 26 for one boarding or one disembarking session, there is a change from an image showing a child outside the vehicle to an image showing no child outside the vehicle. The ECU 43 can also infer that a child has disembarked from the vehicle 1 when, in multiple images of the exterior captured by the exterior camera 26 for one boarding or one disembarking session, there is a change from an image showing no child outside the vehicle to an image showing a child outside the vehicle. In other words, the ECU 43 can infer that a child has boarded or disembarked from the vehicle 1 based on a change in the presence or absence of a child outside the vehicle in multiple images of the exterior captured by the exterior camera 26 for one boarding or one disembarking session. Furthermore, in making these determinations, the ECU 43 changes the angle of view of the exterior camera 26 using the actuator 27, and can estimate and determine whether or not there is a small child outside the vehicle based on multiple images captured by the exterior camera 26 with the angle of view changed.

[0050] FIG. 6 is a flowchart of a child leaving determination control by the occupant leaving alarm device 20 according to the first embodiment of the present invention. The ECU 43 of the occupant monitoring device 28 of the automobile 1 may repeatedly execute the child abandonment determination control of FIG.

[0051] In step ST21, the ECU 43 determines whether the automobile 1 has ended its travel after traveling. The ECU 43 may determine whether the automobile 1 has ended its travel after traveling, for example, based on whether the automobile 1 has stopped after traveling or whether the ignition switch 25 has been turned from an on state to an off state. If the automobile 1 has ended its travel after traveling, the ECU 43 proceeds to step ST22. If the automobile 1 has not ended its travel after traveling, the ECU 43 proceeds to step ST23.

[0052] In step ST22, the ECU 43 determines whether or not the occupant has performed an operation to get out of the automobile 1. The ECU 43 may also determine whether or not the occupant has performed an operation to park or stop the automobile 1. When the occupant gets out of the automobile 1, for example, the occupant turns the ignition switch 25 from an on state to an off state, or opens or closes the door. If the occupant has performed these operations, the ECU 43 proceeds to step ST24. If the occupant has not performed these operations, the ECU 43 proceeds to step ST23.

[0053] In step ST23, the ECU 43 determines whether only children, including infants, are riding in the automobile 1. The ECU 43 compares the child riding flag 51 with the child getting-off flag 52, and if the child riding flag 51 is greater than the child getting-off flag 52, it can determine that children, including infants, are riding in the automobile 1. The ECU 43 can also determine whether there are any adult passengers riding in the automobile 1 by acquiring and analyzing the current image captured by the in-vehicle camera 22. The ECU 43 may determine whether only children, including infants, are riding in the automobile 1 based on this information. If only children, including infants, are not riding in the automobile 1, the ECU 43 returns the process to step ST21. In this case, the ECU 43 can repeat the determination process from step ST21 to step ST23. If only children, including infants, are riding in the automobile 1, the ECU 43 proceeds to step ST24.

[0054] In step ST24, the ECU 43 acquires from the memory 44 the child getting on flag 51 and the child getting off flag 52 as the getting on / off monitoring results updated by the getting on / off monitoring control of FIG.

[0055] In step ST25, the ECU 43 determines the possibility that only children, including infants, have been left in the vehicle based on the boarding / exiting monitoring result acquired in step ST24. If the value of the child boarding flag 51 is greater than the value of the child disembarking flag 52, there is a high possibility that only children, including infants, have been left in the vehicle. In this case, the ECU 43 determines that there is a possibility that only children, including infants, have been left in the vehicle, and proceeds to step ST26. In all other cases, the ECU 43 determines that there is no possibility that a child has been left in the vehicle, and ends this control. This allows the ECU 43, as a determination unit, to estimate and determine the possibility that a child has been left behind in the automobile 1 based on a change in the presence or absence of a person or object outside the vehicle in the image captured by the exterior camera 26. In addition, if the ECU 43 cannot determine whether a child has been left inside the vehicle based on the detection of the in-vehicle camera 22 or the millimeter-wave in-vehicle sensor 21 as an in-vehicle sensor, it can estimate and determine the possibility that a child has been left inside the vehicle 1 based on changes in the presence or absence of people or objects outside the vehicle in the images captured by the exterior camera 26. 5, when the child boarding flag 51 recorded in the memory 44 is counted down based on the change in the number of children due to disembarking, the ECU 43 may acquire only the child boarding flag 51 in step ST24, and determine whether only children, including infants, are left in the vehicle based on whether the value of the child boarding flag 51 is 0 in step ST25. If the value of the child boarding flag 51 is 0, all children have disembarked.

[0056] In step ST26, the ECU 43 outputs a warning that a child, including an infant, has been left behind.

[0057] In this way, ECU 43, as a judgment unit, can estimate and judge the possibility that a child has been left behind in automobile 1 when automobile 1 has finished traveling, when an operation is performed to park or stop automobile 1 or to disembark from automobile 1, or when only a child is on board. In addition, the ECU 43 refers to the values ​​of flags such as the child boarding flag 51 and the child disembarking flag 52 when estimating the possibility of a child being left in the vehicle 1 based on changes in the presence or absence of people or objects outside the vehicle in the images captured by the exterior camera 26. Furthermore, even if the ECU 43 does not determine that a child has been left in the vehicle 1 based on the detection of an occupant inside the vehicle by an in-vehicle sensor, the ECU 43 can estimate and determine that a child has been left in the vehicle 1 based on the values ​​of these flags. Conversely, when it is not determined based on the values ​​of these flags that there is a child left in the vehicle 1, the ECU 43, as the warning control unit, can not output a child abandonment warning.

[0058] FIG. 7 is a flowchart showing abandoned passenger warning output control by the abandoned passenger warning device 20. In FIG. The ECU 43 of the occupant monitoring device 28 of the automobile 1 may repeatedly execute the abandoned alarm output control of Fig. 7. The ECU 43 of the occupant monitoring device 28 of the automobile 1 may repeatedly execute the abandoned alarm output control of Fig. 7 as the processing of step ST26 of Fig. 6.

[0059] In step ST41, the ECU 43 determines whether the automobile 1 has stopped traveling after traveling. The process of step ST41 may be the same as the process of step ST21. If the automobile 1 has not stopped traveling after traveling, the ECU 43 repeats this process. If the automobile 1 has stopped traveling after traveling, the ECU 43 proceeds to step ST42.

[0060] In step ST42, the ECU 43 determines whether or not a parking or stopping operation has been performed after the automobile 1 has been traveling. When an occupant gets out of the automobile 1, for example, the ignition switch 25 is turned from the on state to the off state. If such a parking or stopping operation has not been performed, the ECU 43 repeats this process. If a parking or stopping operation has been performed, the ECU 43 proceeds to step ST43.

[0061] In step ST43, the ECU 43 outputs a warning of abandonment by displaying it on the meter panel 29 in front of the driver. This allows the occupants who have not gotten out of the automobile 1 to visually recognize the warning of abandonment output to the output unit provided in the automobile 1.

[0062] In step ST44, the ECU 43 determines whether or not an operation to have an occupant exit the vehicle 1 has been performed. When an occupant exits the vehicle 1, the occupant opens and closes the door and locks the door. If these operations are detected by the door opening / closing sensor 23 or the door lock sensor 24, the ECU 43 determines that an operation to have an occupant exit the vehicle 1 has been performed, and proceeds to step ST45. If neither operation has been performed, the ECU 43 returns the process to step ST43. The ECU 43 continues to output an abandoned vehicle warning on the meter panel 29.

[0063] In step ST45, the ECU 43 outputs a sound for warning of abandonment from the horn 30. As a result, an occupant who has exited the automobile 1 can hear the abandonment warning sound output from the output unit provided in the automobile 1 when he or she is near the automobile 1.

[0064] In step ST46, the ECU 43 transmits an abandonment warning from the wireless communication device 31 of the automobile 1 to the passenger terminal 60 of the passenger. The passenger terminal 60 outputs the abandonment warning received from the wireless communication device 31 of the automobile 1. This allows the passenger who has exited the automobile 1 to receive the abandonment warning even if they are not near the automobile 1.

[0065] In step ST47, the ECU 43 determines whether or not the abandonment warning has been transmitted to the occupant terminal 60 a predetermined number of times. Here, the predetermined number may be three, for example. If the abandonment warning has not been transmitted to the occupant terminal 60 the predetermined number of times, the ECU 43 returns the process to step ST46. The ECU 43 repeats the process from step ST46 to step ST47 until the abandonment warning has been transmitted to the occupant terminal 60 the predetermined number of times. When the abandonment warning has been transmitted to the occupant terminal 60 the predetermined number of times, the ECU 43 ends this control.

[0066] Next, a specific example of the control of the angle of view of the outside camera 26 by the getting-in / out monitoring control of FIG. 5 will be described.

[0067] FIG. 8 is an explanatory diagram of an example of the angle of view control in a planar direction of the vehicle exterior right rear camera 73 serving as the vehicle exterior camera 26 provided on the right front door 8 in FIG. In FIG. 8, a small child standing outside the rear of the right front door 8 of the automobile 1 is about to get into the automobile 1 and approach the driver's seat 4. FIG. 8 shows the state of this one ride from timing T01 to timing T06.

[0068] 8, the angle of view of the exterior right rear camera 73 provided on the right front door 8 is set so as to capture an image of the right rear of the vehicle 1 to the right of the right side of the vehicle 1 at the normal angle of view. This allows the exterior right rear camera 73 to capture images of other vehicles 1 on the right side of the vehicle 1, other vehicles 1 approaching from the right rear of the vehicle 1, and the like.

[0069] At timing T02, a small child opens the right front door 8 of the automobile 1. The imaging range of the exterior right rear camera 73 provided on the open right front door 8 is oriented to image the area to the right rear of the right front door 8 of the automobile 1. A child who is positioned closer to the automobile 1 than the rear end of the right front door 8 will no longer be imaged by the exterior right rear camera 73. 5, the ECU 43 controls the angle of view of the vehicle exterior right rear camera 73, which serves as the vehicle exterior camera 26. The ECU 43 controls the orientation of the vehicle exterior right rear camera 73 so that it faces the right front door 8. This allows an image of a child who is closer to the vehicle 1 than the rear end of the right front door 8 to be captured through the glass of the right front door 8. Thereafter, as shown at timing T04, the child who has opened the right front door 8 can get into the car 1 through the right front door 8 and sit in the driver's seat 4. Also, as shown at timing T05, the child in the car 1 closes the right front door 8. When the right front door 8 is closed, the ECU 43 controls the angle of view of the vehicle exterior right rear camera 73, which serves as the vehicle exterior camera 26, at timing T06 by processing step ST7 in Fig. 5. The ECU 43 controls the orientation of the vehicle exterior right rear camera 73 so that it captures an image of the area to the right rear of the right front door 8. As a result, a child riding in the vehicle 1 will no longer be captured by the vehicle exterior right rear camera 73. Then, for example, the ECU 43 acquires an image captured by the exterior right rear camera 73 by processing step ST4 of Figure 5 at timing T03, and acquires an image captured by the exterior right rear camera 73 by processing step ST8 of Figure 5 at timing T06. In this case, the child is captured in the image captured outside the vehicle in step ST4, but the child is not captured in the image captured outside the vehicle in step ST8. The ECU 43 determines in step ST11 of Fig. 5 that there has been a change in the presence or absence of a person outside the vehicle, determines in step ST12 that a child has boarded the vehicle, and counts up the value of the child boarding flag 51 by 1 in step ST13.

[0070] When a child seated in the driver's seat 4 gets out of the vehicle 1, the order is reversed from that shown in FIG. 8. In this case, the ECU 43 increments the value of the child getting-out flag 52 by 1 in step ST15 of FIG. 5. In this way, when a child seated in the driver's seat 4 gets out of the vehicle, the values ​​of the child getting-in flag 51 and the child getting-out flag 52 are both incremented by 1. In this state, the flag values ​​match, so the ECU 43 determines in step ST25 of the child leaving-behind determination control in FIG. 6 that a child is not likely to be left behind. In this case, the ECU 43 does not output a child leaving warning in step ST26. On the other hand, if the child seated in the driver's seat 4 has not exited the vehicle 1, the count-up process of the child exit flag 52 is not executed in step ST15 of Fig. 5. In this state, the flag values ​​do not match, so the ECU 43 determines that there is a possibility of a child being left behind in step ST25 of the child leaving determination control in Fig. 6. The ECU 43 executes the process of step ST26 and outputs a child leaving warning. In this way, the ECU 43 causes the actuator 27 to change the orientation of the exterior right rear camera 73 to face backward when capturing images, and can estimate and determine the presence or absence of a small child outside the vehicle based on multiple images of the exterior of the vehicle captured by the exterior right rear camera 73 with the angle of view changed. Because the image captured by the changed-orientation exterior right rear camera 73 includes the road surface around the vehicle body 2, even if a small child is outside the vehicle, it is possible to capture an image of at least the child's feet.

[0071] Fig. 9 is an explanatory diagram of an example of vertical angle control of the vehicle exterior left rear camera 75 as the vehicle exterior camera 26 provided on the left front door 9 in Fig. 4. Fig. 9 shows an example in which the left front door 9 is opened and closed. In FIG. 9, a small child is standing outside the automobile 1 behind the left front door 9 and is about to get into the driver's seat 4 of the automobile 1. FIG. 9 shows the state of this one ride from timing T11 to timing T14.

[0072] 9, the angle of view of the exterior left rear camera 75 provided on the left front door 9 is set to capture an image of the left rear of the vehicle 1 along the fore-and-aft direction of the vehicle 1 at a normal angle of view. This allows the exterior left rear camera 75 to capture images of other vehicles 1 on the left side of the vehicle 1, other vehicles 1 approaching from the left rear of the vehicle 1, etc. A small child standing behind the left front door 9 outside the vehicle 1 can be captured by the exterior left rear camera 75.

[0073] At timing T12, a small child opens the left front door 9 of the automobile 1. The imaging range of the vehicle exterior left rear camera 75 provided at the open left front door 9 is oriented along the fore-and-aft direction of the automobile 1, so as to image the area to the left rear of the left front door 9 of the automobile 1. A child positioned behind the left front door 9 enters a blind spot blocked by the open left front door 9, as shown by diagonal hatching in the drawing, and is no longer imaged by the vehicle exterior left rear camera 75. 5, the ECU 43 controls the angle of view of the vehicle exterior left rear camera 75, which serves as the vehicle exterior camera 26. The ECU 43 controls the orientation of the vehicle exterior left rear camera 75 so that it faces downward so as to capture a wide image of the road surface on the left side of the automobile 1. This allows a small child positioned behind the left front door 9 to be captured through the glass of the left front door 9. Thereafter, as shown at timing T14, the child who opened the left front door 9 can get into the car 1 through the left front door 9 and sit in the passenger seat 5. After getting into the car 1, the child closes the left front door 9. When the left front door 9 is closed, the ECU 43 performs the processing of step ST7 in Fig. 5 to control the angle of view of the vehicle exterior left rear camera 75 serving as the vehicle exterior camera 26. The ECU 43 controls the orientation of the vehicle exterior left rear camera 75 so that it captures an image of the area to the left of the left front door 9 along the front-to-rear direction of the vehicle 1. As a result, a child riding in the vehicle 1 will no longer be captured by the vehicle exterior left rear camera 75. Then, the ECU 43 acquires an image captured by the vehicle exterior left rear camera 75 through the processing of step ST4 in FIG. 5, and acquires an image captured by the vehicle exterior left rear camera 75 through the processing of step ST8. In this case, the child is captured in the image captured outside the vehicle in step ST4, but the child is not captured in the image captured outside the vehicle in step ST8. The ECU 43 determines in step ST11 of Fig. 5 that there has been a change in the presence or absence of a person outside the vehicle, determines in step ST12 that a child has boarded the vehicle, and counts up the value of the child boarding flag 51 by 1 in step ST13.

[0074] When a child seated in the passenger seat 5 gets out of the automobile 1, the order is reversed from that shown in FIG. 9. In this case, the ECU 43 increments the value of the child getting-out flag 52 by 1 in step ST15 of FIG. 5. In this way, when a child seated in the passenger seat 5 gets out of the automobile 1, the values ​​of the child getting-in flag 51 and the child getting-out flag 52 are both incremented by 1. In this state, the flag values ​​match, so the ECU 43 determines in step ST25 of the child leaving-behind determination control in FIG. 6 that a child is not likely to be left behind. In this case, the ECU 43 does not output a child leaving warning in step ST26. On the other hand, if the child seated in the passenger seat 5 has not exited the vehicle 1, the count-up process of the child exit flag 52 is not executed in step ST15 of Fig. 5. In this state, the flag values ​​do not match, so the ECU 43 determines that there is a possibility of a child being left behind in step ST25 of the child leaving determination control in Fig. 6. The ECU 43 executes the process of step ST26 and outputs a child leaving warning. In this way, the ECU 43 causes the actuator 27 to change the orientation of the exterior left rear camera 75 so that it faces downward when capturing images, and can estimate and determine the presence or absence of a small child outside the vehicle based on the multiple images of the exterior captured by the exterior left rear camera 75 with its angle of view changed. Since the image captured by the changed-orientation exterior left rear camera 75 includes the road surface around the vehicle body 2 behind the exterior left rear camera 75, even if, for example, the left rear door 11, which is separate from the left front door 9 on which the exterior left rear camera 75 is provided, is open and a small child is outside the vehicle behind it, it is possible to capture an image of at least the child's feet.

[0075] 9, the ECU 43 changes the orientation of the vehicle exterior left rear camera 75 only in the vertical direction to face downward. In reality, the ECU 43 may change the orientation of the vehicle exterior left rear camera 75 not only in the vertical direction but also in the horizontal direction as shown in FIG.

[0076] Fig. 10 is an explanatory diagram of an example of vertical angle control of the vehicle exterior left rear camera 75 as the vehicle exterior camera 26 provided on the left front door 9 in Fig. 4. Fig. 10 shows an example in which the left rear door 11 is opened and closed. In FIG. 10, a small child standing outside the automobile 1 behind the left rear door 11 is about to get into the rear seat 6 of the automobile 1. FIG. 10 shows the state of this one ride from timing T21 to timing T24.

[0077] 10 , the field of view of the exterior left rear camera 75 provided on the left front door 9 is set to capture an image of the left rear of the vehicle 1 along the fore-and-aft direction of the vehicle 1 at a normal field of view. This allows the exterior left rear camera 75 to capture images of other vehicles 1 on the left side of the vehicle 1, other vehicles 1 approaching from the left rear of the vehicle 1, etc. A small child standing behind the left rear door 11 outside the vehicle 1 can be captured by the exterior left rear camera 75.

[0078] At timing T22, a small child opens the left rear door 11 of the automobile 1. The imaging range of the vehicle exterior left rear camera 75, which is provided on the left front door 9 different from the open left rear door 11, is oriented along the fore-and-aft direction of the automobile 1 so as to image the area to the left rear of the left front door 9 of the automobile 1. The child positioned behind the left rear door 11 enters the blind spot blocked by the open left rear door 11, as shown by the diagonal hatching in the figure, and is therefore not imaged by the vehicle exterior left rear camera 75. 5, the ECU 43 controls the angle of view of the vehicle exterior left rear camera 75, which serves as the vehicle exterior camera 26. The ECU 43 controls the orientation of the vehicle exterior left rear camera 75 so that it faces downward so as to capture a wide image of the road surface on the left side of the vehicle 1. As a result, a small child positioned behind the left rear door 11 can be imaged through the gap between the left rear door 11 and the road surface by moving forward toward the left rear door 11 to get in the vehicle. Thereafter, as shown at timing T24, the child who opened the left rear door 11 gets into the car 1 through the left rear door 11 and can sit in the rear seat 6. After getting into the car 1, the child closes the left rear door 11. When the left rear door 11 is closed, the ECU 43 performs the processing of step ST7 in Fig. 5 to control the angle of view of the vehicle exterior left rear camera 75 serving as the vehicle exterior camera 26. The ECU 43 controls the orientation of the vehicle exterior left rear camera 75 so that it captures an image of the area to the left of the left front door 9 along the front-to-rear direction of the vehicle 1. As a result, a child riding in the vehicle 1 will no longer be captured by the vehicle exterior left rear camera 75. Then, the ECU 43 acquires an image captured by the vehicle exterior left rear camera 75 through the processing of step ST4 in FIG. 5, and acquires an image captured by the vehicle exterior left rear camera 75 through the processing of step ST8. In this case, the child is captured in the image captured outside the vehicle in step ST4, but the child is not captured in the image captured outside the vehicle in step ST8. The ECU 43 determines in step ST11 of Fig. 5 that there has been a change in the presence or absence of a person outside the vehicle, determines in step ST12 that a child has boarded the vehicle, and counts up the value of the child boarding flag 51 by 1 in step ST13.

[0079] When a child seated in the rear seat 6 gets out of the vehicle 1, the order is reversed from that shown in FIG. 10. In this case, the ECU 43 counts up the value of the child getting out flag 52 by 1 in step ST15 of FIG. 5. In this way, when a child seated in the rear seat 6 gets out of the vehicle, the values ​​of the child getting on flag 51 and the child getting out flag 52 are both counted up by 1. In this state, the flag values ​​match, so the ECU 43 determines in step ST25 of the child leaving determination control in FIG. 6 that a child is not likely to be left behind. In this case, the ECU 43 does not output a child leaving warning in step ST26. On the other hand, if a child seated in the rear seat 6 has not exited the vehicle 1, the count-up process of the child exit flag 52 is not executed in step ST15 of Fig. 5. In this state, the flag values ​​do not match, so the ECU 43 determines that there is a possibility of a child being left behind in step ST25 of the child leaving determination control in Fig. 6. The ECU 43 executes the process of step ST26 and outputs a child leaving warning. In this way, the ECU 43 causes the actuator 27 to change the orientation of the exterior left rear camera 75 to face downward when capturing images, and can estimate and determine the presence or absence of a small child outside the vehicle based on the multiple images of the exterior captured by the exterior left rear camera 75 with its angle of view changed. Because the image captured by the changed-orientation exterior left rear camera 75 includes the road surface around the vehicle body 2, even if a small child is outside the vehicle, it is possible to capture an image of at least the child's feet.

[0080] 10, the ECU 43 changes the orientation of the vehicle exterior left rear camera 75 downward only in the vertical direction. In reality, the ECU 43 may change the orientation of the vehicle exterior left rear camera 75 not only in the vertical direction but also in the horizontal direction as shown in FIG.

[0081] FIG. 11 is an explanatory diagram of an example of vertical angle control of the vehicle exterior rear camera 76 serving as the vehicle exterior camera 26 provided on the rear gate 12 in FIG. In FIG. 11, a small child standing outside the automobile 1 at the rear of the tailgate 12 is about to get into the automobile 1 through the tailgate 12 and onto the rear seat 6. FIG. 11 shows the state of this one ride from timing T31 to timing T34.

[0082] 11, the exterior rear camera 76 provided on the rear gate 12 has a normal angle of view set to capture an image of the area behind the vehicle 1 along the fore-and-aft direction of the vehicle 1. This allows the exterior rear camera 76 to capture images of other vehicles 1 behind the vehicle 1, other vehicles 1 approaching from behind the vehicle 1, etc. A small child standing behind the rear gate 12 outside the vehicle 1 can be captured by the exterior rear camera 76.

[0083] At timing T32, a small child opens the rear gate 12 of the automobile 1. The imaging range of the vehicle exterior rear camera 76 provided on the open rear gate 12 is oriented along the fore-and-aft direction of the automobile 1, so as to image the area above and behind the rear gate 12 of the automobile 1. The child positioned behind the rear gate 12 is in the blind spot and will not be imaged by the vehicle exterior rear camera 76. Therefore, at timing T33, the ECU 43 controls the angle of view of the vehicle exterior rear camera 76, which serves as the vehicle exterior camera 26, by processing step ST3 in Fig. 5. The ECU 43 controls the orientation of the vehicle exterior rear camera 76 so that it faces downward so that it can capture a wide image of the road surface behind the automobile 1. This allows a small child positioned behind the tailgate 12 to be captured. Thereafter, as shown at timing T34, the child who opened the rear gate 12 can get into the automobile 1 through the rear gate 12 and sit in the rear seat 6. After getting into the automobile 1, the child closes the rear gate 12. When the rear gate 12 is closed, the ECU 43 performs the processing of step ST7 in Fig. 5 to control the angle of view of the vehicle exterior rear camera 76 serving as the vehicle exterior camera 26. The ECU 43 controls the orientation of the vehicle exterior rear camera 76 so that it captures an image of the area behind the vehicle 1 along the fore-and-aft direction of the vehicle 1. As a result, children riding in the vehicle 1 will no longer be captured by the vehicle exterior rear camera 76. Then, the ECU 43 acquires an image captured by the vehicle exterior rear camera 76 through the process of step ST4 in FIG. 5, and acquires an image captured by the vehicle exterior rear camera 76 through the process of step ST8. In this case, the child is captured in the image captured outside the vehicle in step ST4, but the child is not captured in the image captured outside the vehicle in step ST8. The ECU 43 determines in step ST11 of Fig. 5 that there has been a change in the presence or absence of a person outside the vehicle, determines in step ST12 that a child has boarded the vehicle, and counts up the value of the child boarding flag 51 by 1 in step ST13.

[0084] When a child seated in the rear seat 6 gets off through the rear gate 12, the order is reversed from that shown in FIG. 11. In this case, the ECU 43 counts up the value of the child getting off flag 52 by 1 in step ST15 of FIG. 5. In this way, when a child seated in the rear seat 6 gets off through the rear gate 12, the values ​​of the child getting on flag 51 and the child getting off flag 52 are both counted up by 1. In this state, the flag values ​​match, so the ECU 43 determines in step ST25 of the child leaving determination control in FIG. 6 that a child is not likely to be left behind. In this case, the ECU 43 does not output a child leaving warning in step ST26. On the other hand, if a child seated in the rear seat 6 has not exited the vehicle through the tailgate 12, the count-up process of the child exit flag 52 is not executed in step ST15 of Fig. 5. In this state, the flag values ​​do not match, so the ECU 43 determines that there is a possibility of a child being left behind in step ST25 of the child leaving determination control in Fig. 6. The ECU 43 executes the process of step ST26 and outputs a child leaving warning. In this way, the ECU 43 causes the actuator 27 to change the orientation of the exterior rear camera 76 so that it faces downward when capturing images, and can estimate and determine whether or not a small child is present outside the vehicle based on the multiple images of the exterior captured by the exterior rear camera 76 with its angle of view changed. Because the images captured by the changed-orientation exterior rear camera 76 include the road surface around the rear of the vehicle body 2, even if a small child is outside the vehicle, it is possible to capture an image of at least the child's feet.

[0085] FIG. 12 is an explanatory diagram of a method for determining the physique of an occupant when another exterior rear camera 77 is provided below the tailgate 12. In FIG. The other exterior rear camera 77 in FIG. 12 may be provided below the tailgate 12 alongside the license plate lamp, etc. The other exterior rear camera 77 may be provided to capture an image of the road surface immediately behind the vehicle 1 from below the tailgate 12 in order to provide a rear monitoring image when reversing. When such another vehicle exterior rear camera 77 is provided below the rear gate 12, the vehicle exterior rear camera 76 captures images further rearward than the other vehicle exterior rear camera 77. In this case, the ECU 43 may determine that a person who is captured in both the image captured by the exterior rear camera 76 and the image captured by the other exterior rear camera 77 is an adult with a sufficient physique, and may determine that a person who is captured only in the image captured by the other exterior rear camera 77 is a child with a small physique. Alternatively, for example, the ECU 43 may distinguish between people captured in images as large adults and small children based on the image capture range and image capture position of the images captured by each of the outside cameras 26. In the processing of step ST12 and step ST13 of Figure 5, the ECU 43 may use these discrimination processes to distinguish between external objects present around the automobile 1 and small children, including infants, and other objects, such as adults or luggage.

[0086] As described above, in this embodiment, rather than estimating whether a passenger is getting on or off the vehicle 1 based on the door opening / closing history or the like, the exterior cameras 26 (71-77) capture images of passengers outside the vehicle 1 to get on or off the vehicle 1, and estimate and determine the possibility that a child, including an infant, has been left behind in the vehicle 1 based on changes in the presence or absence of people or objects outside the vehicle in the images captured by the exterior cameras 26 (71-77). For example, a person who gets into the vehicle 1 may change from being present in the images captured outside the vehicle to not being present in the images captured outside the vehicle. Conversely, a person who gets off the vehicle 1 may change from not being present in the images captured outside the vehicle to being present in the images captured outside the vehicle. In this embodiment, the possibility that a child who has not gotten off the vehicle 1 has been left behind in the vehicle 1 is estimated and determined based on such changes in the presence or absence of people or objects outside the vehicle in the images captured by the exterior cameras 26 (71-77). In this embodiment, if a child has been left behind in the vehicle 1, a child leaving warning can be output. In this embodiment, if there is a possibility that a child has been left in the vehicle, a child abandonment alarm can be output. On the other hand, if an occupant opens and closes the door and something other than a child, such as luggage, is left in the vehicle, it can be determined that this is not a case of a child being left in the vehicle. In this case, the child abandonment alarm will not be output. Moreover, in this embodiment, the image captured outside the vehicle by the exterior camera 26 (71-77) is used for the judgment, so the image capture range is not continually obstructed by a seat or the like as in the case of images captured inside the vehicle, and small children, including infants, can be reliably captured. In this embodiment, even when small children, including infants, get on or off the vehicle, the image capture can be reliably captured by the exterior camera without any obstruction, and a leaving warning can be output. In this way, in this embodiment, the actual getting on and off of the automobile 1 is imaged and a change in presence or absence is determined based on the image, so that the leaving of a child can be determined to be distinguishable from the leaving of other items, such as luggage, and the accuracy of the determination of the leaving of a child can be improved compared to when the determination is simply based on the door opening and closing history. Also, in this embodiment, by obtaining the accuracy of the determination of the leaving of a child, the output of the child leaving alarm can be improved so that it is limited to cases when it is almost necessary.

[0087] [Second embodiment] Next, an occupant abandonment warning device 20 for an automobile 1 according to a second embodiment of the present invention will be described. Below, differences from the above-described embodiment will be mainly described.

[0088] FIG. 13 is a flowchart of a child leaving-behind determination control by the occupant leaving-behind alarm device 20 according to the second embodiment of the present invention. The ECU 43 of the occupant monitoring device 28 of the automobile 1 may repeatedly execute the child abandonment determination control of FIG. Steps ST21 to ST23 and steps ST24 to ST26 in Fig. 13 are the same as those in Fig. 6. However, if an operation to get out of the automobile 1 is performed in step ST22, and if it is determined in step ST23 based on the getting-in and getting-out history that only children, including infants, may have been left in the vehicle, the ECU 43 proceeds to step ST51.

[0089] In step ST51, the ECU 43 causes the in-vehicle camera 22 or the millimeter-wave in-vehicle sensor 21 to detect the interior of the vehicle.

[0090] In step ST52, the ECU 43 acquires the current image of the interior of the vehicle captured by the in-vehicle camera 22 and the current detection result by the millimeter-wave in-vehicle sensor 21. The ECU 43 may analyze the acquired current information and determine the occupants, including children, currently riding in the vehicle.

[0091] In step ST53, the ECU 43 determines whether or not only children, including infants, are left in the vehicle based on the vehicle interior detection result acquired in step ST52. The ECU 43 may also determine whether or not only children, including infants, will be left in the vehicle. If it is determined that only children, including infants, have been left in the vehicle based on the current detection results of the vehicle interior by the in-vehicle camera 22 or the millimeter-wave in-vehicle sensor 21, or if it is determined that this is likely, the ECU 43 proceeds to step ST26. In this case, the ECU 43 outputs a child leaving warning in the processing of step ST26 based on the determination based only on the in-vehicle detection results. If it cannot be determined that only children, including infants, are left in the vehicle based on the current detection results of the vehicle interior by the in-vehicle camera 22 or the millimeter-wave in-vehicle sensor 21, the ECU 43 proceeds to step ST24. In this case, the ECU 43 acquires the child boarding flag 51 and the child disembarking flag 52, which indicate the results of monitoring the passengers getting in and out, and determines whether only children, including infants, are left in the vehicle based on the child boarding and disembarking history. If it is determined that only children, including infants, are left in the vehicle based on the child boarding and disembarking history, the ECU 43 outputs a leaving warning in the processing of step ST26 based on the determination of the child boarding and disembarking history. On the other hand, if it cannot be determined that only children, including infants, are left in the vehicle based on the child boarding and disembarking history, the ECU 43 terminates this control without outputting a leaving warning in the processing of step ST26.

[0092] As described above, the automobile 1 of this embodiment has the interior sensors 21, 22 that detect passengers inside the automobile 1. In this embodiment, the possibility that a child has been left behind in the automobile 1 can be estimated and determined based on changes in the presence or absence of people or objects outside the automobile in images captured by the exterior cameras 26 (71-77) and the detection of passengers inside the automobile by the interior sensors 21, 22.

[0093] The above-described embodiment is an example of a preferred embodiment of the present invention, but the present invention is not limited to this, and various modifications and changes are possible within the scope of the gist of the invention.

[0094] For example, in the above-described embodiment, the ECU 43 of the occupant monitoring device 28 executes all of the controls shown in Figures 6 to 8 or the control shown in Figure 13. All or part of these controls may be executed by an ECU 43 other than the occupant monitoring device 28 that is provided in the control system of the automobile 1. Furthermore, multiple ECUs 43 in the control system of the automobile 1 may cooperate to execute these controls. [Explanation of symbols]

[0095] 1...Automobile (vehicle), 2...Vehicle body, 3...Vehicle compartment, 4...Driver's seat, 5...Passenger seat, 6...Rear seat, 7...Child seat, 8...Right front door, 9...Left front door, 10...Right rear door, 11...Left rear door, 12...Tailgate, 20...Occupant abandonment alarm system, 21...Millimeter wave interior sensor (interior sensor), 22...Interior camera (interior sensor), 23...Door opening / closing sensor (vehicle sensor), 24...Door lock sensor (vehicle sensor), 25...Ignition switch, 26...Exterior camera , 27... actuator, 28... occupant monitoring device, 29... meter panel, 30... horn, 31... wireless communication device, 32... in-vehicle network, 41... input / output unit, 42... timer, 43... ECU (determination unit, alarm control unit), 44... memory, 45... internal bus, 51... child boarding flag, 52... child exiting flag, 60... occupant terminal, 71... exterior front camera, 72... exterior right front camera, 73... exterior right rear camera, 74... exterior left front camera, 75... exterior left rear camera, 76... exterior rear camera, 77... other exterior rear camera

Claims

1. An occupant abandonment warning device for a vehicle, the device having an alarm control unit that outputs an abandonment warning when a child, including an infant, is left in the vehicle, an exterior camera that is provided to be able to capture images of occupants outside the vehicle; a determination unit that determines the possibility that a child has been left in the vehicle, The determination unit determining the possibility that a child has been left behind in the vehicle based on a change in the presence or absence of a person or object outside the vehicle in the image captured by the exterior camera; Vehicle occupant abandonment warning device.

2. a vehicle sensor for detecting the movement of the vehicle when a passenger gets on or off the vehicle; The vehicle sensor is capable of detecting multiple movements of the vehicle during a single entry or exit of a passenger, The determination unit determining whether or not there is a child outside the vehicle from the image captured by the exterior camera each time the vehicle sensor detects a movement of the vehicle; determining whether a child has boarded or disembarked from the vehicle based on a change in the presence or absence of a child outside the vehicle in a plurality of images captured by the exterior camera for one boarding or one disembarking; 2. The vehicle occupant abandonment warning device according to claim 1.

3. The vehicle sensor The vehicle door lock sensor includes at least one of a door lock sensor that detects locking and unlocking of the vehicle door, and a door opening / closing sensor that detects opening and closing of the vehicle door.

3. The vehicle occupant abandonment warning device according to claim 2.

4. the vehicle has an actuator that changes the orientation of the exterior camera; The determination unit a vehicle sensor for detecting the movement of the vehicle, and an actuator for turning the exterior camera at least downward or backward, and a determination is made as to whether or not a child is present outside the vehicle from a plurality of images captured by the turned exterior camera; 4. The vehicle occupant abandonment warning device according to claim 2 or 3.

5. the exterior camera is provided on an exterior member that can be opened and closed in the vehicle; The actuator changes the direction of the exterior camera depending on the open / closed state of the exterior member, and maintains an image including a road surface around the vehicle.

5. The vehicle occupant abandonment warning device according to claim 4.

6. a memory for recording a flag whose value is changed when the determination unit determines that a child has got on or got off the vehicle; The determination unit When the vehicle has finished traveling, when an operation is performed to stop the vehicle or to get off the vehicle, or when only a child is on board, the system determines whether a child has been left behind in the vehicle, referring to the value of the flag when determining whether a child has been left in the vehicle based on a change in the presence or absence of a person or object outside the vehicle in the image captured by the exterior camera; The alarm control unit If the determination unit determines based on the value of the flag that a child has been left in the vehicle, an abandonment warning is output; If the determination unit does not determine based on the value of the flag that a child is left in the vehicle, the determination unit does not output a child leaving warning. The vehicle occupant abandonment warning device according to any one of claims 1 to 5.

Images