Apparatus, method and program for detecting getting off of vehicle

Abstract

To provide an alighting action detection device that prevents occupants in front and rear occupant seats from getting out of a vehicle carelessly, an alighting action detection method, and a program for detecting the alighting action of occupants.SOLUTION: An alighting action detection device 1 includes: an image acquisition unit 10 configured to acquire an image including an occupant captured by a camera 20; a base position determination unit 12 configured to determine a base position of an occupant's hand based on the occupant's sitting posture while a vehicle is running; an area determination unit 13 configured to determine an area for detecting an alighting action based on the position of a door handle and the base position; an alighting action detection unit 14 configured to detect the alighting action when the occupant's hand enters the area based on the image; and an output unit 15 configured to output information indicating the occupant's alighting action.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present invention relates to a technique for detecting the getting-off behavior of passengers in a vehicle.

Background Art

[0002] Conventionally, a monitor device that allows a driver to check the state inside the vehicle (especially the rear seats) has been known. Patent Document 1 describes an invention of a monitor device that displays an image of the rear seat. In this invention, the image display is started on the condition that there is a person in the rear seat. Also, Patent Document 2 describes an invention of a device that detects the seating posture of a fellow passenger. In this invention, it is detected whether the fellow passenger is wearing the seat belt in a correct posture.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Patent Document 2

Summary of the Invention

Problems to be Solved by the Invention

[0004] As described above, conventionally, a system for monitoring the state of fellow passengers and ensuring safety has been proposed. However, when the vehicle stops, there has been no known technology to prevent passengers in the front passenger seat or rear seats (especially children) from getting off without sufficiently checking the surroundings.

[0005] Therefore, in view of the above background, an object of the present invention is to provide a technique for preventing passengers in the front passenger seat or rear seats from getting off carelessly.

Means for Solving the Problems

[0006] The vehicle exit behavior detection device of the present invention includes an image acquisition unit that acquires an image including an occupant captured by a camera, a base position determination unit that determines the base position of the occupant's hand based on the seating posture of the occupant while the vehicle is traveling, a region determination unit that determines a region for detecting an exit behavior based on the position of the door handle and the base position, an exit behavior detection unit that detects an exit behavior when the occupant's hand enters the region or is estimated to enter the region based on the image, and an output unit that outputs information indicating the occupant's exit behavior.

[0007] When an occupant performs an exit behavior, it is necessary to operate the door handle. Therefore, it is possible to detect the occupant's hand from the image captured by the camera and detect the exit behavior based on the position of the occupant's hand. However, if the exit behavior is detected when the occupant touches the door handle, the actions such as alerting based on the detection of the exit behavior will be delayed. Therefore, it is desirable to detect the exit behavior when the occupant is about to place a hand on the door handle.

[0008] In order to detect the exit behavior based on the position of the hand before the occupant's hand reaches the door handle, it is necessary to know how much the hand is moved from the normal posture toward the door handle. However, there has been a problem that it is difficult to determine the position of the occupant's hand in the normal posture. This is because the occupants in the passenger seat and the rear seat do not have the driving task of operating the steering wheel, so the degree of freedom of the position where the hand is placed is high. In addition, the position of the seat varies depending on the vehicle type, and the sitting posture also varies depending on the position of the seat. For this reason, it has been difficult to grasp whether the hand has been moved toward the door handle.

[0009] In the present invention, based on the seating posture of the occupant during vehicle travel, the base position of the occupant's hand is determined. Then, the getting-off behavior detection device determines an area for detecting the getting-off behavior based on the base position and the position of the door handle. Thereby, it is possible to appropriately detect whether the occupant is performing a getting-off behavior based on whether the occupant's hand has entered or is estimated to enter the area. Note that the base position is a position where the occupant's hand is presumed to be normally placed. The seating posture is a concept including the position where the occupant is seated in addition to the posture when seated.

[0010] In the getting-off behavior detection device of the present invention, the base position determination unit may determine the base position based on the position of the occupant's waist and the position of the shoulder on the door side. With this configuration, it is possible to determine the position when the hand is placed on the thigh as the base position.

[0011] In the getting-off behavior detection device of the present invention, the base position determination unit may determine the base position based on an image captured when the vehicle is traveling at a speed equal to or higher than a predetermined threshold value. Here, the threshold value may be, for example, the legal speed, or may be determined based on the legal speed. When the vehicle is traveling smoothly at a constant speed, it is considered that the seating posture of the occupant is also stable.

[0012] The getting-off behavior detection device of the present invention includes a stop detection unit that detects the stop of the vehicle, and the getting-off behavior detection unit may perform detection of the getting-off behavior while the stop detection unit detects the stop of the vehicle. Since the getting-off behavior is usually performed when the vehicle stops, it is desirable to perform detection of the getting-off behavior while detecting the stop of the vehicle. By not performing detection of the getting-off behavior when the vehicle is not stopped, false detection of the getting-off behavior can be reduced.

[0013] The method for detecting alighting behavior of the present invention is a method for detecting the alighting behavior of a passenger by an alighting behavior detection device, comprising: a step of the alighting behavior detection device acquiring an image including the passenger captured by a camera; a step of the alighting behavior detection device determining a base position of the passenger's hand based on the seating posture of the passenger during vehicle travel; a step of the alighting behavior detection device determining a region for detecting the alighting behavior based on the position of the door handle and the base position; a step of the alighting behavior detection device detecting the alighting behavior when the passenger's hand enters the region or is estimated to enter the region based on the image; and a step of the alighting behavior detection device outputting information indicating the alighting behavior of the passenger.

[0014] The program of the present invention is a program for detecting the alighting behavior of a passenger, causing a computer to perform: a step of acquiring an image including the passenger captured by a camera; a step of determining a base position of the passenger's hand based on the seating posture of the passenger during vehicle travel; a step of determining a region for detecting the alighting behavior based on the position of the door handle and the base position; a step of detecting the alighting behavior when the passenger's hand enters the region or is estimated to enter the region based on the image; and a step of outputting information indicating the alighting behavior of the passenger.

Advantages of the Invention

[0015] According to the present invention, it is possible to appropriately determine whether a passenger is performing an alighting behavior.

Brief Description of the Drawings

[0016]

Figure 1

Figure 2

Figure 3

Figure 4

Best Mode for Carrying Out the Invention

[0017] Hereinafter, the getting-off behavior detection device according to the present embodiment will be described with reference to the drawings. Note that the following description is merely an example of a preferred embodiment and is not intended to limit the invention described in the claims.

[0018] FIG. 1 is a diagram showing the configuration of the getting-off behavior detection device 1 according to the embodiment. The getting-off behavior detection device 1 is connected to a camera 20 that photographs the interior of the vehicle, and detects the getting-off behavior of a passenger sitting in the rear seat based on the image photographed by the camera 20. The camera 20 is installed near the rear monitor and acquires an image of the rear seat. In the present embodiment, a passenger in the rear seat is taken as an example, but the getting-off behavior detection device of the present invention can of course also be applied to detecting the getting-off behavior of a passenger in the passenger seat.

[0019] The getting-off behavior detection device 1 includes an image acquisition unit 10, an arithmetic unit 11, a storage unit 16, and an output unit 17. The image acquisition unit 10 has a function of acquiring image data from the camera 20. The arithmetic unit 11 has a function of detecting the getting-off behavior based on the camera image. The arithmetic unit 11 includes a base position determination unit 12, a region determination unit 13, a getting-off behavior detection unit 14, and a stop detection unit 15.

[0020] The base position determination unit 12 determines the base position of the hand of a passenger sitting in the rear seat. As described above, since the position of the hand of a passenger without a driving task has a high degree of freedom, the base position determination unit 12 determines the base position of the hand based on the sitting posture and the sitting position of the passenger. In the present embodiment, the base position is determined based on the position of the shoulder on the door handle side of the passenger and the position of the waist. The position of the shoulder on the door handle side is used as the coordinate in the horizontal direction of the base position, and the position of the waist is used as the coordinate in the vertical direction of the base position.

[0021] The area determination unit 13 has a function of determining an area (hereinafter referred to as the "detection area") for detecting the getting-off behavior based on the base position and the position of the door handle. Specifically, an area centered on the door handle and having a radius of X% of the distance between the position of the door handle and the base position is determined as the detection area.

[0022] FIG. 2 is a diagram showing the detection area R. For a circle C centered on the door handle H and having a radius of the distance L between the position of the door handle H and the base position B, the area with a radius of L×X(%) is the detection area R. The position of the door handle H can be initially set based on the shooting direction when the camera 20 is installed. Here, a circular area centered on the door handle is used as the detection area R, but the detection area R does not have to be circular. For example, an elliptical area including the straight line connecting the door handle H and the base position B may be used.

[0023] If the coefficient X for determining the radius of the detection area R is a large value, the chance of missing the getting-off behavior is reduced, but the false detection also increases. The coefficient X needs to be set appropriately, and for example, 60% - 70% is desirable. The coefficient X may be changed according to the situation when the vehicle is parked. For example, when parked on the road, a larger value may be set than when parked in a home parking lot. This is because other vehicles are running on the road, so the degree of caution required is high. The area determination unit 13 stores the data of the determined area in the storage unit 16.

[0024] The getting-off behavior detection unit 14 has a function of detecting whether a passenger is performing a getting-off behavior based on the image acquired by the image acquisition unit 10. The getting-off behavior detection unit 14 detects the getting-off behavior of the passenger when the hand of the passenger shown in the image sequentially acquired by the shooting of the camera 20 enters the detection area. The getting-off behavior detection unit 14 performs the detection process of the getting-off behavior when the parking detection unit 15 detects the parking of the vehicle.

[0025] The parking detection unit 15 receives data on the traveling speed of the vehicle from the vehicle speed sensor 21 and determines whether the vehicle is stopped. The parking detection unit 15 detects that the vehicle is stopped when the vehicle speed is 0. In this specification, "stopping" is a concept that includes parking.

[0026] The output unit 17 has a function of outputting to the in-vehicle device that there has been a getting-off behavior. In the present embodiment, the output unit 17 outputs data indicating that there has been a getting-off behavior to the speaker 22 or the locking device 23.

[0027] When a getting-off behavior is detected, various applications can be considered. For example, it is possible to notify the driver through the speaker 22 that a passenger in the rear seat is about to get off. Also, it is possible to alert the passenger in the rear seat to carefully check the outside of the vehicle. As another example, the rear seat door may be locked by the locking device 23 so that it cannot be opened. The processing performed in response to the detection of the getting-off behavior may be switched according to the age of the passenger in the rear seat. When the age of the passenger is small, notification is made to the driver or the door is locked, and when the age of the passenger is relatively large, the passenger who is about to get off is alerted.

[0028] Figure 3 is a diagram showing the operation of the getting-off behavior detection device 1 of the embodiment for determining the detection area. The getting-off behavior detection device 1 acquires an image from the camera 20 (S10). The getting-off behavior detection device 1 acquires data on the speed of the vehicle from the vehicle speed sensor 21 and determines whether the vehicle speed is equal to or higher than a predetermined threshold (S11). If the vehicle speed is less than the predetermined threshold (NO in S11), the process returns to the process of acquiring an image from the camera 20 (S10).

[0029] When the vehicle speed is equal to or higher than a predetermined threshold value (YES in S11), the getting-off behavior detection device 1 records the image at that time (S12). The getting-off behavior detection device 1 determines whether a predetermined amount of images have been accumulated (S13). If the predetermined amount of images (for example, for 5 minutes) have not been accumulated (NO in S13), the above-described processing (S10 to S12) is repeated. When the predetermined amount of images have been accumulated (YES in S13), the getting-off behavior detection device 1 determines the base position of the passenger's hand based on the accumulated images (S14), determines a detection area for determining the getting-off behavior based on the base position and the position of the door handle, and stores it in the storage unit 16 (S15). Note that the detection area may be stored in the storage unit 16 in association with the passenger, and it is not necessary to perform the operation shown in FIG. 3 every time.

[0030] FIG. 4 is a diagram showing the operation of the getting-off behavior detection device 1 according to the embodiment for detecting the getting-off behavior. The getting-off behavior detection device 1 acquires an image from the camera 20 (S20). The getting-off behavior detection device 1 acquires data on the speed of the vehicle from the vehicle speed sensor 21 and determines whether the vehicle is stopped (S21). When the vehicle is stopped (YES in S21), it is determined whether the hand of the passenger shown in the image of the camera 20 is within the detection area (S22). When the hand of the passenger is within the detection area (YES in S22), the getting-off behavior detection device 1 detects that the passenger is performing a getting-off behavior and outputs data indicating that there is a getting-off behavior (S23). When the hand of the passenger is not within the detection area (NO in S22), the process returns to the process of acquiring the image of the camera 20 (S20).

[0031] As described above, the configuration of the getting-off behavior detection device 1 according to the present embodiment has been described. However, the example of the hardware of the getting-off behavior detection device 1 described above is a computer including an ECU, a RAM, a ROM, a hard disk, a display, a communication interface, and the like. A program having modules for realizing the above-described functions is stored in the RAM or the ROM, and the getting-off behavior detection device 1 described above is realized by executing the program by the ECU. Such a program is also included in the scope of the present invention.

[0032] The getting-off behavior detection device 1 according to this embodiment determines the base position of the hand of an occupant in the rear seat while the vehicle is running, and determines a detection area for detecting the getting-off behavior based on the base position and the position of the door handle. Thereby, the getting-off behavior of the occupant in the rear seat can be appropriately detected at an early stage. Further, according to this embodiment, regardless of the vehicle type and the seat position, the getting-off behavior can be detected with a small amount of learning data even considering machine learning or the like.

[0033] In the above-described embodiment, an example of the configuration of the getting-off behavior detection device has been described, but the getting-off behavior detection device of the present invention can be variously modified. Modification examples will be described below.

[0034] (Timing of getting-off behavior detection) In the above-described embodiment, the getting-off behavior is detected when the vehicle is stopped, but the getting-off behavior may be detected not only when the vehicle is completely stopped but also when the speed of the vehicle reaches a predetermined threshold value (for example, 4 km / h or less). When driving at a low speed, it is possible to get off if one wants to, but it is desirable to prevent such forced getting off.

[0035] Further, when the door handle of the rear seat is locked so as not to be opened by the child lock, the getting-off behavior may not be detected.

[0036] (Target of getting-off behavior detection) When the occupant sitting in the rear seat is an adult, it is also possible to adopt a configuration in which the getting-off behavior is not detected. The determination as to whether the occupant is an adult may be made by machine learning or may be determined by the height of the head.

[0037] (Detection of getting-off behavior) In the above-described embodiment, an example in which the getting-off behavior is detected when the hand of the occupant enters the detection area has been described, but it may be detected that there is a getting-off behavior only when the state in which the hand of the occupant has entered the detection area continues for a predetermined number of frames (for example, 3 frames). Thereby, false detection of the getting-off behavior can be reduced.

[0038] Furthermore, even if the occupant's hands are not in the detection area, the getting-off behavior detection device may detect that the occupant is getting-off behavior when it is estimated that the occupant is moving toward the detection area and entering the detection area from the time-series images of the camera 20. Note that machine learning such as a Kalman filter or dynamic Bayesian may be used to analyze the time-series images.

[0039] (Determining the door handle position) In the above embodiment, the door handle position is initially set, but the door handle position may be determined based on a camera image. For example, the position of the occupant's hand when the door is opened may be set as the door handle position.

[0040] (Determining the base position) In the above embodiment, an example was given in which the base position of the occupant's hands was determined based on the positions of the occupant's shoulders and waist, but the base position may be determined by other methods. For example, the position of the occupant's eyes or ears on the door handle side may be used instead of the shoulder position. Also, the position of the seat may be used instead of the waist position. [Explanation of symbols]

[0041] 1. Exit behavior detection device 10 Image acquisition unit 11 Arithmetic section 12 Base position determination unit 13 Area determination part 14. Exit behavior detection unit 15 Stop detection unit 16 Memory section 17 Output section 20 Camera 21 Vehicle speed sensor 22 Speaker 23 Locking device B Base position H Handle R Detection area

Claims

1. An image acquisition unit that acquires an image including an occupant photographed by a camera; A base position determination unit that determines the base position of the occupant's hand based on the seating posture of the occupant while the vehicle is in motion; An area determination unit that determines an area for detecting a getting-off action based on the position of the door handle and the base position; A getting-off action detection unit that detects a getting-off action when the occupant's hand enters the area or is estimated to enter the area based on the image; An output unit that outputs information indicating the getting-off action of the occupant; A getting-off action detection device comprising the above.

2. The getting-off action detection device according to claim 1, wherein the base position determination unit determines the base position based on the position of the occupant's waist and the position of the shoulder on the door side.

3. The getting-off action detection device according to claim 1 or 2, wherein the base position determination unit determines the base position based on an image taken when the vehicle is traveling at a speed equal to or higher than a predetermined threshold.

4. Comprising a parking detection unit that detects the parking of the vehicle, The getting-off action detection device according to any one of claims 1 to 3, wherein the getting-off action detection unit detects a getting-off action while the parking detection unit detects the parking of the vehicle.

5. A method for detecting a getting-off action of an occupant by a getting-off action detection device, comprising: A step in which the getting-off action detection device acquires an image including an occupant photographed by a camera; A step in which the getting-off action detection device determines the base position of the occupant's hand based on the seating posture of the occupant while the vehicle is in motion; A step in which the getting-off action detection device determines an area for detecting a getting-off action based on the position of the door handle and the base position; A step in which the getting-off action detection device detects a getting-off action when the occupant's hand enters the area or is estimated to enter the area based on the image; A step in which the getting-off action detection device outputs information indicating the getting-off action of the occupant; A getting-off action detection method comprising the above.

6. A program for detecting a getting-off action of an occupant, which causes a computer to: Acquire an image including an occupant photographed by a camera; Determine the base position of the occupant's hand based on the seating posture of the occupant while the vehicle is in motion; Determine an area for detecting a getting-off action based on the position of the door handle and the base position; Based on the image, detecting a getting-off behavior when the occupant's hand enters the area or when it is estimated to enter the area; Outputting information indicating the occupant's getting-off behavior; A program for causing the above to be executed.

Images