Vehicle interior monitoring system

Abstract

To improve an accuracy of a determination on whether an object is in a prohibition area.SOLUTION: An in-cabin monitoring system includes: an imaging apparatus that is disposed in an intermediate position in at least one height direction of a front end portion or a rear end portion in a vehicle longitudinal direction of a prohibition area, is disposed above a reference object (hand rail 20) disposed of an inner side in a vehicle width direction with respect to a vehicle side wall 22 and above a prohibition area A, and captures images of the inside of a cabin including the prohibition area A and the reference object; and an analysis apparatus that analyzes a captured image by the imaging apparatus. The analysis apparatus determines that a part of an object intrudes into the prohibition area A when located in front of the reference object, and there is no intrusion into the prohibition area A when located behind the reference object.SELECTED DRAWING: Figure 6

Description

[Technical field]

[0001] The present disclosure relates to an in-vehicle monitoring system that sets a no-entry area near a vehicle door and determines whether an object such as a passenger is entering the no-entry area. [Background technology]

[0002] In order to ensure the smooth operation of vehicles, systems have been proposed that achieve various functions in addition to simply running the vehicle, one of which is a system that monitors the status of passengers. In particular, in the case of public buses, doors are opened and closed when passengers get on and off at bus stops. When opening and closing the doors, it is desirable to monitor the movements of passengers. In particular, for autonomous vehicles, a conductor system that monitors the status of passengers, rather than an actual conductor, is desirable, and it is desirable to be able to determine whether the doors should be opened or closed remotely.

[0003] Therefore, Patent Document 1 discloses a method of installing cameras that capture images of the interior of the vehicle and monitor the behavior of passengers. In particular, multiple cameras are installed on the ceiling to eliminate blind spots.

[0004] In addition, in Patent Document 2, markers indicating dangerous areas are provided on the platform to detect intrusion into the dangerous areas. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] JP 2020-3935 A [Patent Document 2] Patent Publication No. 2021-71842 Summary of the Invention [Problem to be solved by the invention]

[0006] Here, by providing multiple cameras as in Patent Document 1, it is possible to increase the amount of information and improve the accuracy of object recognition by image analysis. However, this increases the amount of equipment and costs.

[0007] In Patent Document 2, although the position of an object on a platform can be accurately recognized, there is a problem with the recognition of the position in the space. [Means for solving the problem]

[0008] The vehicle interior monitoring system according to the present disclosure is an interior monitoring system for determining whether an object has entered a prohibited area (hereinafter, referred to as the prohibited area) set near a vehicle door, and includes a reference object disposed at a middle position in the height direction of at least one of the front end or rear end in the vehicle longitudinal direction of the prohibited area and disposed inside the vehicle width direction from a vehicle side wall, an imaging device disposed above the prohibited area for capturing an image of the interior of the vehicle interior including the prohibited area and the reference object, and an analysis device for analyzing an image captured by the imaging device. The analysis device determines that the object has entered the prohibited area when a part of the object is located in front of the reference object, and that the object has not entered the prohibited area when a part of the object is located behind the reference object.

[0009] The imaging device may be attached to a ceiling above the prohibited area and positioned biased toward the front or rear of the vehicle in the longitudinal direction, and the reference object may be installed on the side farther from the imaging device.

[0010] The reference object may be a handrail. Effect of the Invention

[0011] Even for an image from a single imaging device, the accuracy of determining whether or not the image is within a prohibited area can be improved. [Brief description of the drawings]

[0012] [Figure 1] 1 is a diagram showing an example of the external appearance of a vehicle 10 equipped with a vehicle interior monitoring system according to an embodiment of the present invention. [Diagram 2] FIG. [Diagram 3] 1A to 1C are diagrams showing an example of the configuration of a handrail 20, where (a) is a plan view, (b) is a front view, and (c) is a side view. [Figure 4] FIG. 2 is a block diagram showing a configuration of a monitoring device 40. [Diagram 5] 13A shows the photographed state, and FIG. 13B shows the state where the arms of the occupant P are not beyond the handrail 20. FIG. [Figure 6] This figure shows and explains the judgment made by the embodiment as viewed from inside the vehicle cabin, where (a) shows a state in which the arm of occupant P is beyond the handrail 20, and (b) shows a state in which the arm of occupant P is not beyond the handrail 20. [Figure 7] 10A to 10C are diagrams showing another configuration example of the handrail 20, where (a) is a plan view, (b) is a front view, and (c) is a side view. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0013] Hereinafter, embodiments of the present disclosure will be described with reference to the drawings. Note that the following embodiments do not limit the present disclosure, and configurations obtained by selectively combining multiple examples are also included in the present disclosure.

[0014] <Vehicle configuration> 1 shows an example of the exterior of a vehicle 10 equipped with an in-vehicle monitoring system according to this embodiment. The vehicle 10 shown here is used, for example, as a public vehicle. For example, the vehicle 10 travels along a set route and stops at bus stops along the route.

[0015] The vehicle 10 may be, for example, an electric vehicle (BEV, Battery electric vehicle) using a rotating electric machine (not shown) as a drive source, or a vehicle using an internal combustion engine as a drive source. The vehicle 10 may be manually driven, but this example is directed to an autonomous vehicle that can be remotely managed.

[0016] A pair of double-hinged doors 12, 12 are provided on the side of the vehicle 10 (the left side when facing forward in this example). The doors 12, 12 are, for example, outside sliding doors, and are located on the outside of the vehicle side when open.

[0017] 2 is a plan view showing the interior of the vehicle. A seat 14 is provided on the right side (opposite the door 12) and in the rear of the vehicle interior, and a driver's seat 16 is provided on the front side. The driver's seat 16 is used during manual driving and may not be used during automatic driving. When the door 12 is open, it moves to the position shown by the dashed line in the figure.

[0018] A prohibited area A is provided on the interior side of the door 12 where the door 12 is prohibited from opening or closing if there are passengers or the like (shown as sand in the figure). If there are passengers, luggage, or other objects in this prohibited area A, the door 12 is not opened or closed and an alarm or the like is sounded to encourage the passengers to move.

[0019] A camera 30 is provided on the ceiling behind the prohibited area A. This camera 30 is a single imaging device that captures the entire interior of the vehicle, including the inside of the door 12, and is configured as a wide-angle camera.

[0020] A handrail 20 is provided at the front-rear boundary of the prohibited area A. The handrail 20 protrudes inward from a side wall 22 of the vehicle.

[0021] 3 is a diagram showing an example of the configuration of the handrail 20. As described above, the upper and lower ends of the handrail 20 are attached to the vehicle side wall 22, extend inward from both ends, and extend in the vertical direction in parallel with the vehicle side wall 22 in the middle.

[0022] 4 is a block diagram showing the configuration of the monitoring device 40. The monitoring device 40 and the handrail 20 serving as a reference object together constitute a vehicle interior monitoring system.

[0023] As described above, the camera 30 is attached above the prohibited area A inside the vehicle cabin, and captures images of the inside of the vehicle cabin. The camera 30 is connected to the control unit 42, and the captured images are supplied to the control unit 42.

[0024] A door opening / closing device 44 is connected to the control unit 42, and the control unit 42 controls the opening and closing of the doors. A communication unit 46 is also connected to the control unit 42, and the communication unit 46 can connect to various sites via various communication lines. In particular, the control unit 42 can connect to a management computer at a management center, and can provide information such as the status inside the vehicle and the running status of the vehicle, as well as receive control commands regarding running and control commands regarding the opening and closing of the doors.

[0025] A speaker 48 is connected to the control unit 42, and can output voices such as warnings and attention calls to the occupants from here. It is preferable to provide a display as an output device for warnings and the like.

[0026] The control unit 42 has an analysis unit 50, which analyzes the image of the inside of the vehicle compartment supplied from the camera 30 and determines whether or not an occupant or the like is present in the prohibited area A.

[0027] “Judgment details” 5 is a diagram showing the photographed state, where (a) shows a state in which the arm of the occupant P is hanging over the handrail 20, and (b) shows a state in which the arm of the occupant P is not hanging over the handrail 20. Note that the camera 30 is a wide-angle camera and the image is significantly distorted, but in the figure, straight lines are drawn as straight lines to avoid distortion.

[0028] The analysis unit 50 recognizes the occupant P and the handrail 20 by image analysis. Here, it is preferable that the handrail 20 is recognized and stored in advance from a camera image when there is no occupant. This recognition may be performed with reference to design data of the handrail 20. In addition, it is preferable that the prohibited area A is also recognized and stored in advance.

[0029] Then, the analysis unit 50 recognizes the position of the arm based on whether the arm is in front or behind the handrail 20. In this way, it is possible to determine whether the occupant P or other objects are in the prohibited area A based on the handrail 20, so that a relatively accurate determination can be made from a single camera image.

[0030] 6A and 6B are diagrams illustrating the state of the determination according to the embodiment as viewed from inside the vehicle cabin, where (a) shows a state in which the arm of the occupant P is beyond the handrail 20, and (b) shows a state in which the arm of the occupant P is not beyond the handrail 20. In the above description, the prohibited area A is shown in a two-dimensional form, but in reality it is a three-dimensional space, for example, a rectangular parallelepiped area extending in the vertical direction. Note that it does not necessarily have to be a rectangular parallelepiped, and it may be an elliptical cylindrical shape or an area whose cross-sectional area varies with the vertical height, for example, an area whose central portion in the vertical direction is bulging or denting.

[0031] When there is only one camera 30 and an image is taken from above, the field of view spreads downward as shown by the dashed line in the figure. Therefore, in the two-dimensional image obtained by the camera 30, the higher an object is located, the farther it is projected and positioned from the prohibited area A, and the more likely it is to be determined to be outside the prohibited area A.

[0032] In this embodiment, a handrail 20 is placed as a reference object for determining the side of prohibited area A. Therefore, when inside this handrail 20, i.e. when the handrail 20 is hidden, it can be determined that the object is inside prohibited area A, and when on the other side of the handrail 20, it can be determined that the object is outside prohibited area A. Therefore, a correct determination can be made at least when the handrail 20 indicates the boundary of prohibited area A.

[0033] In this way, according to this embodiment, it is possible to prevent unnecessary objects from being captured by installing a reference object such as the handrail 20 at the middle position in the height direction of the boundary of the prohibited area A. Therefore, the camera 30 installed in one location can accurately recognize the presence of people, luggage, etc. in the prohibited area A near the door 12.

[0034] Also, a seat 14 may be formed at the rear of the vehicle 10, which may be one step higher. A screen board may be installed between the door 12 and the seat at the rear of the door 12. In such a case, the screen board serves as a means for preventing intrusion into the prohibited area A, making it unnecessary to monitor this boundary. In this case, one camera 30 is preferably installed above the prohibited area A, but on the rear side, so that the handrail 20 on the front side can be reliably monitored. If a screen board is installed on the front side, it is preferable to install the camera 30 closer to the front.

[0035] In the above example, the handrail 20 is used as the reference object. However, the reference object is not limited to the handrail 20 and may be another object. For example, a partition board, a hanging strap, a grip, etc. may be used.

[0036] The target is not limited to passengers, but may be luggage or the like.

[0037] "Other examples of handrails 20" 7 is a diagram showing another example of the configuration of the handrail 20. In this example, as shown in a front view seen from inside the vehicle compartment perpendicular to the vehicle side wall 22, the handrail 20 is curved in the front-to-rear direction of the vehicle. After protruding into the vehicle compartment, it extends diagonally downward and forward, then extends in the vertical direction, then extends rearward, and extends toward the vehicle side wall 22.

[0038] Then, by making the part extending in the vertical direction coincide with the boundary of the prohibited area A, this part can be made to function as a reference object. [Explanation of symbols]

[0039] 10 vehicle, 12 door, 14 seat, 16 driver's seat, 20 handrail, 22 vehicle side wall, 30 camera, 40 monitoring device, 42 control unit, 44 door opening / closing device, 46 communication unit, 48 speaker, 50 analysis unit.

Claims

1. A vehicle interior monitoring system that sets a prohibited area near a door of a vehicle and determines whether an object has entered the prohibited area, a reference object that is disposed at a middle position in the height direction of at least one of the front end portion and the rear end portion of the prohibited area in the vehicle longitudinal direction and that is disposed on the inner side in the vehicle width direction than the vehicle side wall; an imaging device that is disposed above the prohibited area and captures an image of the interior of the vehicle cabin including the prohibited area and the reference object; an analysis device that analyzes an image captured by the imaging device; Including, The analysis device includes: When a part of the object is located in front of the reference object, it is determined that the object has entered a restricted area, and when a part of the object is located behind the reference object, it is determined that the object has not entered the restricted area. In-vehicle monitoring system.

2. 2. The vehicle interior monitoring system according to claim 1, The imaging device is attached to a ceiling above the prohibited area and is disposed biased toward the front or rear side in a vehicle longitudinal direction, and the reference object is disposed on a side farther from the imaging device. In-vehicle monitoring system.

3. 3. The vehicle interior monitoring system according to claim 1, The reference object is a handrail. In-vehicle monitoring system.

Images