Disembarking assistance device

Abstract

To provide a passenger disembarking assistance device that can suppress collisions between passengers and moving objects after disembarking. [Solution] The disembarking assistance device 1 detects disembarking passengers 43A, 43B and passersby 46 located around the vehicle 2 as pedestrians. The disembarking assistance device 1 also detects other vehicles 45 moving around the vehicle as moving objects. After the occupants disembark from the vehicle 2, the device is configured to issue a warning based on the proximity of detected pedestrians and moving objects. This allows the occupants to be notified of the approaching moving object after disembarking.

Description

Technical Field

[0001] The present invention relates to a technology for assisting a passenger who has alighted from a vehicle.

Background Art

[0002] The following Patent Document 1 describes a disembarkation support device that supports a passenger when the passenger gets out of the vehicle. The disembarkation support device in Patent Document 1 detects the disembarkation operation of a passenger who is about to get out of the vehicle from the door of the vehicle based on the image data of an in-vehicle camera. When the disembarkation support device detects the disembarkation operation, it issues a warning signal according to the distance between the host vehicle and other vehicles and the moving speed of the other vehicles.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In the disembarkation support device of Patent Document 1 above, when a passenger gets off the vehicle, if another vehicle approaches the host vehicle, a warning to the other vehicle is executed. However, even if the state is safe during disembarkation, after the passenger has disembarked, there is a possibility that another vehicle may approach the disembarked passenger while moving around the vehicle. Therefore, a technology that can provide some support for the passenger after disembarkation is desired.

[0005] The present invention has been made to solve the above-mentioned conventional problems, and an object thereof is to provide a passenger support device that can suppress the occurrence of a collision between a passenger after disembarkation and a moving object.

Means for Solving the Problems

[0006] To achieve the above objective, the passenger disembarking assistance device according to the present invention comprises: a pedestrian detection unit for detecting pedestrians located around a vehicle; a moving object detection unit for detecting moving objects other than pedestrians moving around the vehicle; and a warning unit for issuing a warning after an occupant has disembarked from the vehicle, based on the proximity of the pedestrian detected by the pedestrian detection unit and the moving object detected by the moving object detection unit. In this specification, the term "pedestrian" is not limited to passengers who have disembarked from a vehicle, but also includes passersby walking around a vehicle. Furthermore, the term "pedestrian" is not limited to people walking, but also includes people in wheelchairs, people pushing strollers, babies in strollers, etc. Therefore, the term "pedestrian" may include passengers who have disembarked from a vehicle and are moving in wheelchairs, or passengers pushing strollers. In addition, the term "moving object" is a concept that includes other vehicles, motorcycles, bicycles, electric scooters and other vehicles, and items that are operated and moved by people, such as shopping carts and trolleys. [Effects of the Invention]

[0007] According to the disembarking assistance device of the present invention having the above configuration, pedestrians and other moving objects around the vehicle are detected. After the occupants disembark, a warning is issued if a pedestrian and a moving object approach each other. This makes it possible to avoid a collision if at least one of the pedestrians or the person on the moving object is unaware of the approach of the other. Therefore, the occupants after disembarking can be monitored as pedestrians, and the occurrence of collisions between occupants and moving objects can be suppressed. [Brief explanation of the drawing]

[0008] [Figure 1] This is a schematic diagram of the vehicle according to this embodiment. [Figure 2] This is a block diagram showing the configuration of the passenger disembarkation assistance device according to this embodiment. [Figure 3] This is a flowchart of the first warning control processing program according to this embodiment. [Figure 4] This diagram shows the state in which a warning is issued in the first warning mode. [Figure 5] This diagram shows the state in which no warning is issued in the first warning mode. [Figure 6] This is a flowchart of the second warning control processing program according to this embodiment. [Figure 7] This diagram shows the state in which a warning is issued in the second warning mode. [Figure 8] This diagram shows the state in which no warning is issued in the second warning mode. [Modes for carrying out the invention]

[0009] Hereinafter, one embodiment of the passenger disembarking assistance device according to the present invention will be described in detail with reference to the drawings. First, the vehicle 2 equipped with the passenger disembarking assistance device 1 according to this embodiment will be described below. Figure 1 is a schematic diagram of the vehicle 2 according to this embodiment. Figure 2 is a block diagram of the passenger disembarking assistance device 1 according to this embodiment. In the following description, the front-rear direction, the left-right direction, and the up-down direction of the vehicle 2 will be simply referred to as the front-rear direction, the left-right direction, and the up-down direction, respectively. In addition, the reference numeral R may be used for devices etc. on the right side of the vehicle 2, and the reference numeral L may be used for devices etc. on the left side of the vehicle 2. Furthermore, in addition to the components shown in Figures 1 and 2, the vehicle 2 is equipped with other basic components as a vehicle 2, but in the following description, the configuration related to the control that executes a warning based on the approach of pedestrians and moving objects around the vehicle, and the control related to said configuration will be mainly described.

[0010] As shown in Figure 1, vehicle 2 is, for example, a vehicle with a steering wheel 3 on the right side, and comprises a body 11, a driver's side front door 12R, a passenger side front door 12L, a driver's side rear door 13R, a passenger side rear door 13L, and a back door 14. Hereafter, when the front doors 12R, 12L, rear doors 13R, 13L, and back door 14 are described collectively, they may be referred to as "each door." Each door is, for example, a swing-type door.

[0011] Furthermore, as shown in Figures 1 and 2, the passenger exit assistance device 1 includes a front camera 5, side cameras 6R and 6L, a rear camera 7, various sensors 8, an in-vehicle camera 9, a vehicle control ECU (Electronic Control Unit) 10, a speaker control device 15, and a projection control device 16. Hereinafter, the front camera 5, side cameras 6R and 6L, and rear camera 7 may be collectively referred to as external cameras.

[0012] The external camera is an imaging device having a solid-state image sensor such as a CCD, and it captures images of the area around the vehicle. The front camera 5 is mounted, for example, above the front bumper of the vehicle 2 or behind the rearview mirror, and is installed with its optical axis facing forward of the vehicle 2. The side cameras 6R and 6L are mounted, for example, on the left and right side mirrors of the vehicle 2, and are installed with their optical axes facing sideways of the vehicle 2. The rear camera 7 is mounted, for example, above the license plate on the rear of the vehicle 2, and is installed with its optical axis facing backward of the vehicle 2.

[0013] The various sensors 8 are sensors that realize various functions of the vehicle 2. For example, sensors 8 can include ultrasonic sensors, millimeter-wave radar, and laser sensors, which can be used to detect people and objects around the vehicle. Alternatively, sensors 8 can include vehicle speed sensors, acceleration sensors, gyro sensors, steering sensors, and shift position sensors, which can be used to assist the vehicle 2 when it is driving or when it is stopped.

[0014] The in-vehicle camera 9, like the exterior camera, is an imaging device having a solid-state image sensor such as a CCD. The in-vehicle camera 9 is, for example, an omnidirectional camera (360-degree camera) and is mounted in the center of the roof in the front-to-back and left-to-right directions. The in-vehicle camera 9 is mounted on the inside of the roof and is capable of imaging the entire interior of the vehicle.

[0015] Furthermore, a speaker 17 is mounted on the roof of vehicle 2. The speaker 17 is configured to emit sound in all 360 degrees based on the control of the speaker control device 15, for example. This allows warnings to be issued to pedestrians and moving objects around the vehicle, as will be described later. In addition, the speaker 17 can control the direction in which it emits sound based on the control of the speaker control device 15. Therefore, the speaker 17 can emit sound from vehicle 2 in a specific direction.

[0016] Vehicle 2 is also equipped with multiple projection devices 18A to 18C. Each projection device 18A to 18C is equipped with a light source such as an LED and projects shapes or characters onto the road surface to warn pedestrians and moving objects around the vehicle. Projection device 18A is provided, for example, at the positions of the two headlights, and projection device 18B is provided at the positions of the two taillights. Projection devices 18A and 18B may also be light-emitting devices that are used in conjunction with these lights. Projection devices 18C are provided on both the left and right sides of vehicle 2. A pair of projection devices 18C are provided, for example, at positions between the front door 12R and the rear door 13R, and between the front door 12L and the rear door 13L. Based on the control of the projection control device 16, the multiple projection devices 18A to 18C can project shapes and the like in all directions around vehicle 2. Furthermore, each of the multiple projection devices 18A to 18C is provided with a mechanism for changing the projection direction (such as an actuator for changing the optical axis). The projection devices 18A to 18C are controlled based on the control of the projection control device 16, which allows them to project figures and other images from the vehicle 2 in a specific direction.

[0017] The configuration of the vehicle 2 shown in FIGS. 1 and 2 is an example. For example, the vehicle 2 is not limited to a vehicle having a steering wheel 3 on the right side, and may also be a vehicle having a steering wheel 3 on the left side. Further, the vehicle 2 may be an internal combustion engine vehicle having an internal combustion engine (such as an engine) as a drive source, an electric vehicle having an electric motor as a drive source, a fuel cell vehicle, etc., or a hybrid vehicle having a plurality of such drive sources. Further, the vehicle type, the number of wheels, etc. of the vehicle 2 are not particularly limited. Further, the vehicle 2 may be a vehicle capable of manual driving, a vehicle capable of automatic driving, or a vehicle capable of switching between both types of driving.

[0018] Also, the mounting position, number, type of camera, etc. of the external camera and the in-vehicle camera 9 are examples. For example, the vehicle 2 may be configured to include a plurality of in-vehicle cameras 9. Further, the vehicle 2 may be configured not to include an external camera or an in-vehicle camera 9. In this case, the vehicle 2 may be configured to detect pedestrians around the vehicle using a millimeter-wave radar or the like.

[0019] Also, the configuration of the speaker 17 described above is an example. The speaker 17 may be configured to be able to emit sound only in a specific direction. Further, the speaker 17 may be provided with an actuator or the like for changing the direction in which the speaker 17 emits sound. Further, the speaker 17 may be configured to be provided at a plurality of locations in the vehicle 2, for example, at the front bumper, rear bumper, roof, side sill, etc. respectively. Also, the sound emitted from the speaker 17 is not limited to sound, and may also be a horn sound or a buzzer sound. Therefore, the speaker 17 may be configured to be shared by a speaker for executing a warning and a speaker for emitting a horn sound. Further, the vehicle 2 may be configured not to include the speaker 17.

[0020] Also, the configuration of the projection devices 18A to 18C described above is an example. The vehicle 2 may be configured to include only one of the projection devices 18A to 18C. Further, the projection devices 18A to 18C may be configured not to include a mechanism for changing the projection direction, such as an actuator. For this reason, the projection devices 18A to 18C may be configured to be able to project only in a specific direction. Further, the vehicle 2 may be configured not to include the projection devices 18A to 18C.

[0021] (Regarding the vehicle control ECU10) The vehicle control ECU (hereinafter simply referred to as ECU) 10 is an electronic control unit that comprehensively controls the entire vehicle 2, including the passenger disembarkation assistance device 1. It includes a CPU 31 as an arithmetic device and control device, a RAM 32 used as working memory when the CPU 31 performs various arithmetic processes, a ROM 33 that stores control programs as well as the first warning control processing program (see Figure 3) and the second warning control processing program (see Figure 6), which will be described later, and an internal storage device such as a flash memory 34 that stores programs and flag values ​​read from the ROM 33.

[0022] The ECU 10 implements various functional units by executing a program on the CPU 31. For example, the pedestrian detection unit 31A is a functional unit that detects pedestrians located around the vehicle 2. The moving object detection unit 31B is a functional unit that detects moving objects other than pedestrians that are moving around the vehicle 2. The warning unit 31C is a functional unit that issues a warning based on the fact that a pedestrian detected by the pedestrian detection unit 31A and a moving object detected by the moving object detection unit 31B are approaching each other after an occupant has disembarked from the vehicle 2. The occupant detection unit 31D is a functional unit that detects occupants of the vehicle 2. In other words, the vehicle control ECU 10 is an example of the pedestrian detection unit, moving object detection unit, warning unit, and occupant detection unit as described herein.

[0023] Furthermore, the ECU 10 is connected to the aforementioned external cameras (forward camera 5, etc.), various sensors 8, interior camera 9, speaker control device 15, and projection control device 16 via an in-vehicle network such as CAN. The ECU 10 performs various calculations based on the information input from the external cameras, each sensor 8, and interior camera 9 to control the vehicle 2. For example, based on the image data captured by the external cameras, the ECU 10 displays bird's-eye view images and overhead view images on the vehicle 2's monitor (not shown) to provide driving assistance.

[0024] Furthermore, the ECU 10 drives the speaker control device 15 to emit sound from the speaker 17. The ECU 10 also drives the projection control device 16 to project graphics and other images from the projection devices 18A to 18C. Note that the sound emission from the speaker 17 and the projection from the projection devices 18A to 18C may be performed by a device other than the ECU 10, such as a car navigation system.

[0025] (Regarding the first and second warning control processing programs) Next, the first and second warning control processing programs executed by the ECU 10 in the passenger exit assistance device 1 having the above configuration will be described. The vehicle 2 of this embodiment includes, for example, a first warning mode in which the first warning control processing program is executed, and a second warning mode in which the second warning control processing program is executed. The ECU 10 can change the mode it is executed in via the touch panel of the car navigation system or the like. First, the first warning mode will be described.

[0026] (Regarding the first warning mode) Figure 3 is a flowchart of the first warning control processing program according to this embodiment. Here, the ECU 10 starts the first warning control processing program when it detects, for example, that the vehicle 2 has stopped from a moving state. The first warning control processing program is a program that executes a warning based on the proximity of pedestrians and moving objects around the vehicle until the occupants of the vehicle 2 have disembarked and have moved a predetermined distance away from the vehicle 2. In the following description, the case in which "pedestrians located around the vehicle" in this specification is taken to mean occupants who have disembarked from the vehicle 2 (disembarking persons) and passersby who are not in the vehicle 2 but are walking around the stopped vehicle 2 will be described. In the first warning mode, a warning is executed when the "pedestrians located around the vehicle" in this specification are disembarking persons. In the second warning mode, a warning is executed when the "pedestrians located around the vehicle" in this specification are disembarking persons and passersby. Furthermore, the case in which a vehicle other than vehicle 2 (hereinafter referred to as "other vehicle") is taken to mean "moving objects other than pedestrians moving around the vehicle" in this specification will be described.

[0027] Therefore, in this specification, the term "pedestrian" is not limited to passengers who have disembarked from vehicle 2, but also includes passersby walking around the vehicle. Furthermore, pedestrians are not limited to people walking. For this reason, pedestrians may include passengers or passersby using wheelchairs, or passengers or passersby pushing strollers. In addition to other vehicles, moving objects may include motorcycles, bicycles, electric scooters, shopping carts, trolleys, and other vehicles operated by people. For example, if a passenger disembarks from vehicle 2 and rides a bicycle, that passenger may be considered a pedestrian. Also, passersby riding bicycles around the vehicle may be considered "moving objects other than pedestrians moving around the vehicle." Therefore, pedestrians and moving objects may be the same vehicle.

[0028] Furthermore, the conditions for initiating the execution of the first warning control program are not limited to those described above. The ECU 10 may initiate the process shown in Figure 3 when the shift lever is moved to the parking position after the vehicle 2 has stopped. The ECU 10 may also initiate the process shown in Figure 3 when it detects that the engine of the vehicle 2 has stopped. The ECU 10 may also initiate the process shown in Figure 3 when it detects that an occupant has exited the vehicle and the vehicle 2 has been locked. Whether or not an occupant has exited can be determined based on the position of the key on the vehicle 2. In addition, each of the initiation conditions for the first warning control program described above can also be used as the initiation conditions for the second warning control program, which will be described later. Furthermore, the program shown in the flowchart in Figure 3 below is stored in the RAM 32 and ROM 33 of the passenger exit assistance device 1 and executed by the CPU 31.

[0029] First, in step 1 of Figure 3 (hereinafter abbreviated as S), the CPU 31 detects all occupants in vehicle 2 using the in-vehicle camera 9 and assigns an ID to each detected occupant. Therefore, the CPU 31 assigns an ID to all occupants in vehicle 2 when vehicle 2 comes to a stop. For example, after detecting the vehicle has stopped, the CPU 31 performs image processing on the image data captured by the in-vehicle camera 9 to extract image features, detect the shape and movement of people inside the vehicle from the image, and determine how many occupants are present. The CPU 31 assigns a different ID to each detected occupant. Alternatively, the CPU 31 may use AI (artificial intelligence) technology to analyze the image and detect the occupants present inside the vehicle. The ID can be a number, a letter, or a string of characters combining them.

[0030] Furthermore, the method for detecting occupants inside the vehicle is not limited to using image data from the in-vehicle camera 9. For example, the CPU 31 may detect occupants inside the vehicle using point cloud data from a millimeter-wave radar mounted inside the vehicle. Therefore, the occupant detection unit in this specification is not limited to a configuration using a camera, but may also be configured to use millimeter-wave radar. The same applies to the pedestrian detection unit, which will be described later for detecting pedestrians located around the vehicle 2. The pedestrian detection unit in this specification is not limited to a configuration using an external camera (such as the front camera 5), ​​but may also be configured to use millimeter-wave radar to detect people outside the vehicle. In addition, the CPU 31 may use both a camera and millimeter-wave radar to detect occupants inside the vehicle and pedestrians around the vehicle.

[0031] Furthermore, the timing for detecting occupants inside the vehicle is not limited to when vehicle 2 is stopped. For example, CPU 31 may detect occupants in vehicle 2 when the shift lever of vehicle 2 is switched to park and assign an ID to each of the multiple occupants. Alternatively, CPU 31 may detect occupants in vehicle 2 at the moment occupants get into vehicle 2, not limited to the time after they have gotten in, and assign an ID to each occupant. For example, if the locks on each door are released while no one is inside vehicle 2, CPU 31 may detect people around the vehicle as occupants based on the image data from the external camera. Alternatively, CPU 31 may perform a comparison between the person detected from the image data from the external camera and the person detected from the image data from the internal camera 9 to detect occupants.

[0032] Furthermore, the CPU 31 may detect and assign an ID to a person who will become an occupant of the vehicle 2 before the occupant boards the vehicle 2. For example, the CPU 31 detects people around the vehicle in response to a person approaching the vehicle 2 or the key to the vehicle 2 approaching. The CPU 31 may detect people around the vehicle based on the image data from the external camera, and for example, detect a person who has approached to a predetermined distance from the vehicle 2 as an occupant. Alternatively, the CPU 31 may determine whether the person detected by the external camera intends to board the vehicle and detect the person who is determined to intend to board as an occupant. The CPU 31 may also perform a comparison between the person detected as an occupant by the external camera and the person detected by the internal camera 9 to detect occupants. Methods for determining the intention to board include the speed of movement of people around the vehicle, deceleration position, stopping position, gaze, shoulder movement, presence or absence of luggage, etc. Alternatively, a method of facial recognition may be used to determine the intention to board the vehicle. Furthermore, the CPU 31 may detect occupants at at least one of the above-mentioned timings: stopping, switching to parking, boarding, and before boarding. Therefore, the CPU 31 may detect occupants at multiple timings and perform matching of the detected occupants.

[0033] After executing S1, CPU 31 executes S2. In S2, CPU 31 begins monitoring the movement of the occupants to whom IDs were assigned in S1. After executing S2, CPU 31 determines whether all occupants to whom IDs were assigned have moved outside the monitoring range, which is a predetermined distance from vehicle 2 (S3). Thus, CPU 31 assigns IDs to occupants while they are in vehicle 2 and monitors the movement of occupants until they disembark from vehicle 2, i.e., until the disembarking occupants move outside the monitoring range.

[0034] Figure 4 shows the state in which a warning is issued in the first warning mode. As shown in Figure 4, the CPU 31 monitors passengers disembarking within a monitoring range 41, which is a predetermined distance L from the vehicle 2. The CPU 31 detects passengers disembarking within the monitoring range 41 based on the image data from the external camera. In the example shown in Figure 4, two passengers disembarking, 43A and 43B, and another vehicle 45 separate from vehicle 2 are shown. Passenger 43A is an occupant who disembarked from the driver's seat. Passenger 43B is an occupant who disembarked from the passenger seat. The following describes the example in which the two passengers 43A and 43B shown in Figure 4 exist. The case in which another vehicle 45 approaches as a moving object will also be described.

[0035] For example, when the CPU 31 detects two occupants using the in-vehicle camera 9, it assigns a different ID to each occupant (S1). The CPU 31 monitors the positions of the two occupants using the in-vehicle camera 9, and when at least one of the two occupants disembarks, it begins monitoring the positions of the disembarking occupants 43A and 43B around the vehicle based on the image data from the external camera (S2). The CPU 31 detects the positions of the disembarking occupants 43A and 43B based on the image data from the external camera, associates the position of each disembarking occupant 43A and 43B with an ID, and manages the position of each disembarking occupant 43A and 43B individually. The CPU 31 periodically acquires the position of each disembarking occupant 43A and 43B and monitors their positions while tracking them until all disembarking occupants 43A and 43B leave the monitoring range 41.

[0036] Furthermore, the method for monitoring the positions of disembarking passengers 43A and 43B is not limited to using image data from an external camera as described above; other methods such as millimeter-wave radar may also be used. Also, the shape of the monitoring range 41 may be an elliptical range matching the shape of the vehicle 2, or a circular range centered on the center of the vehicle 2. Therefore, the position of the center of the monitoring range 41 and the predetermined distance L are appropriately changed according to the shape of the vehicle 2, the imaging range of the external camera, etc. Also, the monitoring range 41 may be the maximum range in which people or other objects can be detected based on the image data from the external camera.

[0037] In S3, CPU 31 determines whether all occupants, i.e., both disembarking passengers 43A and 43B, have left the monitoring range 41. If at least one of the disembarking passengers 43A or 43B is still within the monitoring range 41, CPU 31 makes a negative determination in S3 (S3: NO) and executes S4. CPU 31 may also make a negative determination in S3 if an occupant with an assigned ID remains inside vehicle 2 without disembarking.

[0038] In S4, the CPU 31 determines whether at least one of the disembarking passengers 43A and 43B is approaching the other vehicle 45, which is a moving object. The CPU 31 monitors the position and speed of the other vehicle 45 (moving object) based on, for example, the image data from the external camera. If, for example, the distance between all the disembarking passengers 43A and 43B and the other vehicle 45 is longer than a predetermined reference distance, the CPU 31 makes a negative determination in S4, that is, determines that the disembarking passengers and the moving object are not approaching (S4: NO), and executes S3 again. This allows monitoring of the approach between disembarking passengers 43A and 43B and the other vehicle 45 as long as they are within the monitoring range 41.

[0039] On the other hand, if the distance between at least one of the disembarking passengers 43A and 43B and the other vehicle 45 falls below the reference distance, that is, if any disembarking passenger approaches the moving object, the CPU 31 makes an affirmative judgment in S4 (S4: YES) and executes S5. The CPU 31 issues a warning to the disembarking passengers 43A and 43B who are approaching the other vehicle 45. Note that the method for determining the proximity between disembarking passengers and the moving object is not limited to the method using the reference distance described above. For example, the CPU 31 may determine proximity by considering the speed and direction of movement of the disembarking passengers 43A and 43B and the other vehicle 45. Alternatively, the CPU 31 may change the reference distance according to the orientation of the disembarking passengers 43A and 43B. For example, the CPU 31 may make the reference distance when disembarking passengers 43A and 43B are facing towards the other vehicle 45 shorter than the reference distance when disembarking passengers 43A and 43B are facing away from the other vehicle 45.

[0040] In S5, the CPU 31 issues a warning using the speaker 17 and projection devices 18A to 18C. In the example shown in Figure 4, a person getting off the vehicle 43A is approaching another vehicle 45. For example, when the CPU 31 detects that the other vehicle 45 is approaching and the distance between the person getting off the vehicle 43A and the other vehicle 45 has fallen below a reference distance, it emits sound from the speaker 17 in the direction of the person getting off the vehicle 43A (S5). The CPU 31 controls the direction in which the sound is emitted from the speaker 17 by controlling the speaker control device 15, and emits sound from the speaker 17 towards the person getting off the vehicle 43A. In the example shown in Figure 4, the sound is emitted towards the person getting off the vehicle 43A, who is located diagonally to the left and in front of vehicle 2. For example, the CPU 31 emits a message such as, "A vehicle may be approaching, please be careful." As a result, the person getting off the vehicle 43A can recognize that the other vehicle 45 is approaching and avoid a collision with the other vehicle 45. Furthermore, the system can also warn the drivers of other vehicles 45 with voice, allowing them to slow down before colliding with the disembarking passenger 43A. Note that the above-described method of warning by sound is just one example. The CPU 31 may, for example, emit a warning sound, such as a buzzer, from the speaker 17 in all 360 degrees.

[0041] Furthermore, the CPU 31 controls the projection control device 16 (see Figure 2) to project an image toward the alighting passenger 43A. In the example shown in Figure 4, the alighting passenger 43A is located diagonally to the left and in front of vehicle 2. Therefore, the CPU 31 controls the projection control device 16, for example, to project a warning mark 47 (see Figure 4) from the projection device 18A (see Figure 1), which is located at the position of the headlight, toward the feet of the alighting passenger 43A. The warning mark 47 can be an exclamation mark or other marks that indicate danger. This allows warnings to be issued to the alighting passenger 43A and the drivers of other vehicles 45 through video as well. In addition, even if the noise around the vehicle is loud and the sound from the speaker 17 is difficult for the alighting passenger 43A to hear, projecting the warning mark 47 toward the feet can alert the other vehicles 45.

[0042] The above-described warning method using projection is merely an example. For example, CPU 31 may project text such as "Caution!!" using projection devices 18A-18C, or it may project a combination of images and text. Alternatively, CPU 31 may use all projection devices 18A-18C to project in all 360 degrees from vehicle 2. Or, vehicle 2 may be configured to display warning images or text on its roof or body. Vehicle 2 may also be configured to issue warnings using either sound or projection, or only one of these methods. Furthermore, the method of warning is not limited to the sound or video methods described above. For example, vehicle 2 may be configured to warn disembarking passengers 43A and 43B by illuminating and rotating warning lights. Vehicle 2 may also be configured to issue warnings using devices that are basically installed on vehicle 2, such as the horn, hazard lights, headlights, and taillights.

[0043] Furthermore, as described above, the CPU 31 assigns an ID to each occupant and monitors their location as occupants 43A and 43B after they disembark. Therefore, as shown in Figure 4, in the first warning mode, no warning is issued even if pedestrians 46 other than occupants 43A and 43B approach other vehicles 45. Pedestrians 46 are, for example, people walking around the parking position where vehicle 2 is stopped, and are unrelated to the occupants of vehicle 2. Even if the CPU 31 detects such pedestrians 46 within the monitoring range 41, it does not include them in the decisions of S3 and S4. This prevents the issuing of warnings to people other than occupants.

[0044] Furthermore, if the disembarking passengers 43A and 43B are not within the monitoring range 41, that is, if all occupants have left the monitoring range 41, the CPU 31 makes an affirmative judgment in S3 (S3: YES) and terminates the process shown in Figure 3. Figure 5 shows the state in which no warning is issued in the first warning mode. As shown in Figure 5, once the CPU 31 detects that the disembarking passengers 43A and 43B, whose positions were being monitored, have left the monitoring range 41, it does not issue any further warnings. Also, even after the disembarking passengers 43A and 43B have left the monitoring range 41, the CPU 31 does not issue a warning even if a pedestrian 46 and another vehicle 45 approach each other within the monitoring range 41. In this way, in the first warning mode, the CPU 31 issues warnings in response to the approach of disembarking passengers 43A and 43B and other vehicles 45 within the monitoring range 41. Furthermore, CPU 31 may resume the process shown in Figure 3 from S4 if passengers 43A and 43B leave the monitoring area 41 and then return to it. Alternatively, CPU 31 may terminate the process shown in Figure 3 when at least one of the passengers leaves the monitoring area 41.

[0045] (Regarding the second warning mode) Next, the second warning mode will be described. Figure 6 is a flowchart of the second warning control processing program according to this embodiment. When the ECU 10 detects, for example, that an occupant has exited the vehicle 2 and the door has been closed, it starts the second warning control processing program. As a method for detecting that an occupant has exited, for example, a method using image data from the in-vehicle camera 9 and the exterior camera, or a method based on the position of the key of the vehicle 2 can be employed. The second warning control processing program is a program that executes a warning based on the proximity of pedestrians located around the vehicle 2 and moving objects around the vehicle 2 until a predetermined specified time T has elapsed after the occupant has exited the vehicle 2. Therefore, in the second warning mode, a warning is executed when pedestrians and moving objects approach the vehicle, regardless of whether they are occupants or not. In the following description, the case in this specification where "pedestrians located around the vehicle" refers to the exiting passengers 43A, 43B and passersby 46 will be described.

[0046] Furthermore, the conditions for starting the second warning control program are not limited to those described above. For example, the ECU 10 may start the process shown in Figure 6 when it detects that all occupants have disembarked and all doors of vehicle 2 have been locked. Alternatively, the ECU 10 may start the process shown in Figure 6 when it detects that occupants have disembarked based on image data from the in-vehicle camera 9 or the exterior camera, regardless of door operation. Also, the ECU 10 may start the process shown in Figure 6 when it detects that any door has been opened and closed after vehicle 2 has come to a stop, regardless of whether occupants have disembarked or not. The program shown in the flowchart in Figure 6 is stored in the RAM 32 and ROM 33 of the passenger disembarkation support device 1 and executed by the CPU 31.

[0047] First, in S11 of Figure 6, the CPU 31 starts measuring the elapsed time using a timer. Next, the CPU 31 determines whether a pedestrian around the vehicle is approaching a moving object (another vehicle 45) (S12). Similar to the first warning mode, as shown in Figure 4, when the CPU 31 detects that the person getting out of the vehicle 43A and the other vehicle 45 have come within a predetermined reference distance (S12: YES), it issues a warning (S13). In S13, similar to the first warning mode, the CPU 31 issues a warning by, for example, emitting sound from the speaker 17 towards the person getting out of the vehicle 43A as the other vehicle 45 approaches, and by projecting a warning mark 47 at the feet of the person getting out of the vehicle 43A. After executing S13, the CPU 31 executes S12 again. Note that, similar to the first warning mode, the warning method in the second warning mode is not limited to the method described above.

[0048] Furthermore, in the second warning mode, as shown in Figure 7, the CPU 31 will issue a warning if there is a possibility that another vehicle 45 is approaching, even if the disembarking passengers 43A and 43B have left the monitoring range 41, as long as the specified time T has elapsed. For example, even if disembarking passenger 43A has left the monitoring range 41, if the CPU 31 detects that another vehicle 45 is approaching in the direction of disembarking passenger 43A (S12: YES), it will issue a warning to disembarking passenger 43A (S13).

[0049] Furthermore, in the second warning mode, the CPU 31 also issues a warning to pedestrian 46 if a pedestrian 46 within the monitoring range 41 approaches another vehicle 45 (S12: YES) as long as the specified time T has elapsed (S13). Therefore, in the second warning mode, a warning is issued regardless of whether the pedestrians around the vehicle are occupants of vehicle 2 or not. For example, the CPU 31 issues warnings to people 43A and 43B who get out of vehicle 2, pedestrians 46 detected within the monitoring range 41 within the specified time T, and other vehicles 45 detected within the monitoring range 41 within the specified time T.

[0050] Furthermore, if the CPU 31 does not detect proximity between pedestrians and moving objects in S12 (S12: NO), it executes S15. The CPU 31 executes S15 if all of the disembarking passengers 43A, 43B and the pedestrians 46 within the monitoring range 41 are not approaching other vehicles 45 (S12: NO). In S15, the CPU 31 determines whether the measurement time of the timer started in S11 is equal to or greater than the specified time T. If the measurement time is equal to or greater than the specified time T, that is, if the specified time T has elapsed since the timer started measuring time in S11 (S15: YES), the CPU 31 terminates the process shown in Figure 6. Therefore, in the second warning mode, if proximity between pedestrians and moving objects around the vehicle is detected as long as the specified time T has elapsed, a warning is issued.

[0051] Furthermore, as shown in Figure 8, in the second warning mode, the CPU 31 terminates the process shown in Figure 6 after the specified time T has elapsed, and does not issue a warning even if the disembarking passengers 43A and 43B approach other vehicles 45. Also, the CPU 31 does not issue a warning after the specified time T has elapsed even if a pedestrian 46 within the monitoring range 41 approaches other vehicles 45.

[0052] In S15, if the timer measurement time is less than the specified time T (S15: NO), the CPU 31 determines whether another passenger has disembarked (S16). Here, "another passenger" refers to a passenger other than the passenger who triggered the process in Figure 6. If the CPU 31 does not detect, for example, that another passenger has disembarked and closed the door (S16: NO), it repeats the process from S12. In this case, the CPU 31 continues measuring the measurement time.

[0053] On the other hand, if the CPU 31 detects that another occupant has exited the vehicle and closed the door (S16: YES), it resets the timer's measurement time to zero (S17). After executing S17, the CPU 31 repeats the process from S12. In this way, for example, if the CPU 31 detects that the passenger in the front passenger seat (exiter 43B) has exited the vehicle first and closed the front door 12L, and then detects that the driver's seat occupant (exiter 43A) has exited the vehicle and closed the front door 12R before the specified time has elapsed (S15: NO), it resets the measurement time (S17) and continues monitoring the pedestrian and the moving object. In this manner, when the second warning mode is set, the CPU 31 monitors the proximity of the pedestrian and the moving object until the specified time T has elapsed.

[0054] Furthermore, after the CPU 31 has finished the process in Figure 6, if any occupants other than the occupant who triggered the start of the process in Figure 6 have disembarked, the CPU 31 may execute the process in Figure 6 again. For example, if the CPU 31 detects that the passenger in the front passenger seat (disembarking occupant 43B) has disembarked first and closed the front door 12L, and starts the process from S11, and after the specified time T has elapsed (S15: YES) and the process in Figure 6 has finished, the CPU 31 may start the process from S11 again if it detects that the driver's seat occupant (disembarking occupant 43A) has disembarked and closed the front door 12R.

[0055] Incidentally, the relationship between the content of this specification and the terminology of the above embodiments is as follows: In the above embodiments, the disembarking passengers 43A, 43B and the passerby 46 are examples of pedestrians as defined herein. The other vehicle 45 is an example of a moving object.

[0056] (Effects of this embodiment) As described in detail above, this embodiment provides the following effects. (1) According to the passenger disembarking support device 1 and the computer program executed by the passenger disembarking support device 1 according to this embodiment, the CPU 31 of the ECU 10 detects pedestrians (passengers 43A, 43B, and other passersby 46) located around the vehicle 2 (S3, S4, S12). The CPU 31 also detects other moving objects (other vehicles 45) moving around the vehicle 2 (S4, S12). The CPU 31 issues a warning based on the fact that pedestrians and other vehicles 45 have approached each other after the occupants have disembarked from the vehicle 2 (S5, S13).

[0057] According to this system, if a warning is issued when a pedestrian and a moving object approach each other after the occupants have disembarked, it becomes possible to avoid a collision if at least one of the pedestrians (disembarking occupants 43A, 43B, passersby 46) or the driver of the other vehicle 45 is unaware of the approach of the other. Therefore, occupants who have disembarked can be monitored as pedestrians, and the occurrence of collisions between occupants and moving objects can be suppressed.

[0058] (2) In the first warning mode, the CPU 31 also issues a warning to the passengers 43A and 43B who have disembarked from the vehicle 2. The CPU 31 issues a warning based on the detection that the disembarking passengers 43A and 43B are approaching the other vehicle 45 from the time the occupants disembark from the vehicle 2 until the disembarking passengers 43A and 43B are a predetermined distance L away from the vehicle 2 (S4:YES, S5). The CPU 31 does not issue a warning after the disembarking passengers 43A and 43B are a predetermined distance L away from the vehicle 2 (S3:YES).

[0059] According to this, a warning can be issued if a collision with another vehicle 45 is likely to occur for occupants who are within a monitoring range 41 at a predetermined distance L from vehicle 2. It is possible to avoid collisions between occupants and moving objects until they leave the monitoring range 41. For example, if disembarking occupants 43A and 43B remain within the monitoring range 41 for a long time, such as working around the vehicle or talking to passersby, it is possible to monitor moving objects approaching disembarking occupants 43A and 43B.

[0060] (3) In addition, the CPU 31 detects occupants of the vehicle 2 in the first warning mode (S1). The CPU 31 detects each of the detected occupants as a pedestrian after they have disembarked from the vehicle 2. The CPU 31 issues a warning based on the proximity of pedestrians (occupants) and a moving object until all detected occupants (disembarking occupants 43A, 43B) have moved a predetermined distance L away from the vehicle 2.

[0061] According to this, if multiple occupants are in vehicle 2, all occupants can be monitored as pedestrians. All occupants can be warned of approaching a moving object until they become disembarking passengers 43A, 43B and leave the monitoring range 41 of a predetermined distance L. The safety of all occupants can be ensured until they leave the monitoring range 41.

[0062] (4) In addition, in the second warning mode, the CPU 31 issues a warning regardless of whether the pedestrians around the vehicle are occupants who have disembarked from the vehicle 2. The CPU 31 issues a warning based on the proximity of pedestrians and the moving object from the time occupants disembark from the vehicle 2 until a predetermined specified time T has elapsed (S15: NO) (S12: YES, S13). After the specified time T has elapsed (S15: YES), the CPU 31 does not issue a warning.

[0063] According to this system, a warning can be issued to pedestrians around the vehicle, regardless of whether they are occupants or not, for a specified time T after the occupants have disembarked. This prevents contact between occupants and pedestrians around the vehicle and the moving vehicle. Furthermore, monitoring for approaching objects is terminated after the specified time T has elapsed. This allows monitoring to be terminated if disembarked occupants 43A and 43B remain around the vehicle for a long time, such as working around the vehicle or talking to passersby. This reduces the battery consumption of vehicle 2, suppressing battery degradation and malfunctions such as battery failure. In addition, there is no need to determine whether pedestrians around the vehicle are occupants or not, and a warning can be issued without judgment processing. Even if vehicle 2 does not have an in-vehicle camera 9, a warning can be issued using only the exterior camera.

[0064] It should be noted that the present invention is not limited to the embodiments described above, and various improvements and modifications are possible without departing from the spirit of the invention. For example, the processing content and order of processing in the flowcharts in Figures 3 and 6 of the above embodiment are just examples. For example, in the above embodiment, a warning was issued both when a pedestrian approached the moving object and when the moving object approached the pedestrian, but this is not limited to this. The CPU 31 may issue a warning only when the moving object is approaching the pedestrian. For example, the CPU 31 may issue a warning when the moving object is moving and the distance between the pedestrian and the moving object is less than or equal to a reference distance. Also, the CPU 31 does not have to issue a warning when a pedestrian is approaching the moving object. Furthermore, the CPU 31 may measure the elapsed time in the first warning mode. In the first warning mode, the CPU 31 does not need to issue a warning if the disembarking persons 43A and 43B and other vehicles 45 approach within a specified distance or less within the monitoring range 41, but only a specified time T has elapsed. Also, in the first and second warning modes, the CPU 31 may restart monitoring if the pedestrians being monitored start moving again after the elapsed time since the start of monitoring has reached a specified time T and monitoring has ended. For example, in the first warning mode, if the disembarking persons 43A and 43B, who have been assigned IDs, stop within the monitoring range 41 for a specified time T and the process in Figure 3 has ended, and then the disembarking persons 43A and 43B start moving again from the position they were at when the process in Figure 3 ended, the CPU 31 may monitor the approach of the disembarking persons 43A and 43B to other vehicles 45 and issue a warning. Furthermore, in the second warning mode, after the processing shown in Figure 6 has finished and a specified time T has elapsed, the CPU 31 may monitor the proximity of the pedestrian to another vehicle 45 and issue a warning if a pedestrian within the monitoring range 41 starts moving again. Furthermore, in the second warning mode, the CPU 31 does not need to issue a warning if pedestrians (those getting out of the vehicle 43A, 43B, and passersby 46) leave the monitoring range 41. Furthermore, the CPU 31 may be configured to execute only one of either the first warning mode or the second warning mode.

[0065] Furthermore, in the above embodiment, the ECU 10 of the passenger exit assistance device 1 is configured to execute the processing of the first warning control processing program (Figure 3) and the second warning control processing program (Figure 6), but the executing entity can be changed as appropriate. For example, the processing of Figures 3 and 6 may be configured to be executed by the control unit of the navigation device or other in-vehicle devices. Furthermore, the configuration of the passenger exit assistance device 1 is not limited to the configuration of the embodiment described above. For example, the passenger exit assistance device 1 may be configured to include only the ECU 10, or to include only the ECU 10 and an external camera.

[0066] Next, we will describe the technical ideas derived from the above embodiment. (i) The occupant detection unit is, If the vehicle has multiple passengers, each of the passengers shall be assigned an ID. The pedestrian detection unit is, The passenger disembarking assistance device according to claim 3, which manages the position of all passengers who have disembarked from the vehicle, based on the ID assigned by the passenger detection unit, until each of them has moved a predetermined distance away from the vehicle.

[0067] According to this system, when a vehicle has multiple occupants, each occupant can be assigned an ID, and the position and movement of each occupant can be monitored. By managing occupants within a predetermined distance from the vehicle using their IDs and managing the system to track the position of the occupants assigned IDs, collisions between the managed occupants and moving objects can be avoided.

[0068] (b) The occupant detection unit is, The passenger disembarking assistance device described in Appendix A, which detects the occupants of the vehicle at at least one of the following timings: when the vehicle is stopped, when the vehicle's shift lever is switched to parking, when an occupant gets into the vehicle, or before an occupant gets into the vehicle, and assigns the ID to each of the multiple occupants.

[0069] According to this system, when the vehicle stops or the gear lever is changed, people inside the vehicle are detected as occupants, and their positions after disembarking can be managed using IDs. Similarly, when occupants are detected at the time of boarding, their positions can be managed using IDs after the vehicle stops. Furthermore, when detecting occupants before they board the vehicle, for example, external cameras can be used to detect people before they enter the vehicle. In this case, unlike internal cameras which generally detect the upper body, cameras that image the outside of the vehicle or millimeter-wave radar can be used to detect the entire body of a person. Therefore, by capturing the entire body of a person before they board the vehicle around the vehicle, occupants can be detected and assigned IDs with greater accuracy. This reduces the occurrence of occupant detection errors and ensures that all occupants are reliably assigned IDs.

[0070] (h) The warning unit is, If the vehicle has multiple passengers, After any of the multiple occupants disembark, and until the specified time has elapsed, a warning will be issued based on the proximity of the pedestrian detected by the pedestrian detection unit and the moving object detected by the moving object detection unit. If another passenger disembarks before the specified time has elapsed, the disembarking assistance device according to claim 4 resets the measurement time and starts measuring the specified time from the beginning.

[0071] According to this system, if another crew member disembarks while the elapsed time is being measured, the measurement time can be reset and monitoring can continue. This prevents the time until the specified time elapses from being shortened for crew members who disembark later, allowing monitoring to continue for the specified time after disembarking, thus helping to avoid collisions with moving objects.

[0072] (ii) The warning unit is, The disembarking passenger assistance device according to claim 1 or 2, wherein a warning is issued using at least one of sound and light, and when the pedestrian detected by the pedestrian detection unit and the moving object detected by the moving object detection unit approach each other, a warning is issued by emitting at least one of sound and light towards the pedestrian.

[0073] According to this system, the moving vehicle emits sound and light towards approaching pedestrians. By increasing the directionality of the sound and light, the warning can be more reliably perceived by pedestrians. If the area around the vehicle is bright or if the light is blocked by an object, increasing the directionality of the sound can help pedestrians recognize the warning. Also, if there is a lot of noise around the vehicle, increasing the directionality of the light can help pedestrians recognize the warning. This makes it possible to more reliably avoid collisions between pedestrians and moving vehicles. [Explanation of symbols]

[0074] 1 Alighting assistance device, 2 Vehicle, 10 Vehicle control ECU (pedestrian detection unit, moving object detection unit, warning unit, occupant detection unit), 46 Passerby (pedestrian), 43A, 43B Alighting person (passerby), 31A Pedestrian detection unit, 31B Moving object detection unit, 31C Warning unit, 31D Occupant detection unit, 45 Other vehicle (moving object), L Determined distance, T Designated time.

Claims

1. A pedestrian detection unit that detects pedestrians located around the vehicle, A moving object detection unit that detects moving objects other than pedestrians moving around the vehicle, A warning unit that issues a warning based on the fact that a pedestrian detected by the pedestrian detection unit and a moving object detected by the moving object detection unit have come into close proximity after the occupants have disembarked from the vehicle, A passenger disembarking assistance device equipped with the following features.

2. The aforementioned warning unit is If the pedestrian detected by the pedestrian detection unit is a passenger who has disembarked from the vehicle, a warning will be issued. After the occupants disembark from the vehicle, and until the disembarked occupants have moved a predetermined distance away from the vehicle, a warning is issued based on the proximity of the pedestrian detected by the pedestrian detection unit and the moving object detected by the moving object detection unit. The passenger disembarking assistance device according to claim 1, wherein no warning is issued after the passenger who has disembarked has moved a predetermined distance away from the vehicle.

3. The vehicle further includes an occupant detection unit for detecting the occupants of the vehicle, The pedestrian detection unit is, For each of the occupants detected by the occupant detection unit, after they disembark from the vehicle, they are detected as pedestrians. The aforementioned warning unit is The passenger disembarking assistance device according to claim 2, wherein, until all passengers detected by the passenger detection unit have moved away from the vehicle by the predetermined distance, a warning is issued based on the proximity of a pedestrian detected by the pedestrian detection unit and a moving object detected by the moving object detection unit.

4. The aforementioned warning unit is Regardless of whether the pedestrian detected by the pedestrian detection unit is an occupant who has disembarked from the vehicle, the vehicle will issue a warning. After the occupants disembark from the vehicle, a warning is issued based on the proximity of the pedestrian detected by the pedestrian detection unit and the moving object detected by the moving object detection unit, for a predetermined period of time. The passenger disembarking assistance device according to claim 1 or claim 2, wherein no warning is issued after the specified time has elapsed.

Images