Vehicle monitoring device and vehicle monitoring program
The vehicle monitoring device addresses the risk of unauthorized boarding in autonomous vehicles by identifying cautionary persons and using external alarms and evacuation controls to prevent entry and ensure passenger safety.
Patent Information
- Application Number
- JP2023166447
- Authority / Receiving Office
- JP · JP
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2023-04-26
- Filing Date
- 2023-09-27
- Publication Date
- 2024-11-08
AI Technical Summary
In autonomous vehicles, there is a risk of unauthorized individuals attempting to board the vehicle, posing a threat to passengers and vehicle security when no driver is present.
A vehicle monitoring device equipped with an approach determination unit to identify cautionary persons and an external alarm to notify them that boarding is prohibited, along with evacuation controls to move the vehicle away from potential threats.
Prevents unauthorized boarding and reduces the risk of theft or harm to passengers by alerting cautionary individuals and safely moving the vehicle away from potential threats.
Smart Images

Figure 2024159414000001_ABST
Abstract
Description
[Technical field]
[0001] The disclosure of this specification relates to a monitoring technique for a vehicle that monitors the status of the vehicle itself. [Background technology]
[0002] Patent Document 1 discloses an in-vehicle monitoring system used in a public vehicle that is an autonomous vehicle equipped with an autonomous driving function. The in-vehicle monitoring system recognizes people and luggage in the vehicle, and issues a warning to the passengers in the vehicle when the vehicle is crowded and a seat is occupied by luggage or multiple seats are occupied by one passenger. [Prior art documents] [Patent documents]
[0003] [Patent Document 1] JP 2022-154979 A Summary of the Invention [Problem to be solved by the invention]
[0004] In a vehicle that runs autonomously without a specific driver on board, such as the shared vehicle of Patent Document 1, a person who is not scheduled to board (hereinafter, a cautious person) may try to forcibly board the vehicle. In order to protect passengers and the vehicle from such a cautious person, a technology that can deal with the cautious person may be required.
[0005] The present disclosure has an object to provide a vehicle monitoring device and a vehicle monitoring program that are capable of responding to a warning person approaching the vehicle even when the driver is not on board. [Means for solving the problem]
[0006] In order to achieve the above object, one disclosed embodiment is a vehicle monitoring device that monitors the status of a host vehicle (Am) that can be driven by an automatic driving function, and includes an approach determination unit (73) that determines the approach, to the host vehicle, of a person to beware of (UM) who is not scheduled to board the host vehicle, and an alarm execution unit (72) that, when it is determined that a person to beware of is approaching the host vehicle, issues an external alarm indicating that boarding is not possible, using an external alarm device (24, 25) provided in the host vehicle.
[0007] Another disclosed aspect is a vehicle monitoring program that monitors the status of a host vehicle (Am) that can be driven by an automatic driving function, and causes at least one processing unit (51) to execute processing including determining the approach of a person to beware of (UM) who is not scheduled to board the host vehicle to the host vehicle (S13, S213, S312), and, if it is determined that a person to beware of is approaching the host vehicle, issuing an external alarm indicating that boarding is not permitted using an external alarm device (24, 25) provided in the host vehicle (S14, S219, S317).
[0008] In these aspects, an external alarm is provided to a person approaching the vehicle, indicating that the person cannot board the vehicle. This external alarm can prevent a person who is not scheduled to board the vehicle from getting on the vehicle. Therefore, even if the driver is not on the vehicle, it is possible to respond to a person approaching the vehicle.
[0009] In addition, the reference numbers in parentheses in the above and claims are merely examples of the correspondence with specific configurations in the embodiments described below, and do not limit the technical scope in any way. In addition, claims that are not explicitly stated in the claims can be combined together if there is no particular problem with the combination. [Brief description of the drawings]
[0010] [Figure 1] 1 is a diagram showing an overall view of an autonomous driving system including an autonomous driving ECU according to a first embodiment of the present disclosure. FIG. [Diagram 2]FIG. 2 is a block diagram showing details of a remote system. [Diagram 3] FIG. 2 is a block diagram showing details of a vehicle-side system. [Figure 4] 13 is a diagram showing a time series of responses in response to a determination of an approach of a cautionary person. FIG. [Diagram 5] 10 is a flowchart showing details of a cautionary person response process executed by the autonomous driving ECU. [Figure 6] 10 is a flowchart showing details of an emergency evacuation process performed by an autonomous driving ECU. [Figure 7] 11 is a flowchart showing details of a behavior warning process for dealing with problematic behavior by a passenger. [Figure 8] 10 is a flowchart showing details of a theft notification process for responding to theft of a driverless vehicle. [Figure 9] FIG. 11 is a block diagram showing details of a vehicle-side system according to a second embodiment. [Figure 10] 10 is a flowchart showing details of a video distribution process performed by the autonomous driving ECU. [Figure 11] 10 is a flowchart showing details of a cautionary person response process executed by the autonomous driving ECU. [Figure 12] 6 is a flowchart showing details of a theft notification process performed by the autonomous driving ECU. [Figure 13] 10 is a flowchart showing details of a takeover response process performed by an autonomous driving ECU. [Figure 14] 13A to 13C are diagrams showing, in chronological order, responses in response to the determination of the approach of a person to be warned in the third embodiment. [Figure 15] 10 is a flowchart showing details of a cautionary person response process executed by the autonomous driving ECU. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0011] Hereinafter, a number of embodiments will be described with reference to the drawings. In addition, by assigning the same reference numerals to corresponding components in each embodiment, duplicated descriptions may be omitted. When only a part of the configuration is described in each embodiment, the configuration of the other embodiment described above can be applied to the other parts of the configuration. In addition to the combination of configurations explicitly stated in the description of each embodiment, configurations of multiple embodiments can be partially combined together even if not explicitly stated, as long as there is no particular problem with the combination.
[0012] First embodiment An autonomous driving ECU (Electronic Control Unit) 50 shown in FIG. 1 is mounted on a vehicle (hereinafter, the host vehicle Am). By mounting the autonomous driving ECU 50, the host vehicle Am becomes an autonomous driving vehicle or an autonomous vehicle equipped with an autonomous driving function. With a level 4 autonomous driving function, the host vehicle Am can run as a driverless vehicle ADV without a driver in the driver's seat DS. The autonomous driving level in this disclosure is based on the standards defined by the Society of Automotive Engineers.
[0013] The autonomous driving ECU 50 functions as a vehicle monitoring device according to the first embodiment of the present disclosure. The autonomous driving ECU 50 monitors the conditions inside and outside the vehicle Am in place of the driver in a driverless vehicle ADV where no driver is on board. Specifically, the autonomous driving ECU 50 performs response processing in place of the driver in a scene where a person other than a person (user) who is scheduled to board the vehicle Am tries to board the vehicle Am.
[0014] [Specific autonomous operation of driverless vehicles] The driverless vehicle ADV is used as a vehicle that provides mobility services. The operation of multiple driverless vehicles ADV, including the vehicle Am, is remotely monitored by a remote monitoring facility CNT. The remote monitoring facility CNT is operated by businesses (hereinafter, specified automated driving implementers) that provide level 4 automated driving mobility services (hereinafter, specified automated driving). The remote monitor RO and operation manager OM are permanently stationed at the remote monitoring facility CNT. The remote monitor RO and operation manager OM are employees (specified automated driving business workers) belonging to the specified automated driving implementers.
[0015] The remote monitor RO is a manager (specific automatic operation manager) who manages the specific automatic operation. A plurality of remote monitors RO, such as a main monitor ROm and a sub-monitor ROs, are placed in the remote monitoring facility CNT. The remote monitor RO monitors the operation status of the automatic operation system 100 described later, and when it is determined that the automatic operation system 100 is not operating normally, it terminates the operation of the driverless vehicle ADV.
[0016] The operation manager OM manages the remote monitor RO. The operation manager OM can take over at least part of the duties of the remote monitor RO described above when an abnormality occurs in the remote monitor RO, or when an abnormality occurs in multiple driverless vehicles ADV, etc. Note that if the remote monitoring facility CNT is small or if multiple remote monitors RO are stationed there, the deployment of the operation manager OM may be omitted. In other words, the remote monitor RO may also function as the operation manager OM. Furthermore, there may be only one remote monitor RO deployed at the remote monitoring facility CNT.
[0017] The automated driving system 100 that enables the specific automated driving described above is composed of a remote side system 130, an infrastructure side system 160, and a vehicle side system IVS, etc. The details of these components will be described below in order.
[0018] [Remote system] The remote side system 130 shown in FIG. 1 and FIG. 2 exists remotely from the multiple driverless vehicles ADV. The remote side system 130 realizes operation management by a remote monitor RO of multiple driverless vehicles ADV including the host vehicle Am. The remote side system 130 enables the remote monitor RO to remotely check the situation when an abnormality occurs in the driverless vehicle ADV. When it is difficult to continue the automatic driving of the driverless vehicle ADV, the remote side system 130 transfers the dynamic driving task from the automatic driving ECU 50 to the remote monitor RO. This enables the remote monitor RO to remotely operate the driverless vehicle ADV. The remote side system 130 is composed of a center communication device 131, multiple remote user interfaces 132, a remote control device 140, and the like.
[0019] The center communication device 131 can communicate with the infrastructure side system 160, the vehicle side system IVS, and a user terminal 170 (e.g., a smartphone, etc.) of a user who uses the mobility service via a public communication line network. The center communication device 131 receives infrastructure information collected by the infrastructure side system 160. The center communication device 131 receives vehicle information collected by the vehicle side system IVS, and transmits input information, remote driving information, and monitor camera images acquired by the remote side system 130 to the vehicle side system IVS. The center communication device 131 receives usage application information based on input by a user who plans to use the mobility service from the user terminal 170.
[0020] The remote user interface 132 is provided for each remote user, such as a remote monitor RO and an operation manager OM, who are located in the remote monitoring facility CNT. The remote user interface 132 includes an input device 133, a remote user camera 134, a remote operation unit 135, a remote side display 136, a remote side speaker 137, and the like.
[0021] The input device 133 is a keyboard, a mouse, etc. Terminal operations by the remote person are input to the input device 133. The remote person camera 134 is installed so as to capture the face of the remote person and its surroundings. The remote person camera 134 generates a monitor camera image including the face of the remote person. The remote person camera 134 is provided with a microphone that collects the voice of the remote person. The monitor camera image is accompanied by the sound collected by the microphone.
[0022] The remote control unit 135 includes an accelerator pedal, a brake pedal, a steering wheel, etc. A driving operation for remotely controlling a specific driverless vehicle ADV is input to the remote control unit 135 by a remote person. The remote side display 136 and the remote side speaker 137 present to the remote person information related to remote monitoring of the driverless vehicle ADV, such as infrastructure information and vehicle information received by the center communication device 131.
[0023] The remote control device 140 is a computer mainly including a processing unit 151, a RAM 152, a storage unit 153, an input / output interface 154, and a control circuit including a bus connecting these units. The processing unit 151 executes various processes (instructions) for implementing the remote management method of the present disclosure by accessing the RAM 152. The storage unit 153 stores various programs (remote management program, etc.) executed by the processing unit 151. By executing the programs by the processing unit 151, an information acquisition unit 141, an operation acquisition unit 142, a presentation control unit 143, an operation management unit 144, etc. are constructed in the remote control device 140 (see FIG. 2).
[0024] The information acquisition unit 141 acquires input information of terminal operations input to the input device 133, and monitor camera images generated by the remote person camera 134, etc. The input information and monitor camera images acquired by the information acquisition unit 141 are transmitted to the vehicle side system IVS of the driverless vehicle ADV via the operation management unit 144 and the center communication device 131 when an abnormality occurs in the driverless vehicle ADV, etc.
[0025] The operation acquisition unit 142 acquires remote driving information indicating driving operations input to the remote operation unit 135 when the driverless vehicle ADV is unable to continue autonomous driving. The remote operation information includes, as an example, a plurality of driving instruction values indicating the accelerator operation amount, the brake operation amount, and the steering operation amount. The remote driving information acquired by the operation acquisition unit 142 is transmitted to the vehicle-side system IVS of the driverless vehicle ADV that is unable to continue autonomous driving via the operation management unit 144 and the center communication device 131.
[0026] The presentation control unit 143 presents the infrastructure information and vehicle information acquired by the center communication device 131 and the operation management unit 144 to the remote person using the remote side display 136 and the remote side speaker 137. As an example, video and audio from within the cabin of the driverless vehicle ADV, and video and audio of the driving route of the driverless vehicle ADV, etc. are presented to the remote person by the remote side display 136 and the remote side speaker 137. The remote person understands the situation of the driverless vehicle ADV based on the information provided from the remote side display 136 and the remote side speaker 137.
[0027] The operation management unit 144 cooperates with the autonomous driving ECU 50 via the center communication device 131 to manage the operation of the driverless vehicle ADV. The operation management unit 144 acquires usage application information from the user terminal 170 via the center communication device 131. The operation management unit 144 sets a destination and waypoints for the driverless vehicle ADV under its management based on the usage application information, and transmits the set location information to the autonomous driving ECU 50. The operation management unit 144 may also set a driving route for the driverless vehicle ADV, and transmit the set driving route to the autonomous driving ECU 50.
[0028] The operation management unit 144 shares the usage application information acquired from the user terminal 170 with the autonomous driving ECU 50 of the driverless vehicle ADV that each user plans to use. Specifically, based on the usage application information, the operation management unit 144 transmits identification information of the user (passenger PS) who plans to board the driverless vehicle ADV at each waypoint to the autonomous driving ECU 50 together with location information, etc. In addition, when new usage application information is acquired, the operation management unit 144 sequentially transmits the identification information of the user to the autonomous driving ECU 50 of the driverless vehicle ADV that is suitable for use by this user (for example, the closest).
[0029] When the autonomous driving of the driverless vehicle ADV cannot be continued, the operation management unit 144 cooperates with the autonomous driving ECU 50 to switch from autonomous driving by the autonomous driving ECU 50 to remote driving by the remote monitor RO. After switching to remote driving, the operation management unit 144 cooperates with the presentation control unit 143 to continue presenting the latest status inside and outside the driverless vehicle ADV to the remote monitor, and sequentially transmits remote driving information acquired by the operation acquisition unit 142 to the vehicle-side system IVS.
[0030] [Infrastructure system] 1 includes a surveillance camera 161. The infrastructure-side system 160 is installed on the driving route of the driverless vehicle ADV. A large number of infrastructure-side systems 160 may be installed so as to capture the entire driving route of the driverless vehicle ADV without omission, or may be installed only at specific points such as bus stops and intersections on the driving route.
[0031] The infrastructure-side system 160 can communicate with the remote-side system 130 via a public communication line network. The infrastructure-side system 160 transmits surveillance video of the driving route captured by the surveillance camera 161 to the remote-side system 130 as infrastructure information. The infrastructure-side system 160 may include an infrastructure sensor different from the surveillance camera 161. In this embodiment, detection information of the infrastructure sensor is transmitted from the infrastructure-side system 160 to the remote-side system 130 as infrastructure information.
[0032] [Vehicle system] The host vehicle Am equipped with the vehicle-side system IVS shown in Fig. 1 and Fig. 3 is operated under the management of the remote-side system 130 with a passenger PS inside the vehicle cabin. The host vehicle Am can run autonomously under the control of the autonomous driving ECU 50 even when the driver is not seated in the driver's seat DS. The autonomous driving performed by the autonomous driving ECU 50 may be the above-mentioned level 4 autonomous driving, or may be level 3 autonomous driving that is premised on constant remote monitoring of the surroundings by a remote monitor RO.
[0033] The passenger PS is a user (passenger) of a mobility service using the driverless vehicle ADV. The passenger PS uses the user terminal 170 and an application installed in the user terminal 170 to use the mobility service. The user terminal 170 transmits to the remote system 130 application information including a planned boarding location (or current location), a planned boarding time, a planned disembarking location, and identification information for identifying the passenger PS, as information necessary for using the mobility service. The identification information includes information such as the name, age, sex, and whether or not the passenger PS has a disability. The user terminal 170 is also used for personal authentication of the passenger PS when boarding (checking in) and disembarking (checking out) the driverless vehicle ADV. In addition, the usage fee for the mobility service may be paid through the application of the user terminal 170.
[0034] The vehicle Am is provided with an in-vehicle operation unit 110 for manual driving by the passenger PS (hereinafter, in-vehicle driving). For example, the in-vehicle operation units 110, such as an accelerator pedal, a brake pedal, and a steering wheel, are provided in front of the driver's seat DS as the in-vehicle operation unit 110. In other words, in a driverless vehicle ADV, the seat closest to the in-vehicle operation unit 110 becomes the driver's seat DS. For example, when it becomes difficult to continue the automatic driving and the automatic driving ECU 50 or the remote control device 140 permits the in-vehicle driving, the passenger PS seated in the driver's seat DS becomes the driver, and the driving operations input to the in-vehicle operation unit 110 are accepted.
[0035] The vehicle-side system IVS is mainly composed of an automatic driving ECU 50. The automatic driving ECU 50 is communicatively connected to a communication bus of an in-vehicle network mounted on the vehicle Am. The communication bus is connected to an in-vehicle monitor device 29, a user reception terminal 120, a surroundings monitoring sensor 30, a locator 35, a navigation ECU 38, an in-vehicle communication device 39, a driving control ECU 40, a body ECU 43, an HMI control device 10, and the like. Among these devices and ECUs, specific nodes may be directly electrically connected to each other and be able to communicate without going through a communication bus.
[0036] The in-vehicle monitor device 29 includes a plurality of in-vehicle cameras that capture the interior of the vehicle Am, and a control unit that controls the plurality of in-vehicle cameras. The in-vehicle cameras may be visible light cameras, or may be near-infrared cameras combined with near-infrared light sources. At least one of the plurality of in-vehicle cameras is installed so as to be able to capture an image of a passenger PS (driver) seated in the driver's seat DS, and functions as a driver monitor. Another of the plurality of in-vehicle cameras is installed near the door and is set so as to be able to capture an image of a user about to get into the vehicle Am. The in-vehicle monitor device 29 provides the automatic driving ECU 50, etc. with images captured by the in-vehicle cameras, or the results of image analysis of the captured images, as in-vehicle monitor information. The in-vehicle monitor information includes information indicating the number of passengers PS, the position of each passenger PS in the vehicle, and the state of each passenger PS, such as whether they are standing or sitting.
[0037] The user reception terminal 120 is provided near the door of the vehicle Am so that the passenger PS can hold the user terminal 170 over it when getting on and off. The user reception terminal 120 may be installed inside the vehicle cabin, on the outer side of the vehicle Am (for example, near the button for opening the door), or both inside and outside the vehicle. The user reception terminal 120 acquires identification information (hereinafter, boarding application) of a user who plans to board the vehicle Am, in other words, a reserved user whose usage application information has been acquired by the remote side system 130, from the operation management unit 144 via the in-vehicle communication device 39.
[0038] The user reception terminal 120 receives identification information of a user who is about to board the vehicle Am from the user terminal 170 using near field communication (NFC), a one-dimensional code, a two-dimensional code, or the like. The user reception terminal 120 compares the identification information of the user who is about to board with a boarding application acquired in advance, and detects the boarding of a user who is not scheduled to board, in other words, a user who has not made a correct boarding reservation.
[0039] The periphery monitoring sensor 30 is an autonomous sensor that recognizes the surrounding environment of the host vehicle Am. The periphery monitoring sensor 30 includes, for example, one or more of a camera unit, a millimeter wave radar, a lidar, and a sonar. The detection range of the periphery monitoring sensor 30 is set to include the entire periphery of the host vehicle Am. The periphery monitoring sensor 30 is capable of detecting moving objects and stationary objects from the detection range around the host vehicle. The periphery monitoring sensor 30 provides detection information of objects around the host vehicle to the autonomous driving ECU 50, etc.
[0040] The locator 35 includes a GNSS (Global Navigation Satellite System) receiver, an inertial sensor, etc. The locator 35 sequentially determines the position and traveling direction of the host vehicle Am by combining positioning signals received from a plurality of positioning satellites by the GNSS receiver, the measurement results of the inertial sensor, and vehicle speed information output to the communication bus, etc. The locator 35 sequentially outputs position information and direction information of the host vehicle Am based on the positioning results to the communication bus as locator information.
[0041] Locator 35 further has a map database (hereinafter, map DB) that stores map data. The map DB is mainly composed of a large-capacity storage medium that stores a large amount of three-dimensional map data and two-dimensional map data. The three-dimensional map data is a so-called HD (High Definition) map, and includes road information necessary for autonomous driving. Locator 35 reads map data around the current position from the map DB, and provides it to autonomous driving ECU 50 etc. together with locator information.
[0042] The navigation ECU 38 sets a driving route from the current position to the destination based on point information of the current position based on the locator information and point information of the destination, waypoints, etc. determined by the remote side system 130. The navigation ECU 38 may use the driving route set by the remote side system 130. The navigation ECU 38 provides route information indicating the driving route to the destination to the autonomous driving ECU 50, etc.
[0043] The in-vehicle communication device 39 is an external communication unit mounted on the vehicle Am. The in-vehicle communication device 39 functions as a V2X (Vehicle to Everything) communication device. The in-vehicle communication device 39 performs wireless communication (V2I communication, V2V communication, etc.) between roadside devices installed on the side of the road and other vehicles around the vehicle, etc., and transmits and receives various information. The in-vehicle communication device 39 receives signal information indicating the lighting pattern of traffic signals, as well as detection information of stopped vehicles, parked vehicles, pedestrians, cyclists, etc. from the roadside devices and other vehicles.
[0044] The in-vehicle communication device 39 enables information sharing between the autonomous driving ECU 50 and the remote control device 140 by communication with the center communication device 131 via a public communication line network (V2N communication). The in-vehicle communication device 39 transmits vehicle information indicating the state of the host vehicle Am to the remote side system 130. The vehicle information includes an outside-vehicle camera image captured by the camera unit of the periphery monitoring sensor 30, an inside-vehicle camera image captured by the inside-vehicle monitor device 29 (in-vehicle camera), and the like. The in-vehicle communication device 39 receives input information, remote driving information, monitor camera images, and use application information (boarding application), and the like, from the remote side system 130. The in-vehicle communication device 39 provides the information received from the roadside device, other vehicles, and the remote side system 130 to the HMI control device 10, the autonomous driving ECU 50, and the like.
[0045] The driving control ECU 40 is an electronic control device mainly including a microcontroller. The driving control ECU 40 has at least the functions of a brake control ECU, a drive control ECU, and a steering control ECU. The driving control ECU 40 is electrically connected to an operation amount detection sensor 115. The operation amount detection sensor 115 outputs in-vehicle driving information based on driving operations input to the in-vehicle operation unit 110 to the driving control ECU 40. The driving control ECU 40 acquires from the operation amount detection sensor 115 a plurality of driving instruction values indicating the accelerator operation amount, the brake operation amount, and the steering operation amount input to the in-vehicle operation unit 110 as in-vehicle driving information.
[0046] When the automatic driving ECU 50 is performing automatic driving, the driving control ECU 40 continuously performs brake force control for each wheel, output control of the in-vehicle power source, and steering angle control based on the control command generated by the automatic driving ECU 50. On the other hand, when the host vehicle Am is in a remote driving state, the driving control ECU 40 continuously performs brake force control for each wheel, output control of the in-vehicle power source, and steering angle control according to the command (remote driving information, driving instruction value) of the remote side system 130 input from the in-vehicle communication device 39. Furthermore, when the passenger PS is permitted to drive inside the vehicle, the driving control ECU 40 continuously performs brake force control for each wheel, output control of the in-vehicle power source, and steering angle control according to the in-vehicle driving information (driving instruction value) acquired from the operation amount detection sensor 115.
[0047] The body ECU 43 is an electronic control device mainly including a microcontroller. The body ECU 43 controls a door lock actuator 45 mounted on the host vehicle Am. The door lock actuator 45 switches the state of the lock mechanism of the door provided on the host vehicle Am between an unlocked state and a locked state based on a command signal from the body ECU 43. The door lock actuator 45 may be capable of switching the state of the lock mechanism for the inside and outside of the vehicle separately.
[0048] The HMI (Human Machine Interface) control device 10 is a computer mainly including a control circuit having a processing unit, a RAM, a storage unit, an input / output interface, and a bus connecting these. The HMI control device 10 constitutes an in-vehicle notification system together with an in-vehicle display 21, an in-vehicle speaker 22, an external display 24, and an external speaker 25. The HMI control device 10 functions as a presentation control device and integrally controls the presentation of information inside the vehicle using the in-vehicle display 21 and the in-vehicle speaker 22, etc., and the presentation of information outside the vehicle using the external display 24 and the external speaker 25, etc. Based on a request for notification obtained from the automatic driving ECU 50, the HMI control device 10 issues notifications indicating the driving status of the driverless vehicle ADV and the operating status of the automatic driving function to the inside and outside of the vehicle.
[0049] The in-vehicle display 21 and the in-vehicle speaker 22 are in-vehicle notification devices provided in the cabin of the host vehicle Am. The in-vehicle display 21 displays information to the passengers PS in the cabin. Multiple in-vehicle displays 21 are provided in the cabin of the host vehicle Am so that all passengers PS can see the in-vehicle displays 21. The in-vehicle displays 21 employ color displays capable of displaying various information in color. The in-vehicle speaker 22 reproduces an alarm sound or a voice message, etc., in the cabin. Similar to the in-vehicle display 21, multiple in-vehicle speakers 22 may be provided in the cabin of the host vehicle Am.
[0050] The exterior display 24 and the exterior speaker 25 are exterior alarm devices provided on the host vehicle Am. The exterior display 24 is installed on the outer surface of the host vehicle Am and displays information to the outside of the vehicle. The exterior display 24 is a display capable of displaying characters and is provided at least near the door through which the passenger PS gets in and out. The exterior display 24 may also be provided on the front and rear surfaces of the host vehicle Am. The exterior speaker 25 plays an alarm sound or a voice message to the outside of the vehicle. The exterior display 24 and the exterior speaker 25 notify pedestrians, cyclists, drivers of other vehicles, etc. around the host vehicle of the current operating state of the automatic driving function of the host vehicle Am.
[0051] [Autonomous driving ECU] The autonomous driving ECU 50 is a computer mainly including a processing unit 51, a RAM 52, a storage unit 53, an input / output interface 54, and a control circuit including a bus connecting these units. The processing unit 51 executes various processes (instructions) for implementing the autonomous driving control method of the present disclosure by accessing the RAM 52. The storage unit 53 stores various programs (autonomous driving control program, vehicle monitoring program, etc.) executed by the processing unit 51. By executing the programs by the processing unit 51, the autonomous driving ECU 50 is configured with an information linking unit 61, an environment recognition unit 62, an action determination unit 63, a control execution unit 64, an equipment control unit 65, etc. as multiple functional units for implementing the autonomous driving function and the vehicle monitoring function (see FIG. 3).
[0052] The information linking unit 61 provides information to the HMI control device 10 and acquires information from the HMI control device 10, the in-vehicle monitor device 29, the user reception terminal 120, etc. The information linking unit 61 has a user authentication unit 70, a situation grasping unit 71, and a notification request unit 72 as sub-functional units for information linking with the HMI control device 10, the in-vehicle monitor device 29, and the user reception terminal 120.
[0053] The user authentication unit 70 obtains information about the user (passenger PS) who uses the vehicle Am. The user authentication unit 70 obtains the user's boarding application by sharing the usage application information provided by the remote control device 140 with the user reception terminal 120. Based on the boarding application, the user authentication unit 70 obtains information such as the number, age, and gender of users who plan to board the vehicle Am at each stopover point.
[0054] The situation grasping unit 71 grasps the situation of the passenger PS in the cabin of the host vehicle Am based on the in-vehicle monitor information acquired from the in-vehicle monitor device 29. When driving inside the vehicle is permitted, the situation grasping unit 71 continuously grasps whether the passenger PS is seated in the driver's seat DS and whether the passenger PS seated in the driver's seat DS is in a state where he or she can operate the vehicle.
[0055] The situation grasping unit 71 continuously monitors the interior of the vehicle cabin to detect the occurrence of an injured or suddenly ill person in the vehicle cabin, the occurrence of trouble between passengers PS in the vehicle cabin, problematic behavior of passengers PS, etc. As an example, the situation grasping unit 71 detects, as problematic behavior, the movement of the passenger PS to the driver's seat DS in a situation where driving in the vehicle is not permitted, and behavior of touching a specific object (e.g., a drive key, etc.) in the vehicle cabin of the vehicle Am. Note that the behavior of the passenger PS that is considered to be problematic behavior may be changed as appropriate.
[0056] The situation grasping unit 71 acquires operation information of operations input to the emergency stop button 117, the emergency call button 118, and the emergency evacuation button 119. A plurality of each of the buttons 117-119 are provided in the cabin of the vehicle Am. Each of the buttons 117-119 is disposed near each seat where the passenger PS sits so that the passenger PS can easily operate the button immediately. A pressing operation by the passenger PS is input to the emergency stop button 117 in a situation where it is necessary to immediately stop the traveling of the vehicle Am. A pressing operation by the passenger PS is input to the emergency call button 118 in a situation where it is necessary to report to the remote side system 130 or the like. A pressing operation by the passenger PS is input to the emergency evacuation button 119 in a situation where it is necessary to evacuate the vehicle Am from its current position.
[0057] The notification request unit 72 enables the HMI control device 10 to perform notification in synchronization with the operation state of the automatic driving function by outputting a request for notification to the HMI control device 10. The notification request unit 72 acquires control state information indicating the operation state of the automatic driving function from the action determination unit 63, and outputs an implementation request based on the control state information to the HMI control device 10. As a result, the notification request unit 72 causes the HMI control device 10 to perform notification indicating the operation state of the automatic driving function to the inside and outside of the vehicle. In addition, the notification request unit 72 outputs a request for notification to the remote control device 140 via the in-vehicle communication device 39. The notification request unit 72 uses the remote side display 136 and the remote side speaker 137 (see FIG. 2) to notify information about the vehicle Am to the remote monitor RO and the operation manager OM linked to the remote side system 130.
[0058] The environment recognition unit 62 recognizes the driving environment of the host vehicle Am by combining the locator information and map data acquired from the locator 35 with the detection information acquired from the perimeter monitoring sensor 30. The environment recognition unit 62 can also use the detection information received by the in-vehicle communication device 39 to recognize the driving environment. The environment recognition unit 62 acquires route information from the navigation ECU 38, and provides the acquired route information to the action determination unit 63.
[0059] The environment recognition unit 62 has a target recognition unit 73 and a road recognition unit 74 as sub-functional units for recognizing the driving environment. The target recognition unit 73 recognizes the relative positions and relative speeds of dynamic targets around the vehicle, such as other vehicles traveling around the vehicle Am. The road recognition unit 74 acquires road information related to the road on which the vehicle Am is traveling or the road on which the vehicle Am is scheduled to travel. The environment recognition unit 62 sequentially provides the information recognized by the target recognition unit 73 and the road recognition unit 74, i.e., the recognition result of the driving environment, to the action determination unit 63.
[0060] When the autonomous driving ECU 50 has control of driving operations, the behavior determination unit 63 generates a planned driving line along which the host vehicle Am will travel, based on the recognition result of the driving environment by the environment recognition unit 62 and the route information generated by the navigation ECU 38. The behavior determination unit 63 outputs the generated planned driving line to the control execution unit 64, thereby causing the host vehicle Am to autonomously travel along the planned driving line. The behavior determination unit 63 has a control switching unit 77 as a sub-functional unit that complements the autonomous driving function.
[0061] The control switching unit 77 cooperates with the cruise control ECU 40 to switch the control state of the host vehicle Am among automatic driving, remote driving, and in-vehicle driving. When a remote monitor RO of a remote monitoring facility CNT determines that the host vehicle Am should be switched to remote driving, the control switching unit 77 cooperates with the remote control device 140 to transition the control state of the host vehicle Am from automatic driving to remote driving. By switching to remote driving, the cruise control ECU 40 drives the host vehicle Am based on the driving instruction values received from the remote side system 130 via the in-vehicle communication device 39. When switching the control state of the host vehicle Am between automatic driving and remote driving, the control switching unit 77 always temporarily stops the host vehicle Am.
[0062] When the control state is automatic driving or remote driving, the control switching unit 77 prohibits the passenger PS from driving the vehicle by not reflecting the driving operation input to the in-vehicle operation unit 110 in the behavior of the vehicle Am in the cruise control ECU 40. On the other hand, when the remote monitor RO permits the transition to in-vehicle driving, the control switching unit 77 permits the passenger PS to input driving operations to the in-vehicle operation unit 110. When the control switching unit 77 confirms that the passenger PS is seated in the driver's seat DS and is in a state (posture) in which driving operations are possible, it transitions the control state of the vehicle Am from automatic driving to in-vehicle driving. By transitioning to in-vehicle driving, the cruise control ECU 40 drives the vehicle Am based on the driving instruction value acquired from the operation amount detection sensor 115.
[0063] When the autonomous driving ECU 50 has control of driving operations, the control execution unit 64 cooperates with the cruise control ECU 40 to execute acceleration / deceleration control, steering control, and the like of the host vehicle Am according to the planned driving line generated by the action determination unit 63. Specifically, the control execution unit 64 generates control commands based on the planned driving line and sequentially outputs the generated control commands to the cruise control ECU 40.
[0064] The device control unit 65 controls the operation of various in-vehicle devices mounted on the host vehicle Am by outputting a control command to the body ECU 43. The device control unit 65 cooperates with the body ECU 43 and controls the door lock actuator 45 to switch the state of the door lock mechanism of the host vehicle Am.
[0065] [How to respond to a person approaching your vehicle] In the driverless vehicle ADV described above, as mentioned above, there is a possibility that a person who is not scheduled to board the vehicle (hereinafter, a warning person UM, see Fig. 4) may try to enter the vehicle when the vehicle is temporarily stopped. In such a situation, if no action is taken against the warning person UM, there is a risk of the vehicle Am being stolen and of adverse effects being caused to the passenger PS.
[0066] Therefore, when the automatic driving ECU 50 detects the approach of a noticing person UM, it implements a series of measures to deal with the noticing person as shown in Fig. 4. The measures to deal with the noticing person include at least an outside alarm to the noticing person UM, door unlocking control to enable evacuation outside the vehicle, a report to the remote monitoring facility CNT (hereinafter, center report), and evacuation control to move the host vehicle Am away from the noticing person UM. Details of the approach determination of the noticing person UM and the series of measures to deal with the noticing person including the outside alarm, door unlocking control, center report, and evacuation control will be described below based on Fig. 4 and with reference to Figs. 1 to 3.
[0067] <Judgment of approaching person> The approach judgment of the notable person UM is performed by the target recognition unit 73. The target recognition unit 73 judges the approach of the notable person UM who is not scheduled to board the vehicle Am to the vehicle Am based on the detection information around the vehicle. The target recognition unit 73 refers to the boarding application by sharing information with the user authentication unit 70, and acquires passenger information such as the planned boarding location where the user is scheduled to board, and the number, age, and gender of users who are scheduled to board at each planned boarding location. Based on the passenger information, the target recognition unit 73 judges whether the person approaching the vehicle Am is a user who is scheduled to board or another notable person UM.
[0068] When the vehicle Am is stopped at a location other than the planned boarding location and the planned disembarking location (hereinafter referred to as the boarding / disembarking point) due to a traffic jam or waiting at a traffic light, the target object grasping unit 73 judges all people approaching the vehicle Am to be persons to watch UM. The boarding / disembarking points are the destination and intermediate destinations of the driverless vehicle ADV set by the operation management unit 144 based on the usage application information. The boarding / disembarking points are virtual bus stops in automatic operation. As an example, when a person to watch UM approaches to within a few meters of the stopped vehicle Am, the target object grasping unit 73 judges that the person to watch UM has approached the vehicle Am.
[0069] When the vehicle Am is stopped at a boarding / alighting point and a person of an age or gender not included in the passenger information approaches the vehicle Am, the target recognition unit 73 determines that the person is a person to beware of UM. In addition, when determining whether a person who is about to get on the vehicle Am at the boarding / alighting point is a user or a person to beware of UM, the target recognition unit 73 may use in-vehicle monitor information provided to the situation recognition unit 71 from the in-vehicle monitor device 29.
[0070] When the target grasping unit 73 grasps the inappropriate behavior of a person who is about to get into the vehicle Am, the target grasping unit 73 judges the person as a person to beware of UM. Specifically, when the target grasping unit 73 detects a person who is about to get into the vehicle Am without carrying out the normal boarding procedure, such as holding the user terminal 170 over the user reception terminal 120, based on the in-vehicle monitor information, the target grasping unit 73 judges the person as a person to beware of UM. Furthermore, the target grasping unit 73 also judges a person who tries to forcefully open a closed door or a person who tries to get into the vehicle through a door for getting off, as a person to beware of UM.
[0071] <Responses to those with caution> The outside vehicle alert in response to a noticing person is implemented by cooperation between the autonomous driving ECU 50 and the HMI control device 10, based on the approach judgment of the noticing person UM. The alert request unit 72 outputs a request to the HMI control device 10 to implement an outside vehicle alert (hereinafter, a no-boarding warning) indicating that boarding is not permitted to the noticing person UM. In response, the HMI control device 10 issues a no-boarding warning using the exterior display 24 and the exterior speaker 25. The no-boarding warning continues until movement under evacuation control is terminated.
[0072] In the no-boarding warning, a message such as "No reservation has been made" or "Customers are not allowed to use the service" is displayed on the exterior display 24 provided near the door (on the side or above, see FIG. 1 or FIG. 4). Furthermore, the same voice message as that displayed on the exterior display 24 is played outside the vehicle by the exterior speaker 25. Furthermore, a voice message giving usage guidance such as "If you wish to use the service, please complete the usage procedure through the app" may also be played.
[0073] The unlocking control is performed by cooperation between the device control unit 65 and the body ECU 43 after the initiation of the no-entry warning. When the device control unit 65 determines that the to-be-watched person UM is approaching the vehicle Am, the device control unit 65 unlocks the door on the left and right sides of the vehicle Am that is opposite to the side where the to-be-watched person UM is approaching (hereinafter, the evacuation door) so that it can be opened from inside the vehicle. The evacuation door may be an emergency exit that is not normally used.
[0074] After the evacuation side door is unlocked by the unlocking control, the notification request unit 72 outputs a request for implementing an in-vehicle notification indicating that the evacuation side door is unlocked to the HMI control device 10. The HMI control device 10 notifies the passenger PS that they can open the evacuation side door from within the vehicle cabin and evacuate to the outside of the vehicle by an in-vehicle notification (hereinafter, evacuation door guidance) using the in-vehicle display 21 and the in-vehicle speaker 22. In this evacuation door guidance, the position of the evacuation side door, how to open it, etc. may be displayed on the in-vehicle display 21.
[0075] A center report is made when the state in which it is determined that the watchdog UM is approaching continues for a predetermined time (e.g., about 10 seconds) or more after the initiation of the no-boarding warning. When the state in which it is determined that the watchdog UM is approaching continues for a predetermined time or more, the report request unit 72 cooperates with the HMI control device 10 and uses the in-vehicle display 21 and the in-vehicle speaker 22 to inquire whether or not to report the approach of the watchdog UM to the remote monitoring facility CNT. When it is necessary to report to the remote monitoring facility CNT, the HMI control device 10 uses a display and a voice message to guide the user to press the emergency report button 118. When the passenger PS presses the emergency report button 118 and the passenger PS in the vehicle cabin approves the report, the report request unit 72 makes a report to the remote monitoring facility CNT.
[0076] Evacuation control is a driving control for moving the host vehicle Am from its current position. The behavior determination unit 63 determines whether the host vehicle Am can move from its current stopped position based on the approach determination of the watchful eye UM. When the behavior determination unit 63 determines that the host vehicle Am can move, it starts driving by evacuation control. Evacuation control is started, for example, after a center report. Evacuation control may be performed only when permitted by the remote monitor RO that received the center report. Before starting driving by evacuation control, the equipment control unit 65 locks all doors so that they cannot be opened from inside or outside the vehicle.
[0077] The target recognition unit 73 judges the possibility of contact between the tolerant UM and the vehicle Am before starting travel under evacuation control. When there is a possibility of contact with the tolerant UM, the behavior judgment unit 63 suppresses the movement speed of the vehicle Am during evacuation control more than when there is no possibility of contact. Cases in which the tolerant UM has already touched the vehicle Am are included in cases in which there is a possibility of contact. When the movement speed is suppressed, the behavior judgment unit 63 starts the vehicle Am so as to maintain a slow speed (for example, about several kilometers per hour) while allowing the tolerant UM to contact the vehicle Am. The behavior judgment unit 63 moves the vehicle Am by several meters, and accelerates the vehicle Am normally when there is no possibility of contact with the tolerant UM.
[0078] When the behavior determination unit 63 has moved the host vehicle Am a predetermined distance (for example, several meters to several tens of meters), it ends the evacuation control and temporarily stops the host vehicle Am. In addition, if a boarding application for the tormented person UM is acquired by the user authentication unit 70 after the behavior determination unit 63 has started moving from the current position, the behavior determination unit 63 stops the movement of the host vehicle Am. In this case, the approach determination of the tormented person UM is canceled. Note that the utilization application information for the tormented person UM may be provided to the remote side system 130 by the user terminal 170 of the tormented person UM, or may be provided to the remote side system 130 by the user terminal 170 of the passenger PS in the vehicle cabin.
[0079] If the behavior determination unit 63 determines that the host vehicle Am cannot move after determining that the noticing person UM is approaching, the device control unit 65 locks all doors of the host vehicle Am so that they cannot be opened from the outside. In this case, the boarding prohibition warning continues. If the host vehicle Am becomes movable, the behavior determination unit 63 starts movement by evacuation control while keeping all doors locked.
[0080] The behavior determination unit 63 can perform emergency evacuation control (hereinafter, emergency evacuation control) for immediately moving the host vehicle Am, in addition to the normal evacuation control (hereinafter, normal evacuation control) described above. When the emergency evacuation button 119 provided in the vehicle cabin is operated by the passenger PS, the behavior determination unit 63 starts movement by emergency evacuation control based on this emergency evacuation operation.
[0081] In the emergency evacuation control, the behavior determination unit 63 moves the host vehicle Am to a location farther from the current position than in the normal evacuation control. The behavior determination unit 63 continues moving the host vehicle Am until it receives a stop command from at least one of the passenger PS in the vehicle cabin and the remote monitor RO of the remote monitoring facility CNT. Specifically, when the passenger PS inputs a pressing operation on the emergency stop button 117 or when the remote monitor RO inputs a stop command to the input device 133, the behavior determination unit 63 ends the emergency evacuation control and stops the host vehicle Am.
[0082] [Handling of people with warnings and emergency evacuation] Next, the details of the noting person response processing and emergency evacuation processing performed by the automatic driving ECU 50 to realize the above-described response to a person who has not taken notice and emergency evacuation control will be further described based on Fig. 5 and Fig. 6 and with reference to Fig. 1 to Fig. 4. The noting person response processing and emergency evacuation processing are started by the automatic driving ECU 50 when operation of the host vehicle Am by automatic driving or remote driving is started. The noting person response processing and emergency evacuation processing are continuously performed by the automatic driving ECU 50 until operation of the host vehicle Am is terminated.
[0083] <Handling of caution persons> In S11 of the noticing person response processing shown in Fig. 5, the environment recognition unit 62 judges whether or not the host vehicle Am is stopped. If the host vehicle Am is traveling (S11: NO), the state of waiting for the host vehicle Am to stop continues. If the host vehicle Am is stopped (S11: YES), the target recognition unit 73 performs vehicle exterior monitoring in S12, and judges whether or not the noticing person UM is approaching in S13. If the noticing person UM is not approaching (S13: NO), the processing of S11 to S13 is repeated.
[0084] If it is determined that the not to be warned person UM is approaching (S13: YES), the notification request unit 72 cooperates with the HMI control device 10 in S14 to issue an outside vehicle notification indicating that boarding is not permitted. Furthermore, based on the determination that the not to be warned person UM is approaching, the device control unit 65 unlocks the evacuation side door by the unlocking control described above in S15. In addition, in S15, an evacuation door guide is issued, which is an in-vehicle notification indicating that the evacuation side door is unlocked. This allows the passenger PS to understand that it is possible to open the evacuation side door and evacuate to the outside of the vehicle.
[0085] In S16, the target recognition unit 73 judges whether the approaching state of the to-be-watched person UM has continued for a predetermined time or more. If the approaching state of the to-be-watched person UM has not continued for the predetermined time or more (S16: NO), the processes of S11 to S16 are repeated. On the other hand, if the approach of the to-be-watched person UM has continued for the predetermined time or more (S16: YES), the notification request unit 72 performs a process of confirming the necessity of a center report in S17. If the execution of the center report is approved by the passenger PS, that is, if the emergency report button 118 is operated, the notification request unit 72 executes a center report to the remote monitoring facility CNT.
[0086] In S18, the behavior determination unit 63 determines whether or not the host vehicle Am can move from its current position. If the host vehicle Am cannot move (S18: NO), the equipment control unit 65 locks all the doors of the host vehicle Am in S19. Then, in S20, the target grasping unit 73 determines whether or not the watcher UM has disappeared. If the watcher UM has disappeared (S20: YES), the watcher response processing is temporarily terminated. On the other hand, if the watcher UM has not disappeared and is still being detected (S20: NO), the behavior determination unit 63 waits until the host vehicle Am becomes movable.
[0087] If the host vehicle Am is movable (S18: YES), the target grasping unit 73 determines in S21 whether or not there is a possibility of contact with the host vehicle Am to the watcher UM. If there is a possibility of contact with the host vehicle Am to the watcher UM (S21: YES), the behavior determining unit 63 applies a setting in S22 that suppresses the movement speed of the host vehicle Am more than when there is no possibility of contact. On the other hand, if there is no possibility of contact with the host vehicle Am to the watcher UM (S21: NO), such a speed suppression setting is not applied. In S23, the behavior determining unit 63 starts moving the host vehicle Am from its current position using normal evacuation control while continuing the no-boarding warning.
[0088] After starting the movement of the host vehicle Am, the behavior determination unit 63 determines whether or not the stop condition of the host vehicle Am is satisfied in S24. The behavior determination unit 63 determines that the stop condition is satisfied when the approach of the to-be-watched person UM is resolved, when the vehicle has moved a predetermined distance, when the passenger PS inputs a pressing operation on the emergency stop button 117, or when a boarding application for the to-be-watched person UM is acquired, etc. If the stop condition is not satisfied (S24: NO), the behavior determination unit 63 waits for the stop determination to be satisfied. On the other hand, if the stop condition is satisfied (S24: YES), the behavior determination unit 63 ends the movement of the host vehicle Am by the normal evacuation control in S25. If the to-be-watched person UM approaches the host vehicle Am again after evacuation, the automatic driving ECU 50 again performs the to-be-watched person response process to move the host vehicle Am away from the to-be-watched person UM.
[0089] <Emergency evacuation procedures> In S41 of the emergency evacuation processing shown in Fig. 6, the situation grasping unit 71 judges whether or not an emergency evacuation operation has been performed on the emergency evacuation button 119 by the passenger PS. If there has been no emergency evacuation operation (S41: NO), the judgment of S41 is repeated, thereby maintaining a state of waiting for the input of an emergency evacuation operation. On the other hand, if there has been an emergency evacuation operation (S41: YES), emergency evacuation control is started. Emergency evacuation control is carried out with priority over normal evacuation control. In S42, the equipment control unit 65 locks all doors of the host vehicle Am so that they cannot be opened from inside or outside the vehicle.
[0090] In S43, the behavior determination unit 63 determines whether or not the host vehicle Am can move from its current position. If the host vehicle Am cannot move (S43: NO), the situation grasping unit 71 determines in S44 whether or not an operation to cancel the emergency evacuation control has been performed. As an example, the operation of pressing the emergency stop button 117 by the passenger PS or the input operation to the input device 133 by the remote monitor RO is regarded as the operation to cancel the emergency evacuation control. If a cancellation operation has been performed (S44: YES), the emergency evacuation processing returns to a state waiting for the input of an emergency evacuation operation. On the other hand, if a cancellation operation has not been performed (S44: NO), the behavior determination unit 63 waits for the host vehicle Am to become movable.
[0091] If the host vehicle Am is movable (S43: YES), the behavior determination unit 63 starts emergency movement of the host vehicle Am from the current position by emergency evacuation control in S45. At this time, the notification request unit 72 may cooperate with the HMI control device 10 to perform an outside-vehicle notification to notify the host vehicle of the start of movement. The behavior determination unit 63 determines whether or not a stop instruction has been acquired from at least one of the passenger PS and the remote monitor RO in S46. If a stop instruction has not been acquired (S46: NO), the determination in S46 is repeated to wait for the acquisition of a stop instruction. As a result, the behavior determination unit 63 continues traveling of the host vehicle Am in a travelable direction until a stop instruction is acquired from the passenger PS or the remote monitor RO. Then, when a stop instruction is acquired (S46: YES), the behavior determination unit 63 ends the movement of the host vehicle Am by emergency evacuation control in S47.
[0092] [Responding to problematic behavior by passengers] In addition to responding to warning persons UM outside the vehicle as described above, the automatic driving ECU 50 responds to problematic behavior by passengers PS inside the vehicle cabin by continuously monitoring the interior of the vehicle. Details of the behavior warning process and theft notification process performed by the automatic driving ECU 50 will be further explained below based on Figs. 7 and 8 and with reference to Figs. 1 to 4. The behavior warning process and theft notification process are started by the automatic driving ECU 50 when operation of the host vehicle Am by automatic driving or remote driving begins. The behavior warning process and theft notification process are started by the automatic driving ECU 50 at predetermined time intervals until operation of the host vehicle Am ends.
[0093] <Behavior warning processing> The behavior warning process shown in Fig. 7 is a process for issuing an in-vehicle warning against problem behavior of a passenger PS while aboard. In S61 of the behavior warning process, the situation grasping unit 71 performs in-vehicle monitoring using in-vehicle monitor information. Based on the in-vehicle monitor information acquired in S61, the situation grasping unit 71 determines in S62 whether or not problem behavior of the passenger PS has been detected by the in-vehicle monitor device 29. As described above, the in-vehicle monitor device 29 detects, as problem behavior, the movement of the passenger PS to the driver's seat DS in a situation where driving in the vehicle is not permitted, and the contact of the passenger PS with the drive key. If no problem behavior is detected (S62: NO), the current behavior warning process is terminated.
[0094] On the other hand, when the situation grasping unit 71 determines that a problematic behavior has been detected (S62: YES), the notification request unit 72 cooperates with the HMI control device 10 in S63 to issue an in-vehicle warning associated with the detected problematic behavior. Specifically, when the movement of the passenger PS to the driver's seat DS is detected, the notification request unit 72 issues an in-vehicle warning to encourage the passenger to leave the driver's seat DS, using at least one of the in-vehicle display 21 and the in-vehicle speaker 22. In this case, the notification request unit 72 may issue an in-vehicle warning to caution the passenger not to touch the in-vehicle operation unit 110. In addition, when the touch of the passenger PS to the drive key is detected, the notification request unit 72 issues an in-vehicle warning to caution the passenger not to touch the drive key, using at least one of the in-vehicle display 21 and the in-vehicle speaker 22.
[0095] After issuing an in-vehicle warning, the notification request unit 72 notifies the remote monitor RO of the remote monitoring facility CNT that remotely monitors the vehicle Am of the occurrence of problematic behavior by the passenger PS in S64. In such a center report, the remote monitor RO is also notified of the content of the problematic behavior. Specifically, the remote monitor RO is notified of the occurrence of the passenger PS moving to the driver's seat DS, touching the drive key, etc., by the center report. As described above, the remote monitor RO can determine whether the behavior of the passenger PS is not problematic behavior but is an action to respond to an emergency by directly checking with the passenger PS.
[0096] <Theft notification processing> The theft notification process shown in Fig. 8 is a process for notifying the surroundings of the vehicle and the remote monitoring facility CNT of the occurrence of theft (bus jacking) of the driverless vehicle ADV. In S71 of the theft notification process, the situation grasping unit 71 determines whether or not a theft has occurred. If a theft has not occurred (S71: NO), the current theft notification process is terminated.
[0097] On the other hand, if it is determined that a theft has occurred (S71: YES), the notification request unit 72 notifies the remote monitoring facility CNT of the theft in S72. The notification request unit 72 may also notify the police, a security company, or the like in addition to the remote monitoring facility CNT. Furthermore, in S73, the notification request unit 72 cooperates with the HMI control device 10 to notify the surroundings of the vehicle of the theft using the exterior display 24. As an example, a text message such as "A vehicle has been stolen. Please move away from this vehicle" is displayed in a flashing manner on the exterior display 24. The exterior notification notifying the theft of the driverless vehicle ADV continues until a predetermined stop operation is input by the remote monitor RO of the remote monitoring facility CNT.
[0098] (Summary of the first embodiment) In the first embodiment described so far, an external warning is given to a torrent person UM approaching the host vehicle Am, indicating that boarding is not permitted, using the in-vehicle display 21 and the in-vehicle speaker 22 provided in the host vehicle Am. This external warning can prevent a torrent person UM who is not scheduled to board the host vehicle Am from getting into the host vehicle Am. Therefore, even if a driver is not on board, it is possible to respond to a torrent person UM approaching the host vehicle Am. As a result, it is possible to reduce the risk of theft of a vehicle operated as a driverless vehicle ADV, the risk of adverse effects on a passenger PS boarding the driverless vehicle ADV, and the like.
[0099] Additionally, in the first embodiment, when a person UM who is not to be warned approaches the host vehicle Am while the vehicle is stopped, the behavior determination unit 63 determines whether the host vehicle Am can move from the current position while the vehicle is stopped. If the behavior determination unit 63 determines that the host vehicle Am can move, the host vehicle Am moves from the current position. By moving the host vehicle Am away from the person UM who is not to be warned by such evacuation control, the risk to the host vehicle Am caused by the person UM who is not to be warned can be reliably reduced.
[0100] Furthermore, in the first embodiment, when the emergency evacuation button 119 provided in the vehicle cabin is operated by the passenger PS, the behavior determination unit 63 moves the host vehicle Am to a location farther away from the current location than in the case where the emergency evacuation button 119 is not operated. According to such emergency evacuation control, it is possible to effectively reduce the risk to the host vehicle Am caused by the to-be-watched person UM by utilizing the judgment of the passenger PS.
[0101] Furthermore, in the first embodiment, the notification request unit 72 determines the possibility of contact between the tolerant person UM and the vehicle Am. If there is a possibility of contact with the tolerant person UM, the behavior determination unit 63 suppresses the movement speed more than when there is no possibility of contact. As a result, even if there is a possibility of contact with the tolerant person UM, it is possible to reliably move the vehicle Am away from the tolerant person UM while allowing contact with the tolerant person UM. As a result, the risk to the vehicle Am caused by the tolerant person UM can be reliably reduced.
[0102] Additionally, in the first embodiment, when the behavior determination unit 63 determines that the host vehicle Am cannot be moved, the doors of the host vehicle Am are locked so that they cannot be opened from the outside. As a result, even in a scene in which the host vehicle Am cannot be moved away from the watchful person UM, the risk of adverse effects on the passenger PS in the vehicle compartment can be reliably reduced.
[0103] Also, in the first embodiment, an application to board the host vehicle Am is acquired by the user authentication unit 70. Then, after the host vehicle Am starts moving from its current position, if a boarding application for the watchdog UM is acquired by the user authentication unit 70, the behavior determination unit 63 stops the movement of the host vehicle Am. As a result of the above, it is possible to avoid a situation in which a user who has no intention of harming the driverless vehicle ADV and has simply not made a boarding application continues to be determined to be a watchdog UM.
[0104] Furthermore, in the first embodiment, the behavior determination unit 63 continues moving the host vehicle Am until a stop command is received from at least one of the passenger PS in the vehicle cabin and the remote monitor RO that remotely monitors the host vehicle Am. As a result, the host vehicle Am can be reliably evacuated to a position where it is not adversely affected by the cautionary person UM.
[0105] Additionally, in the first embodiment, when the state in which the approach of the person to be warned UM is determined continues for a predetermined time or more, an inquiry is made using the in-vehicle display 21 and the in-vehicle speaker 22 provided in the vehicle cabin as to whether or not to report the approach of the person to be warned UM to the remote monitoring facility CNT. Then, when the passenger PS in the vehicle cabin approves the execution of the report, the notification request unit 72 issues a report to the remote monitoring facility CNT. As described above, when a person to be warned UM determined to be high risk by the passenger PS approaches the vehicle Am, the remote monitor RO of the remote monitoring facility CNT can grasp the approach of the person to be warned UM without fail.
[0106] In the first embodiment, movement of the passenger PS to the driver's seat DS closest to the in-vehicle operation unit 110 provided in the host vehicle Am is detected. When it is determined that the passenger PS has moved to the driver's seat DS, the notification request unit 72 issues an in-vehicle warning to encourage the passenger to leave the driver's seat DS using the in-vehicle display 21 and the in-vehicle speaker 22 provided in the vehicle cabin. As a result of the above, it is possible to appropriately deter problematic behavior of the passenger PS in the vehicle cabin that may lead to theft of the driverless vehicle ADV.
[0107] Furthermore, in the first embodiment, after an in-vehicle warning is issued to encourage the driver to leave the driver's seat DS, the remote monitor RO that remotely monitors the vehicle Am is notified of the passenger PS's movement to the driver's seat DS. Therefore, the remote monitor RO can quickly and thoroughly grasp the behavior of the passenger PS that may lead to theft of the driverless vehicle ADV.
[0108] Additionally, in the first embodiment, it is detected that the passenger PS has touched the drive key in the vehicle interior of the vehicle Am. When it is determined that the passenger PS has touched the drive key, the notification request unit 72 issues an in-vehicle warning to the passenger PS not to touch the drive key using the in-vehicle display 21 and the in-vehicle speaker 22 provided in the vehicle interior. This in-vehicle warning also makes it possible to appropriately deter problematic behavior of the passenger PS that may lead to theft of the driverless vehicle ADV.
[0109] In the first embodiment, when it is determined that the watchful person UM is approaching the vehicle Am, the evacuation door on the opposite side of the vehicle Am from the side where the watchful person UM is approaching is unlocked so that it can be opened from inside the vehicle. Therefore, if the watchful person UM is a high-risk person, the passenger PS can open the evacuation door and evacuate to the outside of the vehicle. As a result, the risk of adverse effects on the passenger PS can be more reliably reduced.
[0110] Furthermore, in the first embodiment, after the evacuation door is unlocked, an in-vehicle notification is given to indicate that the evacuation door is unlocked using the in-vehicle display 21 and the in-vehicle speaker 22 provided in the vehicle cabin. As a result, the passenger PS can be sure that he or she can open the evacuation door and evacuate to the outside of the vehicle. As a result, the risk of adverse effects on the passenger PS can be further reduced.
[0111] In the first embodiment, the remote monitoring facility CNT corresponds to the "report destination", the remote monitor RO corresponds to the "monitor", the in-vehicle display 21 and the in-vehicle speaker 22 correspond to the "in-vehicle alarm", and the outside display 24 and the outside speaker 25 correspond to the "exterior alarm". The autonomous driving ECU 50 corresponds to the "vehicle monitoring device", the action determination unit 63 corresponds to the "action control unit", the equipment control unit 65 corresponds to the "lock control unit", and the user authentication unit 70 corresponds to the "application acquisition unit". Furthermore, the situation grasping unit 71 corresponds to the "in-vehicle monitoring unit", the notification request unit 72 corresponds to the "notification implementation unit", the target grasping unit 73 corresponds to the "approach determination unit", and the in-vehicle operation unit 110 corresponds to the "driving operation unit".
[0112] Second Embodiment The second embodiment of the present disclosure is a modified example of the first embodiment. The autonomous driving ECU 50 of the second embodiment shown in Fig. 9 further includes a video distribution unit 66 and a security function unit 67 by executing a program by the processing unit 51. Hereinafter, the details of the video distribution process, the caution person response process, theft notification process, and the takeover response process performed by the autonomous driving ECU 50 of the second embodiment will be described based on Figs. 10 to 13 with reference to Figs. 1, 2, and 9.
[0113] <Video distribution processing> The video distribution process shown in FIG. 10 is a process of distributing in-vehicle camera video from the in-vehicle monitor device 29 (in-vehicle camera) to the user terminal 170 linked to the user of the driverless vehicle ADV in approximately real time. The video distribution process is mainly performed by the video distribution unit 66. A user who is scheduled to board the driverless vehicle ADV and has made a reservation (approver) whose boarding application has been accepted by the remote side system 130 requests distribution of in-vehicle camera video from an application on the user terminal 170. The video distribution unit 66 starts the video distribution process based on obtaining the distribution request from the user terminal 170.
[0114] In S201 of the video distribution process, the video distribution unit 66 identifies the user who is the requester of the distribution. The video distribution unit 66 uses the usage application information provided by the remote control device 140 and the user identification information acquired together with the distribution request, etc., to identify the user. In S202, the video distribution unit 66 determines whether the user identified in S201 is an approved user whose boarding application has been accepted. If the user who has requested the distribution is not an approved user (S202: NO), the video distribution unit 66 does not provide the in-car camera video. In S205, the video distribution unit 66 transmits an error message to the requesting user terminal 170.
[0115] If the user who has made the distribution request is an approved user (S202: YES), the video distribution unit 66 determines in S203 whether or not it is a distribution possible period in which the in-vehicle camera video can be provided to the approved user. Whether or not it is a distribution possible period is determined based on the positional relationship between the approved user's planned boarding point and the current position of the vehicle Am, the time relationship between the approved user's planned boarding time and the current time, etc.
[0116] The video distribution unit 66 acquires information indicating the planned boarding point of the approved user and information indicating the current position of the vehicle Am. The video distribution unit 66 determines that the distribution is possible during a period when the vehicle Am approaches within a predetermined distance (e.g., about several hundreds of meters to 1 km) from the planned boarding point. The video distribution unit 66 also acquires information indicating the planned boarding time of the approved user and information indicating the current time. The video distribution unit 66 determines that the distribution is possible during a period when the planned boarding time is within a predetermined time (e.g., about several minutes to 10 minutes). The video distribution unit 66 may permit distribution of the in-car camera video when one of the above-mentioned position condition and time condition is satisfied, or may permit distribution of the in-car camera video when both conditions are satisfied.
[0117] Video distribution unit 66 continues the delivery permitted period until a predetermined time (for example, several minutes to 10 minutes) has elapsed after the approved user gets off the vehicle, and continues to permit delivery of the in-vehicle camera video to user terminal 170. Video distribution unit 66 ends the delivery permitted period when the predetermined time has elapsed after the approved user gets off the vehicle.
[0118] When the video distribution unit 66 determines that it is not a period during which distribution to an approved user is possible (S203: NO), it does not provide the in-vehicle camera video, and transmits an error message to the requesting user terminal 170 in S205. In this case, for example, a notification such as "It is outside the period during which distribution is possible" is transmitted to the user terminal 170 of the approved user. On the other hand, when it determines that it is a period during which distribution to an approved user is possible (S203: YES), the video distribution unit 66 starts distribution of the in-vehicle camera video from the in-vehicle camera to the user terminal 170 in S204.
[0119] When multiple in-vehicle cameras are installed in the vehicle cabin, the video distribution unit 66 switches between the in-vehicle camera videos to be provided to the user terminal 170 in response to a request from the user terminal 170. In addition, the video distribution unit 66 may be able to change the resolution of the in-vehicle camera video to be provided to the user terminal 170 in response to a request from the user terminal 170, and may enable zooming in and out of the in-vehicle camera video.
[0120] Some or all of the steps of the above video distribution process may be performed by the remote control device 140. That is, a functional unit equivalent to the video distribution unit 66 may be constructed in the remote control device 140, not in the autonomous driving ECU 50. The video distribution unit 66 provided in the remote control device 140 acquires in-vehicle camera video from the driverless vehicle ADV, and provides the acquired in-vehicle camera video to the user terminal 170 linked to the approved user.
[0121] <Handling of cautionary individuals> The noting person response process shown in Fig. 11 is a process for responding to a noting person UM approaching the host vehicle Am at a location different from a preset boarding / disembarking point. The noting person response process is mainly performed by the user authentication unit 70, the notification request unit 72, the target recognition unit 73, the safety function unit 67, etc. In the noting person response process, the passenger PS in the vehicle cabin determines whether or not personal authentication of the noting person UM may be performed. As in the first embodiment, the noting person response process is started based on the start of operation of the host vehicle Am by automatic driving or remote driving, and is continuously performed until the operation of the host vehicle Am ends.
[0122] In S211 of the notifier response process, the environment recognition unit 62 judges whether or not the host vehicle Am is stopped at a location other than the boarding / alighting point. When the host vehicle Am is traveling or stopped at a boarding / alighting point (S211: NO), the host vehicle continues to wait for stopping at a location other than the boarding / alighting point. On the other hand, when the host vehicle Am is stopped at a location other than the boarding / alighting point (S211: YES), the target recognition unit 73 performs vehicle exterior monitoring in S212 and judges whether or not a notifier UM is approaching in S213. When the notifier UM is not approaching (S213: NO), the processes of S211 to S213 are repeated.
[0123] If it is determined that the watchdog UM is approaching (S213: YES), the notification request unit 72 cooperates with the HMI control device 10 and uses the in-vehicle display 21 and the in-vehicle speaker 22 to inquire of the passenger PS in the vehicle cabin as to whether or not to perform personal authentication of the watchdog UM in S214. In this case, the in-vehicle inquiry displays a message such as "If you know someone outside the vehicle, please let us know" on the in-vehicle display 21. The in-vehicle display 21 further displays a touch icon such as "Allow boarding". In addition, the in-vehicle speaker 22 reproduces the same voice message as that displayed on the in-vehicle display 21 in the vehicle cabin.
[0124] In S215, the situation grasping unit 71 judges whether or not there is in-vehicle approval from the passenger PS. The passenger PS performs in-vehicle approval by inputting a touch operation on a touch icon, etc. The passenger PS may perform in-vehicle approval using an application of the user terminal 170. If there is no in-vehicle approval from the passenger PS (S215: NO), the notification request unit 72 performs a center report in S216. As a result, the remote monitoring facility CNT is notified of the approach of a watchful person UM not connected to the passenger PS to the vehicle Am. Furthermore, in S219, the notification request unit 72 cooperates with the HMI control device 10 to perform an outside notification using the outside display 24 and the outside speaker 25. In this case, a no-boarding warning indicating that boarding is not permitted is issued to the watchful person UM.
[0125] On the other hand, if there is in-vehicle approval by the passenger PS (S215: YES), the user authentication unit 70 performs personal authentication of the not to beware UM in S217. If the not to beware UM is registered in advance as a user of the mobility service, the not to beware UM can complete personal authentication, for example, by holding the user terminal 170 over the user reception terminal 120. The personal authentication may be performed using an application on the user terminal 170, or may be performed by image analysis of an outside-vehicle camera image.
[0126] If personal authentication is completed (S217: YES), the security function unit 67, in S218, determines that the alert person UM is an authenticated user and allows him / her to board the vehicle Am. In this case, the boarding door is unlocked through cooperation between the equipment control unit 65 and the body ECU 43. As a result, the user detected as the alert person UM is allowed to board the vehicle. On the other hand, if personal authentication is not completed (S217: NO), the security function unit 67 does not allow the alert person UM to board. In this case, in S219, a no-boarding warning indicating that boarding is not permitted is also issued to the alert person UM.
[0127] <Theft notification processing> The theft notification process shown in Fig. 12 is a process for issuing an external warning that the vehicle Am has become a stolen vehicle when the vehicle Am is stolen from a parked vehicle outside of operating hours. The theft notification process assumes that the vehicle Am is transported in a non-self-propelled state by a tow truck or a forwarding vehicle. The theft notification process is started by the security function unit 67 based on the execution of a function-on process for making the vehicle Am drivable, such as by turning on the main power or the ignition.
[0128] In S231 of the theft notification process, the security function unit 67 determines whether the theft detection function is on or not. The theft detection function can only be turned off by an operator with special authority, such as the operations manager OM. To switch the theft detection function off, input of a password or the like may be required. For example, the theft detection function is turned off when the vehicle Am is being transported by the legitimate owner. If the theft detection function is off (S231: NO), the security function unit 67 determines in S235 that it is in a normal state and ends the theft notification process.
[0129] When the theft detection function is on (S231: YES), the security function unit 67 grasps the current position of the host vehicle Am in S232. The security function unit 67 cooperates with the environment recognition unit 62 and the locator 35 to acquire the position information of the host vehicle Am grasped based on the function on processing as information indicating the activation location. Furthermore, the security function unit 67 reads out information indicating the stopping location stored at the time of the previous function off processing from the position memory unit 79 in S233. The position memory unit 79 acquires the position information of the host vehicle Am based on the function off processing of the host vehicle Am, such as turning off the main power supply or the ignition. The position memory unit 79 stores the acquired position information as information indicating the stopping location of the host vehicle Am. The order of execution of the above S232 and S233 may be changed as appropriate.
[0130] In S234, the security function unit 67 determines whether the location information indicating the start location and the location information indicating the stop location are consistent. If the difference between the start location and the stop location is equivalent to the accuracy of the GNSS (for example, within a few meters), the security function unit 67 determines that the start location and the stop location are consistent. If the start location and the stop location are consistent (S234: YES), the security function unit 67 determines in S235 that theft has not occurred and that the vehicle is in a normal state.
[0131] On the other hand, if the start location and the stop location do not match (S234: NO), the security function unit 67 determines in S236 that a theft has occurred. In this case, the notification request unit 72 issues an external vehicle warning indicating that a theft has occurred. Specifically, the notification request unit 72 causes the external vehicle display 24 to display a text message such as "Your vehicle has been stolen. This vehicle is a stolen vehicle." In addition, the notification request unit 72 causes the external vehicle speaker 25 to play a voice message saying "A theft has occurred" or a loud alarm sound. Furthermore, the notification request unit 72 transmits a notice indicating that a theft has occurred together with the position information of the start location acquired in S232 to the remote monitoring facility CNT, the police, the security company, and the like.
[0132] <Hookup response processing> The takeover response process shown in FIG. 13 is a process for reporting the occurrence of theft of the vehicle Am to the remote monitoring facility CNT. In the takeover response process, it is assumed that a thief takes over some of the functions of the vehicle-side system IVS. The takeover response process enables reporting to the remote monitoring facility CNT by cooperating with the user terminal 170 of the passenger PS even when some of the vehicle-side system IVS is not functioning. The takeover response process is started based on the start of operation of the vehicle Am by automatic driving or remote driving, and is continuously performed until the operation of the vehicle Am ends.
[0133] In S251 of the takeover response process, the security function unit 67 determines whether or not a theft has occurred based on the situation inside the vehicle compartment grasped by the situation grasping unit 71. If a theft has not occurred (S251: NO), the current takeover response process is terminated. On the other hand, if it is determined that a theft has occurred (S251: YES), the security function unit 67 determines whether or not an abnormality has occurred in the vehicle-side system IVS, in other words, whether or not a takeover of a function has occurred in connection with the theft.
[0134] Specifically, in S252, the security function unit 67 judges whether or not there is an abnormality in the external communication system mainly including the in-vehicle communication device 39. When there is an abnormality in the external communication system (S252: YES) and there is a possibility that a report cannot be made to the remote monitoring facility CNT, etc., the security function unit 67 performs cooperation with the user terminal 170 in the vehicle cabin in S253. The user terminal 170 to be the cooperation destination is the user terminal 170 owned by the passenger PS, and is, for example, the user terminal 170 connected to the in-vehicle Wi-Fi (registered trademark) of the vehicle Am. The security function unit 67 may cooperate with a plurality of user terminals 170 in the vehicle cabin. In S259, the security function unit 67 requests the user terminal 170 that cooperated in S253 to perform a center report. Based on the request from the security function unit 67 in S259, the user terminal 170 transmits the current location information acquired by the terminal itself to the remote monitoring facility CNT, etc., together with a notice of the theft.
[0135] On the other hand, if there is no abnormality in the exterior vehicle communication system (S252: NO), the security function unit 67 determines in S254 whether or not there is an abnormality in the positioning system mainly including the locator 35. If there is an abnormality in the positioning system (S254: YES) and there is a possibility that correct position information of the vehicle Am cannot be acquired, the security function unit 67 performs a link with the user terminal 170 in the vehicle cabin in S256, as in S253. The security function unit 67 acquires the latest position information from the user terminal 170 linked in S256 in S257.
[0136] If there is no abnormality in both the exterior communication system and the positioning system (S254: NO), in S255 the security function unit 67 acquires the position information of the vehicle Am from the locator 35. In S258, the notification request unit 72 uses the in-vehicle communication device 39 to notify the center of the occurrence of the theft. In this center notification, the position information acquired in S255 or S257 and indicating the current position of the vehicle Am is transmitted to the remote monitoring facility CNT or the like. Furthermore, in S260, the notification request unit 72 also notifies the surroundings of the vehicle of the occurrence of the theft using the exterior display 24 and the exterior speaker 25.
[0137] (Summary of the second embodiment) In the second embodiment described above, the outside alarm to the to-be-watched person UM approaching the host vehicle Am is also provided to prevent the to-be-watched person UM from getting into the host vehicle Am. As a result, the same effect as in the first embodiment is achieved, and it becomes possible to respond to the to-be-watched person UM approaching the host vehicle Am even when the driver is not in the vehicle.
[0138] In addition, in the second embodiment, when it is determined that the to-be-watched person UM approaches the vehicle Am at a location different from the pre-set boarding and disembarking point, an inquiry as to whether or not to perform personal authentication of the to-be-watched person UM is made using the in-vehicle display 21 and in-vehicle speaker 22 provided in the vehicle cabin. Then, when the performance of personal authentication is permitted by the passenger PS in the vehicle cabin, personal authentication of the to-be-watched person UM is performed. Therefore, even if a person related to the passenger PS of the vehicle Am approaches the vehicle Am without performing the proper procedure, it is possible to have this person board the vehicle Am with the cooperation of the passenger PS. As a result, the convenience of the driverless vehicle ADV is further improved.
[0139] In the second embodiment, in-vehicle camera video is distributed to the user terminal 170 based on a request from the user terminal 170 linked to an approved user whose reservation for boarding the vehicle Am has been accepted. Therefore, a user who plans to use the driverless vehicle ADV can check before boarding whether there is a suspicious person or the like on board, whether it is excessively crowded, etc. In addition, if the driverless vehicle ADV is a shuttle vehicle for ride-hailing and is not a vehicle intended for carpooling, the user can check before boarding that no other passengers have boarded by mistake.
[0140] Furthermore, in the second embodiment, when the vehicle Am approaches within a predetermined distance from the intended boarding point of the approved user, delivery of the in-vehicle camera video to the user terminal 170 is permitted. By setting such a delivery possible period, the delivery destination of the in-vehicle camera video can be limited to the range of appropriate users.
[0141] Additionally, in the second embodiment, when the approved user is within a predetermined time until the scheduled boarding time, delivery of the in-car camera video to the user terminal 170 is permitted. By setting such a delivery possible period, the delivery destination of the in-car camera video can be limited to the range of appropriate users.
[0142] In the second embodiment, after the approved user gets off the vehicle, delivery of the in-vehicle camera image to the user terminal 170 is permitted until a predetermined time has elapsed. As described above, the user can check whether or not he or she has left something behind in the vehicle by viewing the in-vehicle camera image after getting off the vehicle.
[0143] Furthermore, in the second embodiment, the stopping location of the host vehicle Am is stored in the position storage unit 78 based on the function-off processing of the host vehicle Am. Then, if the start location of the host vehicle Am determined based on the function-on processing is different from the stopping location, an outside-vehicle warning indicating the occurrence of theft is implemented. As a result of the above, even if the host vehicle Am is stolen after the function-off processing and taken far away, the processing leading to the discovery of the host vehicle Am can be reliably executed.
[0144] Additionally, in the second embodiment, when a theft such as a bus jacking occurs on the vehicle Am while it is in operation, the location information of the user terminal 170 linked to the passenger PS in the vehicle cabin is notified to the remote monitoring facility CNT. Therefore, even in a situation where a part of the vehicle-side system IVS is not functioning due to theft and it is impossible to report from the vehicle-side system IVS to the remote monitoring facility CNT, a report can be sent to the remote monitoring facility CNT by using the function of the user terminal 170 or its application. In the second embodiment, the in-vehicle monitor device 29 corresponds to the "in-vehicle camera."
[0145] Third embodiment The third embodiment of the present disclosure is a modified example of the second embodiment. The autonomous driving ECU 50 of the third embodiment shown in Fig. 14 is mounted on a POV (Personally Owned Vehicle) that is assumed to be owned by an individual, not on a vehicle that provides mobility services. The autonomous driving ECU 50 can autonomously drive the host vehicle Am, which is the POV, at least in a specific area (for example, a parking lot of a commercial facility, etc.).
[0146] The user terminal 170 of the owner OW who owns the vehicle Am is preregistered in the automatic driving ECU 50. The user terminal 170 transmits a command to call the vehicle Am (hereinafter, a call command) to the automatic driving ECU 50 together with the location information of the terminal. In response to a remote call from the user terminal 170 linked to the owner OW, the automatic driving ECU 50 moves the vehicle Am to the vicinity of the owner OW without the driver being on board. The user terminal 170 linked to the owner OW may be a general-purpose terminal such as a smartphone, or may be a dedicated terminal including a FoB key function.
[0147] When a notable person UM different from the owner OW approaches the host vehicle Am during a movement period in response to the call command, the automatic driving ECU 50 performs a notable person response process corresponding to the notable person UM. The notable person response process is started by the automatic driving ECU 50 based on the acquisition of the call command by the information linking unit 61. Hereinafter, the details of the notable person response process of the third embodiment will be described based on FIG. 15 with reference to FIG. 14 and FIG. 9.
[0148] In S311 of the watcher response process, the environment recognition unit 62 judges whether the host vehicle Am has stopped or not. If the host vehicle Am is moving (S311: NO), the environment recognition unit 62 repeats the judgment of S311 and waits for the host vehicle Am to stop. Then, if the host vehicle Am has stopped (S311: YES), the target recognition unit 73 performs vehicle exterior monitoring in S312 and judges whether the watcher UM is approaching.
[0149] If the noticing person UM is not approaching (S312: NO), the environment recognition unit 62 determines in S319 whether the host vehicle Am has moved to the vicinity of the owner OW, in other words, whether the summons have been completed. If the host vehicle Am has stopped near the owner OW and the summons have been completed (S319: YES), the noticing person response process ends. On the other hand, if the summons have not been completed (S319: NO), the determinations of S311 and S312 are repeated.
[0150] When the tormentor UM is approaching and the tormentor UM is blocking the travel of the vehicle Am moving to the position of the owner OW (S312: YES), the notification request unit 72 notifies the user terminal 170 of the approach of the tormentor UM in S313. In addition, the video distribution unit 66 transmits to the user terminal 170 in S314 the outside camera video captured by the camera unit of the perimeter monitoring sensor 30, which includes the tormentor UM.
[0151] The owner OW checks the exterior camera video delivered to the user terminal 170 and determines whether or not to allow the alert person UM to board. If the owner OW allows the alert person UM to board, it inputs a permission operation to the user terminal 170. In S315, the information linking unit 61 determines whether or not the owner OW has obtained permission to board. If the information linking unit 61 has obtained permission for the alert person UM to board (S315: YES), the security function unit 67 allows the alert person UM to board. As a result, in S316, the equipment control unit 65 cooperates with the body ECU 43 and the door lock actuator 45 to unlock the doors of the host vehicle Am so that they can be opened from the outside.
[0152] On the other hand, if the information linking unit 61 does not obtain permission for the not to be warned UM to board (S315: NO), the security function unit 67 does not permit the not to be warned UM to board. In this case, the notification request unit 72 performs an exterior notification (boarding prohibition warning) indicating that boarding is prohibited, using the exterior display 24 and the exterior speaker 25, in S317. The exterior display 24 is provided, for example, on the front of the vehicle Am. In the exterior notification in S317, the spoken voice of the owner OW collected by the user terminal 170 may be reproduced by the exterior speaker 25.
[0153] The target recognition unit 73 performs vehicle exterior monitoring in S318 and determines whether the tormentor UM has disappeared. If the tormentor UM continues to obstruct the travel of the vehicle Am (S318: NO), the owner OW is awaited to decide whether to permit the tormentor UM to board the vehicle Am. In this case, the vehicle exterior notification may be continued. On the other hand, if the tormentor UM has disappeared (S318: YES), the completion of the summons is determined in S319.
[0154] (Summary of the third embodiment) In the third embodiment described above, the vehicle driver is notified of the approaching person UM from the vehicle Am, thereby preventing the approaching person UM from getting into the vehicle Am. As a result, the same effects as those of the first and second embodiments are achieved, and it becomes possible to respond to the approaching person UM even when the driver is not in the vehicle.
[0155] Additionally, in the third embodiment, a command to summon the host vehicle Am is acquired from the user terminal 170 linked to the owner OW of the host vehicle Am. Then, when the travel of the host vehicle Am moving to the position of the owner OW is hindered by the tormentor UM, the notification request unit 72 notifies the user terminal 170. As described above, even if the host vehicle Am is a privately owned vehicle, a response to the tormentor UM approaching the host vehicle Am can be appropriately implemented in a situation where the owner OW is not aboard the vehicle.
[0156] In the third embodiment, when permission to board the vehicle is obtained for the person to beware UM, the door of the vehicle Am is unlocked so that it can be opened from the outside. Therefore, even if a person related to the owner OW approaches the vehicle Am before the owner OW, the person can board the vehicle Am. As a result, the convenience of the vehicle Am can be improved. On the other hand, when permission to board the vehicle is not obtained for the person to beware UM, an outside alarm is issued to indicate that boarding is not possible. As a result, it is possible to encourage the person to beware UM to take appropriate action. In the third embodiment, the information linking unit 61 corresponds to the "command acquisition unit."
[0157] (Other embodiments) Although several embodiments of the present disclosure have been described above, the present disclosure should not be construed as being limited to the above-described embodiments, and can be applied to various embodiments and combinations within the scope not departing from the gist of the present disclosure.
[0158] The autonomous driving ECU 50 according to the first modification of the above embodiment is mounted on a POV that is assumed to be privately owned, not on a vehicle that provides mobility services, as in the third embodiment. The autonomous driving ECU 50 is capable of autonomously driving the vehicle Am, which is the POV, at level 4 or higher autonomous driving. As in the first modification, the vehicle mounted with the autonomous driving ECU 50 is not limited to a vehicle for mobility services, but may be a general private passenger car, a rental car vehicle, a manned taxi vehicle, a freight vehicle, a bus, and the like. Furthermore, the function of remote driving in cooperation with the remote side system 130 may be omitted as appropriate.
[0159] In the autonomous driving ECU 50 of the first modification, when the vehicle Am is stopped at a location other than a pre-registered location such as a home and a pre-registered user is in the vehicle cabin, the target object recognition unit 73 determines that any person approaching the vehicle Am is a person to be warned about UM. In addition, when a person without a FoB key approaches the vehicle Am, the target object recognition unit 73 may determine that a person to be warned about UM has approached.
[0160] In the second modification of the above embodiment, the no-boarding warning indicating that boarding is not permitted is issued using only one of the exterior display 24 and the exterior speaker 25. Furthermore, the exterior display 24 may issue a no-boarding warning indicating that boarding is not permitted, for example, by flashing a warning color instead of displaying a text message.
[0161] In the third modification of the above embodiment, the in-vehicle display 21 has a function as a touch panel. At least some of the emergency stop button 117, the emergency call button 118, and the emergency evacuation button 119 are made into touch icons displayed on the in-vehicle display 21 instead of physical switches. The autonomous driving ECU 50 and the HMI control device 10 cause each of the buttons 117 to 119 to be displayed on the in-vehicle display 21 at a timing when an operation is required.
[0162] The content of the evacuation control in response to a warning person in the first embodiment may be changed as appropriate. As an example, even in normal evacuation control, the behavior determination unit 63 may continue moving the host vehicle Am until a stop command is obtained from at least one of the passenger PS and the remote monitor RO. In addition, in emergency evacuation control, if a passenger PS who can drive is present in the driver's seat DS, driving inside the vehicle may be permitted. Furthermore, as long as at least an outside vehicle alarm is performed in response to a warning person, at least one of the unlocking control, the center notification, and the evacuation control may be omitted as appropriate. In addition, the function of monitoring the inside of the vehicle and the function of alarming theft may also be omitted as appropriate.
[0163] The vehicle equipped with the autonomous driving ECU 50 may be a right-hand drive vehicle or a left-hand drive vehicle. Furthermore, the traffic environment in which the vehicle travels may be a traffic environment based on left-hand traffic or a traffic environment based on right-hand traffic. The positions of the doors provided in the driverless vehicle ADV, as well as the positions of the exterior display 24 and the exterior speaker 25, may be optimized as appropriate according to the road traffic laws of the countries and regions in which the driverless vehicle ADV is operated, and further according to the position of the steering wheel of the vehicle, etc.
[0164] The form of the in-vehicle operation unit 110 provided in the host vehicle Am is not limited to an accelerator pedal, a brake pedal, a steering wheel, etc. For example, a lever-shaped controller, a joystick, a controller (game pad) including a cross key, etc. may be provided as the in-vehicle operation unit 110.
[0165] In the above embodiment, each process related to monitoring inside and outside the vehicle (see Figures 5 to 8, 10 to 13, and 15) was performed by the autonomous driving ECU 50 mounted on the driverless vehicle ADV. On the other hand, in Variation 4 of the above embodiment, some or all of the processes are performed by the remote control device 140. As described above, at least some of the functions of the autonomous driving ECU 50 may be implemented in the remote control device 140. In such Variation 4, the remote control device 140 corresponds to the "autonomous driving control device" together with the autonomous driving ECU 50 or alone.
[0166] Furthermore, the functions of the vehicle monitoring device may be implemented in an on-board ECU other than the autonomous driving ECU 50, such as the HMI control device 10. Furthermore, the functions of the vehicle monitoring device may be implemented in a distributed state in multiple on-board ECUs mounted on the host vehicle Am, such as a central ECU and a zone ECU. Also, a single on-board ECU having the functions of the vehicle monitoring device may be mounted on the driverless vehicle ADV. In these modified examples, the on-board ECU, either alone or in multiple units, corresponds to the "vehicle monitoring device."
[0167] In the above embodiment, each function provided by the autonomous driving ECU, etc. can be provided by software and hardware that executes the software, software only, hardware only, or a combination of these. Furthermore, when such functions are provided by electronic circuits as hardware, each function can also be provided by digital circuits including a large number of logic circuits, or analog circuits. Furthermore, the software for realizing such functions may include at least a part of code automatically generated by a neural network or language model trained using, for example, camera images of the real world.
[0168] Each processing unit in the above-described embodiment includes at least one arithmetic core such as a central processing unit (CPU) and a graphics processing unit (GPU). The processing unit may further include a field-programmable gate array (FPGA), a neural network processing unit (NPU), and an IP core having other dedicated functions. The processing unit may be individually mounted on a printed circuit board, or may be mounted on an application specific integrated circuit (ASIC), a system on chip (SoC), and an FPGA.
[0169] The form of the storage medium (non-transitory tangible storage medium) that stores various programs and the like may be changed as appropriate. Furthermore, the storage medium is not limited to a configuration in which it is provided on a circuit board, and may be provided in the form of a memory card or the like, inserted into a slot portion, and electrically connected to a control circuit of the autonomous driving ECU or remote control device. Furthermore, the storage medium may be an optical disk, hard disk drive, solid state drive, or the like that is the source from which the program is copied or distributed to the autonomous driving ECU or remote control device.
[0170] The control unit and the method described herein may be implemented by a special-purpose computer having a processor programmed to execute one or more functions embodied in a computer program. Alternatively, the device and the method described herein may be implemented by a special-purpose hardware logic circuit. Alternatively, the device and the method described herein may be implemented by one or more special-purpose computers configured by a combination of a processor that executes a computer program and one or more hardware logic circuits. The computer program may also be stored in a computer-readable non-transitory tangible recording medium as instructions to be executed by the computer.
[0171] (Disclosure of technical ideas) This specification discloses multiple technical ideas described in the following multiple dependent claims. Some of the claims may be described in a multiple dependent form, where the subsequent claim alternatively refers to the preceding claim. Furthermore, some of the claims may be described in a multiple dependent form, where the subsequent claim alternatively refers to the preceding claim. The claims described in these multiple dependent forms define multiple technical ideas.
[0172] (Technical thought 1) A vehicle monitoring device that monitors the status of a vehicle (Am) that can be driven by an automatic driving function, an approach determination unit (73) that determines an approach of a person to be warned (UM) who is not scheduled to board the vehicle to the vehicle; a notification execution unit (72) that, when it is determined that the warning person is approaching the vehicle, issues an exterior notification indicating that boarding is not permitted, using an exterior alarm device (24, 25) provided in the vehicle; A vehicle monitoring device comprising: (Technical thought 2) The vehicle monitoring device described in Technical Idea 1 further includes a behavior control unit (63) that, when the person to be warned approaches the vehicle while it is stopped, determines whether the vehicle can move from its current position while it is stopped, and if it determines that the vehicle can move, moves the vehicle from its current position. (Technical Thought 3) The behavior control unit of the vehicle monitoring device according to Technical Idea 2 is configured to, when an emergency evacuation button (119) provided in the vehicle cabin is operated by a passenger (PS), move the vehicle to a location farther than the current location in a case where the emergency evacuation button is not operated. (Technical Thought 4) The approach determination unit determines a possibility of contact between the warning person and the host vehicle, The vehicle monitoring device according to Technical Idea 2 or 3, wherein the behavior control unit suppresses the moving speed when there is a possibility of contact with the alert person, more than when there is no possibility of contact. (Technical Thought 5) The vehicle monitoring device according to any one of technical ideas 2 to 4, further comprising a lock control unit (65) that locks the door of the host vehicle so that it cannot be opened from the outside when the behavior control unit determines that the host vehicle is not movable. (Technical Thought 6) An application acquisition unit (70) for acquiring an application for boarding the vehicle, The vehicle monitoring device described in any one of technical ideas 2 to 5, wherein the behavior control unit stops the movement of the vehicle when the boarding application for the person to be warned is acquired by the application acquisition unit after starting movement from the current location. (Technical Thought 7) The vehicle monitoring device described in any one of Technical Ideas 2 to 6, wherein the behavior control unit continues moving the vehicle until it receives a stop command from at least one of a passenger (PS) in the vehicle cabin and a monitor (RO) remotely monitoring the vehicle. (Technical Thought 8) The notification execution unit, When the state in which it has been determined that the person in question is approaching continues for a predetermined time or more, an in-vehicle alarm (21, 22) provided in the vehicle cabin is used to inquire whether or not to report the approach of the person in question to a predetermined notification destination (CNT); The vehicle monitoring device according to any one of Technical Ideas 1 to 7, wherein the vehicle monitor performs a report to the report destination when the report is approved by a passenger (PS) in the vehicle cabin. (Technical Thought 9) The notification execution unit, When it is determined that the person to be warned approaches the vehicle at a location different from a preset boarding / alighting point, an interior alarm (21, 22) provided in the vehicle cabin is used to inquire whether or not to perform personal authentication of the person to be warned; The vehicle monitoring device according to any one of Technical Ideas 1 to 8, wherein the vehicle monitoring device performs the personal authentication of the warning person when the performance of the personal authentication is permitted by a passenger (PS) in the vehicle compartment. (Technical Thought 10) The vehicle monitoring device described in any one of technical ideas 1 to 9 further includes a video distribution unit (66) that distributes in-vehicle camera video taken by an interior camera (29) installed in the vehicle cabin to a user terminal (170) linked to an approver whose reservation for boarding the vehicle has been accepted, based on a request from the user terminal. (Technical Thought 11) The video distribution unit of the vehicle monitoring device described in technical idea 10 allows the distribution of the in-vehicle camera video to the user terminal when the vehicle approaches within a predetermined distance of the approved person's intended boarding point. (Technical Thought 12) The vehicle monitoring device according to technical idea 10 or 11, wherein the video distribution unit permits the distribution of the in-vehicle camera video to the user terminal when a predetermined time has passed until the approved person's scheduled boarding time. (Technical Thought 13) The vehicle monitoring device according to any one of Technical Ideas 10 to 12, wherein the video distribution unit continues to permit distribution of the in-vehicle camera video to the user terminal until a predetermined time has elapsed after the approved person has exited the vehicle. (Technical Thought 14) The vehicle further includes an in-vehicle monitoring unit (71) that detects movement of a passenger (PS) to a driver's seat (DS) that is closest to a driving operation unit (110) provided in the vehicle, The vehicle monitoring device described in any one of technical ideas 1 to 13, wherein the notification implementation unit, when it is determined that the occupant has moved to the driver's seat, uses an in-vehicle alarm (21, 22) provided in the vehicle cabin to implement an in-vehicle warning to encourage the occupant to leave the driver's seat. (Technical Thought 15) The vehicle monitoring device according to technical idea 14, wherein the notification implementation unit, after implementing the in-vehicle warning, notifies a monitor (RO) remotely monitoring the vehicle that the occupant has moved to the driver's seat. (Technical Thought 16) The vehicle further includes an interior monitoring unit (71) that detects whether a passenger (PS) has touched a drive key located in a vehicle interior of the vehicle, The vehicle monitoring device described in any one of technical ideas 1 to 15, wherein when it is determined that the occupant has touched the drive key, the alarm implementation unit implements an in-vehicle warning using an in-vehicle alarm (21, 22) provided in the vehicle cabin to warn the occupant not to touch the drive key. (Technical Thought 17) The vehicle monitoring device described in any one of technical ideas 1 to 16 further includes a locking control unit (65) that, when it is determined that the warning person is approaching the vehicle, unlocks, of the doors on both the left and right sides of the vehicle, the door opposite the side from which the warning person is approaching so that it can be opened from inside the vehicle cabin. (Technical Thought 18) The vehicle monitoring device according to technical idea 17, wherein the notification implementation unit uses an interior alarm (21, 22) provided in the vehicle cabin to implement an interior notification indicating that the door on the opposite side is unlocked after the door on the opposite side is unlocked. (Technical Thought 19) a position storage unit (78) that stores a stopping location of the host vehicle based on the function-off process of the host vehicle; The vehicle monitoring device described in any one of technical ideas 1 to 18, wherein the notification implementation unit implements an outside vehicle warning indicating a theft when the start location of the vehicle, which is determined based on the function on processing of the vehicle, is different from the stopped location. (Technical Thought 20) The vehicle monitoring device described in any one of technical ideas 1 to 19, wherein the notification implementation unit notifies a predetermined notification destination (CNT) of the location information of a user terminal (170) linked to a passenger (PS) inside the vehicle cabin in the event of theft of the vehicle. (Technical Thought 21) The vehicle control system further includes a command acquisition unit (61) that acquires a command to call the vehicle from a user terminal (170) associated with an owner (OW) of the vehicle, The vehicle monitoring device according to any one of technical ideas 1 to 20, wherein the notification implementation unit notifies the user terminal when the vehicle's travel to the owner's location is obstructed by the cautionary person. (Technical Thought 22) a lock control unit (65) that unlocks a door of the vehicle so that the door can be opened from the outside when the command acquisition unit acquires permission for the warning person to get on the vehicle, The vehicle monitoring device described in technical idea 21, wherein the notification implementation unit implements the vehicle exterior notification indicating that boarding is not permitted using the vehicle exterior alarm when the boarding permission is not acquired by the command acquisition unit. (Technical Thought 23) A vehicle monitoring method for monitoring the status of a vehicle (Am) that can be driven by an automatic driving function, comprising: A check is made to see if a person (UM) who is not scheduled to board the vehicle approaches the vehicle (S13). When it is determined that the warning person is approaching the vehicle, an exterior alarm (24, 25) provided in the vehicle is used to issue an exterior alarm indicating that the person is not allowed to board the vehicle (S14). The vehicle monitoring method includes the steps of: [Explanation of symbols]
[0173] Am: vehicle itself, CNT: remote monitoring facility (report destination), DS: driver's seat, OW: owner, PS: passenger, RO: remote monitor (monitor), UM: alert person, 21: in-vehicle display (in-vehicle alarm), 22: in-vehicle speaker (in-vehicle alarm), 24: outside-vehicle display (out-vehicle alarm), 25: outside-vehicle speaker (out-vehicle alarm), 29: in-vehicle monitor device (in-vehicle camera), 50: autonomous driving ECU (vehicle monitoring device), 51: processing unit, 61: information linking unit (command acquisition unit), 63: action determination unit (action control unit), 65: device control unit (lock control unit), 66: video distribution unit, 70: user authentication unit (application acquisition unit), 71: situation recognition unit (in-vehicle monitoring unit), 72: notification request unit (notification implementation unit), 73: target recognition unit (approach determination unit), 78: position memory unit, 110 In-vehicle operation unit (driving operation unit), 119 emergency evacuation button, 170 user terminal
Claims
1. A vehicle monitoring device that monitors the status of a vehicle (Am) that can be driven by an automatic driving function, An approach determination unit (73) that determines an approach of a person to be warned (UM) who is not scheduled to board the vehicle to the vehicle; a notification implementation unit (72) that, when it is determined that the warning person is approaching the vehicle, issues an external notification indicating that boarding is not permitted, using an external alarm device (24, 25) provided in the vehicle; A vehicle monitoring device comprising:
2. 2. The vehicle monitoring device according to claim 1, further comprising an action control unit (63) that, when the person to be warned approaches the vehicle while stopped, determines whether the vehicle can move from its current position while stopped, and, if it is determined that the vehicle can move, moves the vehicle from the current position.
3. 3. The vehicle monitoring device according to claim 2, wherein when an emergency evacuation button (119) provided in the vehicle cabin is operated by a passenger (PS), the behavior control unit moves the vehicle to a location farther than the current location when the emergency evacuation button is not operated.
4. The approach determination unit determines a possibility of contact between the warning person and the host vehicle, The vehicle monitor device according to claim 2 , wherein the behavior control unit is configured to suppress a moving speed when there is a possibility of contact with the person to be warned, compared to a case where there is no possibility of contact.
5. 3. The vehicle monitoring device according to claim 2, further comprising a lock control unit (65) that locks a door of the host vehicle so that it cannot be opened from the outside when the behavior control unit determines that the host vehicle is not movable.
6. An application acquisition unit (70) for acquiring an application for boarding the vehicle, The vehicle monitoring device according to claim 2 , wherein the behavior control unit stops the movement of the vehicle when, after starting movement from the current position, the boarding application for the person to be warned is acquired by the application acquisition unit.
7. The vehicle monitoring device according to claim 2 , wherein the behavior control unit continues moving the vehicle until it receives a stop command from at least one of a passenger (PS) in the vehicle cabin and a monitor (RO) remotely monitoring the vehicle.
8. The notification execution unit, When the state in which it has been determined that the person in question is approaching continues for a predetermined time or more, an in-vehicle alarm (21, 22) provided in the vehicle cabin is used to inquire whether or not to report the approach of the person in question to a predetermined notification destination (CNT); The vehicle monitoring device according to claim 1 , wherein when the execution of the report is approved by a passenger (PS) in the vehicle cabin, the vehicle monitor executes the report to the report destination.
9. The notification execution unit, When it is determined that the person to be warned approaches the vehicle at a location different from a preset boarding / alighting point, an in-vehicle alarm (21, 22) provided in the vehicle cabin is used to inquire whether or not to perform personal authentication of the person to be warned; The vehicle monitoring device according to claim 1 , wherein the personal authentication of the warning person is performed when the execution of the personal authentication is permitted by a passenger (PS) in the vehicle compartment.
10. The vehicle monitoring device of claim 1, further comprising a video distribution unit (66) that distributes in-vehicle camera video captured by an interior camera (29) installed in the vehicle cabin to a user terminal (170) linked to an approver whose reservation for boarding the vehicle has been accepted, based on a request from the user terminal.
11. The vehicle monitoring device according to claim 10, wherein the video distribution unit allows the distribution of the in-vehicle camera video to the user terminal when the vehicle approaches within a predetermined distance of the intended boarding point of the approver.
12. The vehicle monitoring device according to claim 10 , wherein the video distribution unit permits the distribution of the in-vehicle camera video to the user terminal when a predetermined time period has elapsed until the scheduled boarding time of the approved person.
13. The vehicle monitoring device according to claim 10 , wherein the video distribution unit continues to permit distribution of the in-vehicle camera video to the user terminal until a predetermined time has elapsed after the approved person has exited the vehicle.
14. The vehicle further includes an in-vehicle monitoring unit (71) that detects movement of a passenger (PS) to a driver's seat (DS) that is closest to a driving operation unit (110) provided in the vehicle itself, The vehicle monitoring device according to claim 1, wherein when it is determined that the occupant has moved to the driver's seat, the notification implementation unit uses an interior alarm (21, 22) installed in the vehicle cabin to implement an interior warning to encourage the occupant to leave the driver's seat.
15. The vehicle monitoring device according to claim 14 , wherein the notification unit notifies a remote operator (RO) who remotely monitors the vehicle of the movement of the passenger to the driver's seat after issuing the in-vehicle warning.
16. A vehicle interior monitoring unit (71) that detects that a passenger (PS) has touched a drive key located in a vehicle interior of the vehicle, The vehicle monitoring device according to claim 1, wherein when it is determined that the occupant has touched the drive key, the notification implementation unit implements an in-vehicle warning using an in-vehicle alarm (21, 22) provided in the vehicle cabin to warn the occupant not to touch the drive key.
17. The vehicle monitoring device as described in claim 1, further comprising a lock control unit (65) that, when it is determined that the warning person is approaching the vehicle, unlocks, of the doors on both the left and right sides of the vehicle, the door opposite the side from which the warning person is approaching so that it can be opened from inside the vehicle interior.
18. The vehicle monitoring device according to claim 17, wherein the notification implementation unit, after unlocking the opposite door, uses an interior alarm (21, 22) provided in the vehicle compartment to implement an interior notification indicating that the opposite door is unlocked.
19. A location storage unit (78) that stores a location where the vehicle is stopped based on the function-off process of the vehicle, The vehicle monitoring device according to claim 1 , wherein the notification unit issues an outside vehicle warning indicating that a theft has occurred when a start location of the vehicle identified based on a function-on process of the vehicle is different from the stop location.
20. The vehicle monitoring device of claim 1, wherein the notification unit notifies a predetermined notification destination (CNT) of location information of a user terminal (170) linked to a passenger (PS) inside the vehicle cabin when the vehicle is stolen.
21. A command acquisition unit (61) that acquires a command to call the vehicle from a user terminal (170) associated with an owner (OW) of the vehicle, The vehicle monitoring device according to claim 1 , wherein the notification unit notifies the user terminal when the vehicle moving to the owner's location is blocked by the cautioner.
22. a lock control unit (65) that unlocks a door of the vehicle so that the door can be opened from the outside when the command acquisition unit acquires a boarding permission for the warning person, The vehicle monitor device according to claim 21 , wherein the notification implementation unit implements the vehicle exterior notification indicating that boarding is not permitted using the vehicle exterior alarm when the boarding permission is not acquired by the command acquisition unit.
23. A vehicle monitoring program for monitoring a status of a vehicle (Am) that can be driven by an automatic driving function, A warning person (UM) who is not scheduled to board the vehicle is determined to be approaching the vehicle (S13, S213, S312). When it is determined that the warning person is approaching the vehicle, an exterior alarm (24, 25) provided in the vehicle is used to issue an exterior alarm indicating that boarding is not permitted (S14, S219, S317). A vehicle monitoring program that causes at least one processing unit (51) to execute a process including the steps of:
Citation Information
Patent Citations
Automatic operation device, automatic operation method, and program
JP2022154979A
