Autonomous Driving Assistance Systems

Abstract

To provide an automatic driving auxiliary system for quickly performing assistance for an automatic driving vehicle in a prioritized state.SOLUTION: An automatic driving auxiliary system comprises a remote monitoring device 20 and an on-vehicle device 10a. The on-vehicle device 10a comprises a vehicle controller 12a that outputs auxiliary requirement when not being able to autonomously travel. The remote monitoring device 20 comprises: an automatic driving vehicle information display 22 for displaying peripheral information from the on-vehicle device 10a; an automatic driving vehicle selector 23 for selecting a selection vehicle from among automatic driving vehicles 100a that transmit the auxiliary requirement; an operation order input unit 24 for receiving an operation order to the selection vehicle; and a remote monitoring transmitter 21 for transmitting stand-by information to the on-vehicle device 10a when the operation order is a stand-by order, transmitting manpower-dispatch requesting information to an operator terminal 30 when the operation order is an on-site operation request, and transmitting remote operation information input into a remote operation input unit 25 to the on-vehicle device 10a in the selection vehicle when the operation order is a remote operation order.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present application relates to an automated driving assistance system. [Background technology]

[0002] To implement autonomous driving in society, a system that provides manual assistance in the event of a breakdown or unexpected event is necessary in preparation for situations that are difficult or inefficient to handle using the autonomous driving system alone. However, increasing the number of assistant personnel assisting the autonomous driving system would prevent the achievement of the goal of autonomous driving—labor reduction—and therefore requires a small number of personnel to assist a large number of autonomous vehicles. In this case, there is a risk that vehicles in more urgent situations will not be properly assisted if requests for assistance are concentrated in a short period of time. In situations that cannot be handled by the autonomous driving system alone, a control system has been proposed in which a remote monitor in a remote location remotely instructs the vehicle's driving operations. This system determines priorities based on the vehicle's situation and assigns remote control tasks to the remote monitor (see, for example, Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent No. 7014018 Summary of the Invention [Problem to be solved by the invention]

[0004] The control device shown in Patent Document 1 has the problem that the remote monitor can grasp the situation of the self-driving vehicle after the remote control work has been assigned to the remote monitor, which may result in delays in providing assistance to the self-driving vehicle when it should be given priority.

[0005] The present application has been made to solve the above-mentioned problems, and aims to provide an automatic driving assistance system that quickly provides assistance to an automatic driving vehicle in a situation where priority should be given. [Means for solving the problem]

[0006] The autonomous driving assistance system disclosed in the present application is an autonomous driving assistance system comprising a remote monitoring device and an on-board device provided in each of a plurality of autonomous vehicles, wherein the on-board device comprises a vehicle communication device that communicates with the remote monitoring device, and a vehicle controller that controls the driving of the autonomous vehicle based on peripheral information of the autonomous vehicle acquired from peripheral information sensors and vehicle information of the autonomous vehicle acquired from vehicle information sensors, and that transmits an assistance request signal to the remote monitoring device via the vehicle communication device when autonomous driving is not possible, and the remote monitoring device comprises a remote monitoring communication device that communicates with the outside of the remote monitoring device, an autonomous vehicle information display that displays peripheral information acquired from the on-board device that sent the assistance request when an assistance request is received from the on-board device, and an autonomous driving assistance system comprising the on-board device that sent the assistance request. The system includes an autonomous vehicle selector that accepts an instruction to select one of the operating vehicles as the selected vehicle, a work instruction input device that accepts work instructions for the selected vehicle, and a remote operation input device that accepts input of remote operation information for the selected vehicle, and the remote monitoring communication device transmits standby information instructing the vehicle to wait to an on-board device of the selected vehicle when the work instruction is a standby instruction, transmits information requesting the dispatch of personnel to the selected vehicle when the work instruction is a request for on-site work to a worker terminal, and transmits the remote operation information input to the remote operation input device to the on-board device of the selected vehicle when the work instruction is a remote operation instruction, and the vehicle controller performs driving control to maintain the autonomous vehicle in a stopped state when standby information is sent from the remote monitoring device, and performs driving control of the autonomous vehicle based on the remote operation information when remote operation information is sent from the remote monitoring device. The standby information includes information about a standby time for maintaining the standby state, and the vehicle controller, upon receiving the standby information from the remote monitoring device, resumes the automatic driving if autonomous driving by automatic driving becomes possible between the time the standby state is entered and the time the standby time has elapsed, and transmits a signal requesting assistance to the remote monitoring device through the vehicle communication device if autonomous driving by automatic driving becomes impossible after the time the standby state is entered and the time the standby time has elapsed. . [Effects of the Invention]

[0007] The autonomous driving assistance system disclosed in the present application is an autonomous driving assistance system comprising a remote monitoring device and an on-board device provided in each of a plurality of autonomous vehicles, wherein the on-board device comprises a vehicle communication device that communicates with the remote monitoring device, and a vehicle controller that controls the driving of the autonomous vehicle based on peripheral information of the autonomous vehicle acquired from peripheral information sensors and vehicle information of the autonomous vehicle acquired from vehicle information sensors, and that transmits an assistance request signal to the remote monitoring device via the vehicle communication device when autonomous driving is not possible, and the remote monitoring device comprises a remote monitoring communication device that communicates with the outside of the remote monitoring device, an autonomous vehicle information display that displays peripheral information acquired from the on-board device that sent the assistance request when an assistance request is received from the on-board device, and an autonomous driving assistance system comprising the on-board device that sent the assistance request. The system includes an autonomous vehicle selector that accepts an instruction to select one of the operating vehicles as the selected vehicle, a work instruction input device that accepts work instructions for the selected vehicle, and a remote operation input device that accepts input of remote operation information for the selected vehicle, and the remote monitoring communication device transmits standby information instructing the vehicle to wait to an on-board device of the selected vehicle when the work instruction is a standby instruction, transmits information requesting the dispatch of personnel to the selected vehicle when the work instruction is a request for on-site work to a worker terminal, and transmits the remote operation information input to the remote operation input device to the on-board device of the selected vehicle when the work instruction is a remote operation instruction, and the vehicle controller performs driving control to maintain the autonomous vehicle in a stopped state when standby information is sent from the remote monitoring device, and performs driving control of the autonomous vehicle based on the remote operation information when remote operation information is sent from the remote monitoring device. The standby information includes information about a standby time, which is a time for maintaining a standby state, and when the standby information is sent from the remote monitoring device, the vehicle controller resumes autonomous driving if autonomous driving by automatic driving becomes possible between the time the standby state is entered and the time the standby time has elapsed, and transmits a signal requesting assistance to the remote monitoring device through the vehicle communication device if autonomous driving by automatic driving becomes impossible after the time the standby state is entered and the time the standby time has elapsed. This allows for quick assistance to be provided to self-driving vehicles in situations where priority is required. [Brief explanation of the drawings]

[0008] [Figure 1] 1 is a block diagram showing a configuration of an automatic driving assistance system according to a first embodiment. [Figure 2] FIG. 2 is a block diagram showing the configuration of a modified example of the automatic driving assistance system according to the first embodiment. [Figure 3] 4 is a flowchart illustrating the operation of the remote monitoring device according to the first embodiment. [Figure 4] 5 is a flowchart for explaining the operation of the in-vehicle device when a standby instruction is received in the first embodiment. [Figure 5] FIG. 10 is a block diagram showing the configuration of an automatic driving assistance system according to a second embodiment. [Figure 6] 10 is a flowchart illustrating the operation of the remote monitoring device in the second embodiment. [Figure 7] 11 is a flowchart illustrating the operation of the remote monitoring device according to the third embodiment. [Figure 8] 11 is a flowchart illustrating the operation of the remote monitoring device according to the third embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0009] Hereinafter, an automatic driving assistance system according to an embodiment of the present invention will be described in detail with reference to the drawings. Note that the same reference numerals in the drawings indicate the same or corresponding parts.

[0010] Embodiment 1 FIG. 1 is a block diagram showing the configuration of an autonomous driving assistance system according to a first embodiment. The autonomous driving assistance system according to the first embodiment includes on-board devices 10a and 10b provided in multiple autonomous vehicles 100a and 100b, respectively, a remote monitoring device 20, and an operator terminal 30. The autonomous vehicle 100a includes a vehicle information sensor 1a, a surrounding information sensor 2a, and the on-board device 10a, and the on-board device 10a includes a vehicle communication device 11a and a vehicle controller 12a. The autonomous vehicle 100b includes a vehicle information sensor 1b, a surrounding information sensor 2b, and the on-board device 10b, and the on-board device 10b includes a vehicle communication device 11b and a vehicle controller 12b. Although the example shown in FIG. 1 shows two autonomous vehicles each equipped with an on-board device, the number of autonomous vehicles each equipped with an on-board device may be any number. The remote monitoring device 20 includes a remote monitoring communication device 21, an autonomous vehicle information display device 22, an autonomous vehicle selector 23, a work instruction input device 24, and a remote operation input device 25. The worker terminal 30 includes a terminal communication device 31 and a dispatch request information output device 32.

[0011] The vehicle controllers 12a and 12b are realized by a processor such as a CPU (Central Processing Unit) that executes a program stored in a memory. The memory is also used as a temporary storage device for each process executed by the processor. The memory is, for example, a non-volatile or volatile semiconductor memory such as a RAM, a ROM, a flash memory, or an EPROM, a magnetic disk, an optical disk, or a combination thereof. The vehicle controllers 12a and 12b may be realized by a processing circuit. When the processing circuit is dedicated hardware, it may be, for example, a single circuit, a composite circuit, a programmed processor, an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or a combination thereof.

[0012] The operation of the on-vehicle device 10a provided in the autonomous vehicle 100a will be described below, but the operation of on-vehicle devices provided in other autonomous vehicles is the same. The vehicle information sensor 1a is a sensor that acquires vehicle information about the autonomous vehicle 100a and outputs vehicle information including, for example, the current position, traveling direction, and traveling speed of the autonomous vehicle 100a. The vehicle information sensor 1a receives signals from artificial satellites such as the Global Navigation Satellite System (GNSS) to detect the current position of the autonomous vehicle 100a. The vehicle information sensor 1a may acquire information about the autonomous vehicle 100a, such as the steering angle, power plant output, and braking force, and output it as vehicle information.

[0013] The surrounding information sensor 2a is a sensor that acquires information about the surroundings of the autonomous vehicle, and is, for example, a camera that captures images of the surroundings of the autonomous vehicle 100a and outputs images of the surroundings of the autonomous vehicle 100a as surrounding information. The surrounding information sensor 2a may, for example, include a radar that monitors the surroundings of the autonomous vehicle 100a, and may output surrounding information that includes radar output.

[0014] Vehicle controller 12a acquires vehicle information from vehicle information sensor 1a and peripheral information from peripheral information sensor 2a, and controls the driving of autonomous vehicle 100a based on the vehicle information and peripheral information, for example, controlling the power plant, braking device, and steering device of autonomous vehicle 100a to enable autonomous driving of the autonomous vehicle. Vehicle controller 12a outputs a signal requesting assistance when autonomous driving is not possible. Vehicle controller 12a outputs a signal requesting assistance when, for example, the path is blocked by an obstacle or another vehicle and autonomous driving is not possible.

[0015] The vehicle communication device 11a communicates with the remote monitoring communication device 21 of the remote monitoring device 20, and communicates wirelessly with the remote monitoring communication device 21 using a wireless communication standard such as 4G or 5G. When an assistance request is output from the vehicle controller 12a, the vehicle communication device 11a transmits the assistance request to the remote monitoring communication device 21, and transmits vehicle information acquired from the vehicle information sensor 1a and peripheral information acquired from the peripheral information sensor 2a to the remote monitoring communication device 21. When an assistance request is output from the vehicle controller 12a, the vehicle communication device 11a may acquire, from the vehicle controller 12a, a log of the autonomous driving processing or a system message associated with the autonomous driving processing, and transmit this information, which indicates the reason why the autonomous vehicle has come to require assistance or the current situation, together with the vehicle information.

[0016] Next, the operation of the remote monitoring device 20 will be described. The remote monitoring communication device 21 communicates with the outside of the remote monitoring device 20, and communicates with the vehicle communication devices 11a and 11b of the on-board devices 10a and 10b, respectively, and the terminal communication device 31 of the worker terminal 30. The remote monitoring communication device 21 receives an assistance request, vehicle information, and surrounding information when they are sent from the vehicle communication device 11a or the vehicle communication device 11b. The autonomously driven vehicle information display 22 displays the surrounding information acquired from the on-board device that sent the assistance request. For example, when an assistance request is sent from the on-board device 10a, the autonomously driven vehicle information display 22 displays the surrounding information acquired from the on-board device 10a. When the remote monitoring communication device 21 receives assistance requests from multiple on-board devices, the autonomously driven vehicle information display 22 displays the surrounding information acquired from each of the on-board devices that sent the assistance requests. Autonomous vehicle information display 22 may display surrounding information and vehicle information obtained from the in-vehicle device that sent the assistance request. By checking the information displayed on autonomous vehicle information display 22, the remote monitor can confirm which autonomous vehicle has sent the assistance request, and can confirm the situation around the autonomous vehicle that sent the assistance request and the situation of the vehicle itself.

[0017] The autonomous vehicle selector 23 accepts an instruction to select one of the autonomous vehicles that have issued an assistance request as the selected vehicle. When assistance requests are received from both the in-vehicle device 10a and the in-vehicle device 10b, the autonomous vehicle selector 23 accepts an instruction to select, for example, the autonomous vehicle 100a equipped with the in-vehicle device 10a as the selected vehicle. The remote monitor checks the information about the autonomous vehicles that have issued the assistance request displayed on the autonomous vehicle information display 22 and inputs to the autonomous vehicle selector 23 which autonomous vehicle that has issued the assistance request should be selected as the selected vehicle. The work instruction inputter 24 accepts a work instruction for the selected vehicle. The work instruction is either a standby instruction, a request for on-site work, or a remote operation instruction. The remote monitor inputs a standby instruction, a request for on-site work, or a remote operation instruction into the work instruction inputter 24 as a work instruction for the selected vehicle.

[0018] The remote monitoring communication device 21 acquires information about the selected vehicle from the autonomous vehicle selector 23 and acquires information about the work instruction from the work instruction input device 24. If the work instruction is a standby instruction, it transmits standby information instructing the in-vehicle device of the selected vehicle to wait. If the selected vehicle is the autonomous vehicle 100a equipped with the in-vehicle device 10a and the work instruction is a standby instruction, the remote monitoring communication device 21 transmits standby information instructing the vehicle to wait to the vehicle communication device 11a of the in-vehicle device 10a. If the work instruction is a standby instruction, the remote monitoring communication device 21 may transmit information about a standby time, which is the time for which the standby state is maintained, to the vehicle communication device 11a of the in-vehicle device 10a. Upon receiving the standby information, the vehicle communication device 11a outputs the standby information to the vehicle controller 12a. When the vehicle controller 12a acquires the standby information, it controls the power plant and the brake device to maintain the stopped state of the autonomous vehicle 100a. For example, the vehicle controller 12a may resume autonomous driving if autonomous driving becomes possible before the waiting time has elapsed since the vehicle entered the standby state. Alternatively, if autonomous driving becomes impossible after the waiting time has elapsed since the vehicle entered the standby state, the vehicle controller 12a may transmit a signal requesting assistance to the remote monitoring device 20 via the vehicle communication device 11a. A standby instruction is selected, for example, when an obstacle ahead of the autonomous vehicle has not moved for a certain period of time, preventing the autonomous vehicle from proceeding, and when the remote monitor determines that the obstacle ahead of the autonomous vehicle is a bus or taxi that is taking a long time to settle its fare. A standby instruction is selected as an operation instruction, for example, when it is expected that the bus or taxi will complete settlement and proceed after a short wait, allowing the selected vehicle to resume proceeding. For example, the vehicle controller 12a resumes autonomous driving if an obstacle ahead of the autonomous vehicle moves from the travel path before the waiting time has elapsed since the vehicle entered the standby state. Alternatively, the vehicle controller 12a transmits a signal requesting assistance to the remote monitoring device 20 via the vehicle communication device 11a when an obstacle ahead of the autonomous vehicle is still on the travel path after the waiting time has elapsed since the vehicle entered the standby state. The remote monitor can switch to assisting another autonomous vehicle by simply selecting a standby instruction as a work instruction, allowing the monitor to assist an autonomous vehicle that is in a situation where priority should be given to another vehicle.

[0019] When the work instruction is a request for on-site work, the remote monitoring communication device 21 transmits information requesting the dispatch of personnel to the selected vehicle to the terminal communication device 31 of the worker terminal 30. The information requesting the dispatch of personnel to the selected vehicle includes location information of the selected vehicle, and may include, for example, vehicle information and surrounding information of the selected vehicle. The dispatch request information output device 32 of the worker terminal 30 acquires and outputs the information requesting the dispatch of personnel to the selected vehicle from the terminal communication device 31. When the information requesting the dispatch of personnel to the selected vehicle is text information or image information, the dispatch request information output device 32 is, for example, a display. When the information requesting the dispatch of personnel to the selected vehicle is audio information, the dispatch request information output device 32 is, for example, a speaker. FIG. 2 is a block diagram showing the configuration of a modified example of the autonomous driving assistance system according to the first embodiment. The remote monitoring device 20 shown in FIG. 2 includes a dispatch request information input device 26 that receives input of information requesting the dispatch of personnel to the selected vehicle. When the information requesting the dispatch of personnel to the selected vehicle is text information, the dispatch request information input device 26 is, for example, a keyboard. When the information requesting the dispatch of personnel to the selected vehicle is audio information, the dispatch request information input device 26 is, for example, a microphone. For example, the dispatch request information input device 26 may be a microphone of a voice call device in the remote monitoring device 20 that makes a voice call with the worker terminal 30, and the dispatch request information output device 32 may be a speaker of a voice call device in the worker terminal 30 that makes a voice call with the remote monitoring device 20. After checking the information output from the dispatch request information output device 32 of the worker terminal 30, the worker can head to the location where the selected vehicle is parked and perform the work on site, or instruct another worker to head to the location where the selected vehicle is parked and perform the work. A request for on-site work is selected as a work instruction, for example, when there is a fallen object in front of the autonomous vehicle and there is no route to go around it. The remote monitor can then move on to assisting another autonomous vehicle by simply selecting an on-site work request as a work instruction, allowing the remote monitor to provide assistance to an autonomous vehicle in a situation that requires priority.

[0020] Next, a description will be given of the case where the work instruction is a remote operation instruction. The remote operation input device 25 accepts input of remote operation information for the selected vehicle and is, for example, an interface that acquires operation input from a remote monitor to remotely operate the autonomous vehicle. When the remote monitor selects a remote operation instruction as the work instruction, he or she operates the remote operation input device 25 to input the remote operation information while checking the information about the selected vehicle displayed on the autonomous vehicle information display 22. When the work instruction is a remote operation instruction, the remote monitoring communication device 21 acquires the remote operation information from the remote operation input device 25 and transmits the remote operation information input to the remote operation input device to the in-vehicle device of the selected vehicle. When the selected vehicle is an autonomous vehicle 100a equipped with the in-vehicle device 10a and the work instruction is a remote operation instruction, the remote monitoring communication device 21 transmits information about the remote operation instruction to the vehicle communication device 11a of the in-vehicle device 10a and transmits the remote operation information input to the remote operation input device 25. Upon receiving the remote operation instruction and the remote operation information, the vehicle communication device 11a outputs the remote operation information to the vehicle controller 12a. Vehicle controller 12a acquires remote operation information and controls the driving of autonomous vehicle 100a based on the acquired remote operation information, thereby realizing the driving of the autonomous vehicle by remote operation. A remote operation instruction is selected as a work instruction, for example, when there is an obstacle ahead of the autonomous vehicle but there is a route to bypass it. A remote operation instruction is selected as a work instruction, for example, when there is an emergency situation, such as when there is little time left until the selected vehicle is to arrive at the destination. When the work instruction is a remote operation instruction, it is expected that the selected vehicle will resume operation more quickly than when the work instruction is a standby instruction or a request for on-site work. By selecting a remote operation instruction as a work instruction, the remote monitor can prioritize responding to a selected vehicle that is in a situation that requires priority.

[0021] 3 is a flowchart for explaining the operation of remote monitoring device 20 in embodiment 1. Step S01, steps S05 to S09, and step S11 are remote monitoring communication steps. Step S01 is an assistance request confirmation step, step S05 is a standby instruction confirmation step, step S06 is a standby instruction transmission step, step S07 is an on-site work request confirmation step, step S08 is a personnel dispatch request transmission step, step S09 is a remote operation instruction confirmation step, and step S11 is a remote operation information transmission step. Step S02 is an autonomous vehicle information display step, step S03 is an autonomous vehicle selection step, step S04 is a work instruction input step, and step S10 is a remote operation input step.

[0022] In step S01, the remote monitoring communication device 21 checks whether an assistance request has been received from any of the vehicle communication devices. If an assistance request has been received, the process proceeds to step S02. If an assistance request has not been received, the process ends. In step S02, the autonomously driven vehicle information display 22 displays surrounding information acquired from each of the in-vehicle devices that transmitted the assistance requests, and the process proceeds to step S03. In step S03, the autonomously driven vehicle selector 23 accepts an instruction to select one of the autonomously driven vehicles that transmitted the assistance request and make it the selected vehicle, and the process proceeds to step S04. In step S04, the work instruction input device 24 accepts a work instruction for the selected vehicle, and the process proceeds to step S05. In step S05, the remote monitoring communication device 21 checks whether the work instruction is a standby instruction. If the work instruction is a standby instruction, the process proceeds to step S06. If the work instruction is not a standby instruction, the process proceeds to step S07. In step S06, the remote monitoring communication device 21 transmits standby information instructing the selected vehicle to wait, and the process ends.

[0023] In step S07, the remote monitoring communication device 21 checks whether the work instruction is a request for on-site work, and if the work instruction is a request for on-site work, the process proceeds to step S08. If the work instruction is not a request for on-site work, the process proceeds to step S09. In step S08, the remote monitoring communication device 21 transmits information to the terminal communication device 31 of the worker terminal 30 requesting that personnel be dispatched to the selected vehicle, and the process ends. In step S09, the remote monitoring communication device 21 checks whether the work instruction is a remote operation instruction, and if the work instruction is a remote operation instruction, the process proceeds to step S10. If the work instruction is not a remote operation instruction, the process ends. In step S10, the remote operation input device 25 accepts input of remote operation information for the selected vehicle, and the process proceeds to step S11. In step S11, the remote monitoring communication device 21 transmits the remote operation information to the vehicle communication device of the selected vehicle, and the process ends. The processes of steps S10 and S11 are continuously performed while the selected vehicle is being remotely operated.

[0024] 4 is a flowchart for explaining the operation of the on-board device when a standby instruction is received in embodiment 1. The following describes the operation when the selected vehicle is an autonomously driven vehicle 100a equipped with on-board device 10a. Steps S21 and S27 are vehicle communication steps, with step S21 being a standby instruction receiving step and step S27 being an assistance request sending step. Steps S22 to S26 are vehicle control steps, step S22 being a standby control step, step S23 being an autonomous driving confirmation step, step S24 being an autonomous driving resumption step, step S25 being a standby time confirmation step, and step S26 being an assistance request output step.

[0025] In step S21, the vehicle communication device 11a receives standby information and proceeds to step S22. The standby information includes information on a standby time, which is the time for which the standby state is maintained. In step S22, the vehicle controller 12a acquires the standby information, controls the power plant and the brake device to maintain the stopped state of the autonomously driven vehicle, and proceeds to step S23. In step S23, the vehicle controller 12a acquires surrounding information from the surrounding information sensor 2a and checks whether autonomous driving by autonomous driving is possible. If autonomous driving by autonomous driving is possible, the process proceeds to step S24, and if autonomous driving by autonomous driving is not possible, the process proceeds to step S25. In step S23, the vehicle controller 12a acquires surrounding information from, for example, the surrounding information sensor 2a and checks whether obstacles ahead of the autonomously driven vehicle have disappeared and the autonomous driving by autonomous driving is now possible. If autonomous driving by autonomous driving is possible, the process proceeds to step S24, and if autonomous driving by autonomous driving is not possible, the process proceeds to step S25.

[0026] In step S24, the vehicle controller 12a cancels the standby state, resumes autonomous driving, and ends the processing. When the autonomous vehicle is in the standby state and autonomous driving becomes possible, for example, because an obstacle in front of the autonomous vehicle has disappeared, autonomous driving can be resumed, and the processing related to the assistance request for the selected vehicle can be ended.

[0027] In step S25, vehicle controller 12a checks whether the waiting time has elapsed since entering the standby state, i.e., since acquiring the standby information and starting to maintain the stopped state of autonomously driven vehicle 100a. If the waiting time has elapsed, the process proceeds to step S26. If the waiting time has not elapsed, the process returns to step S23 and checks again whether autonomous driving by automatic driving is possible. In step S26, because autonomous driving by automatic driving is not possible even after the waiting time has elapsed since entering the standby state, vehicle controller 12a again outputs an assistance request and proceeds to step S27. In step S27, vehicle communication device 11a again transmits the assistance request acquired from vehicle controller 12a to remote monitoring communication device 21, and ends processing. When the assistance request is transmitted again in step S27, the remote monitoring device 20 receives the assistance request again, which allows the remote monitor to know that the problem with the selected vehicle was not resolved by the standby instruction, and to decide what action to take next with respect to this selected vehicle.

[0028] As described above, the autonomous driving assistance system according to embodiment 1 is an autonomous driving assistance system that includes a remote monitoring device 20 and on-board devices 10a, 10b that are provided in a plurality of autonomous vehicles 100a, 100b, respectively. On-board device 10a includes vehicle communication device 11a that communicates with remote monitoring device 20, and vehicle controller 12a that controls driving of autonomous vehicle 100a based on peripheral information of autonomous vehicle 100a acquired from peripheral information sensor 2a and vehicle information of autonomous vehicle 100a acquired from vehicle information sensor 1a, and that transmits an assistance request signal to remote monitoring device 20 via vehicle communication device 11a when autonomous driving is not possible. Remote monitoring device 20 includes remote monitoring communication device 21 that communicates with the outside of remote monitoring device 20, autonomous vehicle information display 22 that, when an assistance request is received from on-board device 10a, displays peripheral information acquired from on-board device 10a that has transmitted the assistance request, and an autonomous driving assistance system that includes on-board device 10a that has transmitted the assistance request. The vehicle controller 12a is provided with an autonomous vehicle selector 23 that receives an instruction to select one of the autonomous vehicles 100a and make it the selected vehicle, a work instruction input device 24 that receives work instructions for the selected vehicle, and a remote operation input device 25 that receives input of remote operation information for the selected vehicle, and when the work instruction is a wait instruction, the remote monitoring communication device 21 sends standby information instructing the vehicle to wait to the on-board device 10a of the selected vehicle, and when the work instruction is a request for on-site work, sends information requesting the dispatch of personnel to the selected vehicle to the worker terminal 30, and when the work instruction is a remote operation instruction, sends the remote operation information input to the remote operation input device 25 to the on-board device 10a of the selected vehicle, and when the wait information is sent from the remote monitoring device 20, the vehicle controller 12a performs driving control to maintain the stopped state of the autonomous vehicle 100a, and when remote operation information is sent from the remote monitoring device 20, performs driving control of the autonomous vehicle 100a based on the remote operation information, so that assistance can be provided quickly to autonomous vehicles in situations where priority should be given.

[0029] Remote control is not the only way to resolve situations that an autonomous driving system alone cannot handle, and remote control cannot resolve all situations. For example, if an autonomous vehicle's route is blocked by a roadway obstacle and there is no alternative route, a rational solution would be to remove the fallen object on-site. In the autonomous driving assistance system according to embodiment 1, the remote monitor can choose to respond to the situation of the autonomous vehicle by issuing a standby instruction, requesting on-site work, or issuing a remote control instruction. This allows the remote monitor to shorten the time the autonomous vehicle is in a certain situation compared to a system with only remote control as a means, and to shift to assisting other vehicles with higher priority. Furthermore, even when assistance requests are sent to the remote monitoring device 20 from multiple autonomous vehicles, the impact of the autonomous vehicle's suspension on overall traffic can be reduced by issuing a remote control instruction to an autonomous vehicle in a more urgent situation and issuing a standby instruction or requesting on-site work to an autonomous vehicle in a situation where the remote monitor will be less likely to be involved.

[0030] Embodiment 2 Fig. 5 is a block diagram showing the configuration of an automatic driving assistance system according to embodiment 2. Comparing the automatic driving assistance system according to embodiment 2 shown in Fig. 5 with the automatic driving assistance system according to embodiment 1 shown in Fig. 1, an alert information output device 13a is added to in-vehicle device 10a, an alert information output device 13b is added to in-vehicle device 10b, and an alert information input device 27 is added to remote monitoring device 20. Other configurations of the automatic driving assistance system according to embodiment 2 are the same as those of the automatic driving assistance system according to embodiment 1.

[0031] The operation of on-board device 10a provided in autonomous vehicle 100a will be described below, but the operation of on-board devices provided in other autonomous vehicles is the same. Warning information output device 13a transmits warning information to people or vehicles around autonomous vehicle 100a to alert them. Warning information output device 13a acquires a warning information signal and outputs the warning information. If the warning information is audio, it is a speaker, if the warning information is flashing light, it is a light, and if the warning information is text information, it is a display that displays text. Warning information output device 13a may also transmit warning information to crew members or passengers inside autonomous vehicle 100a. Warning information output device 13a notifies people or vehicles around autonomous vehicle 100a of the presence of autonomous vehicle 100a, the next planned action of autonomous vehicle 100a, requests to those around, etc. For example, when there is a person standing in front of the autonomous vehicle 100a, the warning information output device 13a can use a voice to call out to the person standing in front of the autonomous vehicle 100a, asking them to move out of the path of the autonomous vehicle 100a.

[0032] The warning information input device 27 of the remote monitoring device 20 inputs the warning information output from the warning information output device 13a and outputs a warning information signal corresponding to the input warning information. If the warning information is audio, the warning information input device 27 is a microphone or the like that converts audio into an audio signal, and if the warning information is text information, the warning information input device 27 is a text input device such as a keyboard. The warning information input device 27 may select one of multiple pieces of warning information prepared in advance. The warning information signal output from the warning information input device 27 is transmitted from the remote monitoring communication device 21 to the vehicle communication device 11a of the selected autonomous vehicle 100a, and the vehicle communication device 11a outputs the received warning information signal to the warning information output device 13a. The warning information output device 13a, which has received the warning information signal from the vehicle communication device 11a, outputs warning information corresponding to the warning information signal to the surroundings of the autonomous vehicle 100a.

[0033] Fig. 6 is a flowchart for explaining the operation of the remote monitoring device 20 in the second embodiment. Comparing the flowchart in the second embodiment shown in Fig. 6 with the flowchart in the first embodiment shown in Fig. 3, steps S12 and S13 are added between steps S03 and S04. The other processes in the flowchart in the second embodiment shown in Fig. 6 are the same as those in the flowchart in the first embodiment shown in Fig. 3. Step S12 is an alert information input step, and step S13 is an alert information signal transmission step included in the remote monitoring communication step.

[0034] In step S12, the remote monitor checks the information about the selected autonomous vehicle 100a displayed on the autonomous vehicle information display 22, and inputs the warning information into the warning information input device 27. The warning information input device 27 converts the input warning information into a warning information signal, outputs it to the remote monitoring communication device 21, and the process proceeds to step S13. In step S13, the remote monitoring communication device 21 transmits the acquired warning information signal to the vehicle communication device 11a of the in-vehicle device 10a of the selected autonomous vehicle 100a, and the process proceeds to step S04. The warning information signal transmitted from the remote monitoring communication device 21 to the vehicle communication device 11a is output to the warning information output device 13a, and the warning information output device 13a outputs warning information corresponding to the acquired warning information signal to the surroundings of the autonomous vehicle 100a.

[0035] For example, even if vehicle controller 12a of on-vehicle device 10a has a function to automatically generate or select an alert information signal based on peripheral information and alert information is output from alert information output device 13a based on the generated or selected alert information signal, vehicle controller 12a may not be able to appropriately select a message to be transmitted to the surroundings. In the autonomous driving assistance system according to embodiment 2, the remote monitor can check peripheral information about autonomous vehicle 100a, which is the selected vehicle, and generate or select an alert information signal, and alert information that is appropriate for the situation around autonomous vehicle 100a can be output from alert information output device 13a of autonomous vehicle 100a.

[0036] 6, the remote monitor inputs warning information into warning information input device 27, and then in step S04 inputs a standby instruction as a work instruction to the selected vehicle. In this case, for example, if a person standing in front of autonomous vehicle 100a moves in response to the warning information output around autonomous vehicle 100a, the path of autonomous vehicle 100a, which has been put into a standby state, is cleared, and autonomous driving becomes possible, allowing autonomous driving to resume. This makes it possible to resolve the situation in which the autonomous vehicle finds itself without remote operation by the remote monitor, and to respond to requests for assistance from other autonomous vehicles in more complicated situations.

[0037] As described above, in the automated driving assistance system according to the second embodiment, the remote monitoring device 20 includes the warning information input device 27 that outputs a warning information signal corresponding to the input warning information, the remote monitoring communication device 21 transmits the warning information signal to the on-board device 10a of the selected vehicle, and the on-board device 10a includes the warning information output device 13a that outputs warning information corresponding to the acquired warning information signal around the automated driving assistance system 100a. This increases the likelihood that the remote monitor can resolve the request for assistance from the automated driving assistance system in a simple manner, reduces the workload of the remote monitor, and allows the remote monitor to perform other prioritized tasks. For example, in a situation where the route of the automated driving assistance system is blocked by an obstacle on the road and there is no alternative route, if the obstacle on the road is a vehicle, urging the vehicle to move may be a reasonable solution. In the automated driving assistance system according to the second embodiment, depending on the situation of the automated driving assistance system, the time the remote monitor is detained can be shortened, allowing the remote monitor to shift to assisting other vehicles with higher priority.

[0038] Embodiment 3 The configuration of the autonomous driving assistance system according to the third embodiment is the same as the configuration of the autonomous driving assistance system according to the first embodiment shown in FIG. 1. The operation of the remote monitoring device 20 in the third embodiment differs from the operation of the remote monitoring device 20 in the first embodiment. FIGS. 7 and 8 are flowcharts for explaining the operation of the remote monitoring device 20 in the third embodiment. Comparing the flowcharts of the third embodiment shown in FIGS. 7 and 8 with the flowchart of the first embodiment shown in FIG. 3, step S04 has become step S04a, and the subsequent processing when the work instruction in step S09 is not a remote operation instruction is different. In FIG. 8, steps S14 and S15 are remote monitoring communication steps, step S14 is a monitor on-site work confirmation step, and step S15 is a second standby instruction transmission step.

[0039] In the autonomous driving assistance system according to the third embodiment, the work instructions received by work instruction input device 24 are any one of a standby instruction, an on-site work request, a remote operation instruction, and on-site monitor work. On-site monitor work involves the remote monitor personally going to the selected autonomous vehicle 100a, performing on-site work, resolving any problems, and resuming autonomous driving. On-site monitor work is selected when the distance from remote monitoring device 20 to autonomous vehicles 100a, 100b is short, such as when transport vehicles in a factory are automated to become autonomous vehicles 100a, 100b.

[0040] In step S04a, the work instruction input device 24 accepts the work instruction for the selected vehicle, and proceeds to step S05. In step S09, the remote monitoring communication device 21 checks whether the work instruction is a remote operation instruction, and if the work instruction is a remote operation instruction, proceeds to step S10. If the work instruction is not a remote operation instruction, proceeds to step S14. In step S14, the remote monitoring communication device 21 checks whether the work instruction is for on-site supervisor work, and if the work instruction is for on-site supervisor work, proceeds to step S15. If the work instruction is not for on-site supervisor work, the process ends. In step S15, the remote monitoring communication device 21 transmits standby information to the vehicle communication device of the autonomous vehicle that newly transmitted the assistance request, instructing it to wait until the remote supervisor returns from on-site work and the remote monitoring device 20 is operated again, for example, from the time when on-site supervisor work is selected as the work instruction in the work instruction input device 24 until the remote supervisor returns from on-site work and the remote monitoring device 20 receives an instruction to cancel on-site supervisor work, which is the work instruction for the selected vehicle. Furthermore, in step S15, the remote monitoring communication device 21 may transmit standby information instructing the vehicle communication devices of autonomous vehicles other than the selected vehicle that has already transmitted an assistance request to wait. In step S15, for example, if the selected vehicle is autonomous vehicle 100a equipped with on-board device 10a and the vehicle communication device 11b of autonomous vehicle 100b transmits a new assistance request by the time the remote monitor returns from on-site work on autonomous vehicle 100a and operates the remote monitoring device 20 again, the remote monitoring communication device 21 transmits standby information to the vehicle communication device 11b. In step S15, for example, if the selected vehicle is autonomous vehicle 100a equipped with on-board device 10a and the remote monitoring communication device 21 has already received standby information from the vehicle communication device 11b of autonomous vehicle 100b, the remote monitoring communication device 21 transmits standby information to the vehicle communication device 11b. In step S15, the processing ends when the monitor on-site work, which is the work instruction for the selected vehicle, is canceled.

[0041] As described above, in the autonomous driving assistance system according to embodiment 3, when the work instruction is for the monitor to perform on-site work, the remote monitoring communication device 21 sends standby information to the in-vehicle device that has newly sent the assistance request, so that the remote monitor can go to the autonomous vehicle himself / herself to perform the on-site work, and further, the autonomous vehicle that has newly sent the assistance request can be made to wait while the remote monitor is performing the on-site work.

[0042] Although the present application describes various exemplary embodiments, the various features, aspects, and functions described in one or more embodiments are not limited to application to a particular embodiment, but may be applied to the embodiments alone or in various combinations. Therefore, countless variations not illustrated are conceivable within the scope of the technology disclosed in this application, including, for example, cases where at least one component is modified, added, or omitted, and cases where at least one component is extracted and combined with a component of another embodiment.

[0043] Although the preferred embodiments have been described above in detail, the present invention is not limited to the above-described embodiments. The above-described embodiments may be modified without departing from the scope of the claims. Various modifications and substitutions can be made to the above.

[0044] Various aspects of the present disclosure are summarized below as appendices.

[0045] (Appendix 1) An autonomous driving assistance system including a remote monitoring device and an on-board device provided in each of a plurality of autonomous vehicles, The in-vehicle device a vehicle communication device that communicates with the remote monitoring device; a vehicle controller that controls driving of the autonomous vehicle based on peripheral information of the autonomous vehicle acquired from a peripheral information sensor and vehicle information of the autonomous vehicle acquired from a vehicle information sensor, and transmits a signal requesting assistance to the remote monitoring device through the vehicle communication device when autonomous driving is not possible; The remote monitoring device a remote monitoring communication device that communicates with an external device of the remote monitoring device; an autonomously driven vehicle information display that displays, when the assistance request is received from the in-vehicle device, the surrounding information obtained from the in-vehicle device that transmitted the assistance request; and an autonomous vehicle selector that selects one of the autonomous vehicles equipped with the on-board device that has transmitted the assistance request and receives an instruction to select the selected vehicle; a work instruction input device that receives work instructions for the selected vehicle; a remote control input device that accepts input of remote control information for the selected vehicle; The remote monitoring communication device includes: When the work instruction is a standby instruction, standby information instructing the vehicle to wait is transmitted to the vehicle-mounted device of the selected vehicle; When the work instruction is a request for on-site work, information requesting the dispatch of personnel to the selected vehicle is transmitted to the worker terminal; When the work instruction is a remote operation instruction, the remote operation information input to the remote operation input device is transmitted to the in-vehicle device of the selected vehicle; The vehicle controller When the standby information is received from the remote monitoring device, driving control is performed to maintain the automatically driven vehicle in a stopped state; An automatic driving assistance system characterized in that, when the remote operation information is sent from the remote monitoring device, the system controls the driving of the automatic driving vehicle based on the remote operation information. (Appendix 2) the standby information includes information about a standby time, which is a time for which a standby state is maintained; The autonomous driving assistance system described in Appendix 1 is characterized in that, when the standby information is sent from the remote monitoring device, the vehicle controller resumes autonomous driving if autonomous driving by autonomous driving becomes possible between the time the vehicle entered standby mode and the time the standby time has elapsed, and transmits a signal requesting assistance to the remote monitoring device through the vehicle communication device if autonomous driving by autonomous driving becomes impossible after the time the vehicle entered standby mode and the time the standby time has elapsed. (Appendix 3) the remote monitoring device includes an alert information input device that outputs an alert information signal corresponding to the input alert information; the remote monitoring communication device transmits the warning information signal to the in-vehicle device of the selected vehicle; The autonomous driving assistance system described in Appendix 1 or 2, characterized in that the on-board device is equipped with a warning information output device that outputs the warning information corresponding to the acquired warning information signal to the surroundings of the autonomous vehicle. (Appendix 4) The remote monitoring communication device includes: When the work instruction is for a supervisor to perform on-site work, 4. The automatic driving assistance system according to claim 1, wherein the standby information is transmitted to the in-vehicle device that has recently transmitted the assistance request. [Explanation of symbols]

[0046] 1a, 1b Vehicle information sensors, 2a, 2b Peripheral information sensors, 10a, 10b In-vehicle devices, 11a, 11b Vehicle communication devices, 12a, 12b Vehicle controllers, 13a, 13b Warning information output devices, 20 Remote monitoring devices, 21 Remote monitoring communication devices, 22 Self-driving vehicle information display devices, 23 Self-driving vehicle selector devices, 24 Work instruction input devices, 25 Remote operation input devices, 26 Dispatch request information input devices, 27 Warning information input devices, 30 Worker terminals, 31 Terminal communication devices, 32 Dispatch request information output devices, 100a, 100b Self-driving vehicles.

Claims

1. An autonomous driving assistance system including a remote monitoring device and an on-board device provided in each of a plurality of autonomous vehicles, The in-vehicle device a vehicle communication device that communicates with the remote monitoring device; a vehicle controller that controls driving of the autonomous vehicle based on peripheral information of the autonomous vehicle acquired from a peripheral information sensor and vehicle information of the autonomous vehicle acquired from a vehicle information sensor, and transmits a signal requesting assistance to the remote monitoring device through the vehicle communication device when autonomous driving is not possible; The remote monitoring device a remote monitoring communication device that communicates with an external device of the remote monitoring device; an autonomously driven vehicle information display that displays, when the assistance request is received from the in-vehicle device, the surrounding information obtained from the in-vehicle device that transmitted the assistance request; and an autonomous vehicle selector that selects one of the autonomous vehicles equipped with the on-board device that has transmitted the assistance request and receives an instruction to select the selected vehicle; a work instruction input device that receives work instructions for the selected vehicle; a remote control input device that accepts input of remote control information for the selected vehicle; The remote monitoring communication device includes: When the work instruction is a standby instruction, standby information instructing the vehicle to wait is transmitted to the vehicle-mounted device of the selected vehicle; When the work instruction is a request for on-site work, information requesting the dispatch of personnel to the selected vehicle is transmitted to the worker terminal; When the work instruction is a remote operation instruction, the remote operation information input to the remote operation input device is transmitted to the in-vehicle device of the selected vehicle; The vehicle controller When the standby information is received from the remote monitoring device, driving control is performed to maintain the automatically driven vehicle in a stopped state; When the remote operation information is sent from the remote monitoring device, driving control of the autonomous vehicle is performed based on the remote operation information, the standby information includes information about a standby time, which is a time for which a standby state is maintained; When the vehicle controller receives the standby information from the remote monitoring device, the vehicle controller resumes autonomous driving if autonomous driving by automatic driving becomes possible between the time the vehicle entered standby mode and the time the standby time has elapsed, and transmits a signal requesting assistance to the remote monitoring device through the vehicle communication device if autonomous driving by automatic driving becomes impossible after the time the vehicle entered standby mode and the time the standby time has elapsed.

2. the remote monitoring device includes an alert information input device that outputs an alert information signal corresponding to the input alert information; the remote monitoring communication device transmits the warning information signal to the in-vehicle device of the selected vehicle; The autonomous driving assistance system according to claim 1, characterized in that the in-vehicle device includes an alert information output device that outputs the alert information corresponding to the acquired alert information signal to the surroundings of the autonomous driving vehicle.

3. The remote monitoring communication device includes: When the work instruction is for a supervisor to perform on-site work, The automatic driving assistance system according to claim 1, characterized in that the standby information is transmitted to an in-vehicle device of the automatically driven vehicle that has not been selected and that has newly transmitted the assistance request.

Images