Remote driving system and remote driving method

Abstract

Provided are a remote driving system and a remote driving method that are easily applicable to a moving body. A moving body monitoring device monitors the traveling state of a moving body and the surrounding environment of the moving body, and outputs monitoring information indicating the monitoring result. A remote command device provides a remote driver with information for remotely driving the moving body on the basis of the monitoring information, and outputs a driving command generated on the basis of the information for the remote driver to remotely drive the moving body. A driving control device is provided in the moving body and controls remote driving of the moving body in accordance with the driving command.

Description

Remote operation system and remote operation method

[0001] The present disclosure relates to a remote driving system and a remote driving method.

[0002] In recent years, technological developments related to the operation of moving objects such as vehicles have progressed, for example, the development of technologies that allow an external operator to remotely operate a moving object has progressed.

[0003] For example, Patent Document 1 proposes a method for remotely controlling a moving object by simulating a group of objects including the moving object based on information collected by the moving object.

[0004] Japanese Patent Application Laid-Open No. 2023-169715

[0005] The method described in Patent Document 1 requires that the mobile object be equipped with an advanced system that integrates devices such as cameras and sensors that collect information about the mobile object's surrounding environment to the extent that remote driving is possible, and a control device that enables remote driving. Therefore, building the system requires significant costs, including equipment costs, system maintenance costs, and equipment for compliance with regulations.

[0006] Therefore, there is a demand for a system that can be introduced more simply and at low cost to remotely operate moving objects such as vehicles.

[0007] The remote driving system according to the present disclosure comprises a monitoring means for monitoring the running state of a mobile body and the environment surrounding the mobile body and outputting monitoring information indicating the monitoring results, a remote command means for providing a remote driver with information for remotely driving the mobile body based on the monitoring information and outputting driving commands generated based on the information for the remote driver to remotely drive the mobile body, and a driving control means provided on the mobile body for controlling the remote driving of the mobile body in accordance with the driving commands.

[0008] The remote driving method disclosed herein monitors the running state of a mobile body and the surrounding environment of the mobile body, outputs monitoring information indicating the monitoring results, provides a remote driver with information for remotely driving the mobile body based on the monitoring information, outputs driving commands generated based on the information for the remote driver to remotely drive the mobile body, and controls the remote driving of the mobile body in accordance with the driving commands using a driving control means provided on the mobile body.

[0009] According to the present disclosure, it is possible to provide a remote driving system and a remote driving method that can be easily applied to a moving body.

[0010] FIG. 1 is a diagram schematically showing the configuration of a remote driving system according to an embodiment. FIG. 1 is a diagram showing an example when a moving body is a vehicle. FIG. 2 is a block diagram schematically showing the configuration of a remote command device. FIG. 2 is an example of an overhead view of the vicinity of a moving body. FIG. 3 is a diagram showing an example of an image from a viewpoint from a moving body. FIG. 1 is a block diagram schematically showing the configuration of a remote command device. FIG. 2 is a diagram showing a first example of a display using auxiliary information. FIG. 3 is a diagram showing a second example of a display using auxiliary information. FIG. 4 is a diagram showing a third example of a display using auxiliary information. FIG. 4 is a diagram showing an example when a remote driving system according to an embodiment controls a plurality of vehicles. FIG. 5 is a diagram showing an example of the configuration of a computer for realizing one or both of a remote command device and a driving control device.

[0011] Hereinafter, embodiments of the present invention will be described with reference to the drawings. In the drawings, the same elements are designated by the same reference numerals, and redundant explanations will be omitted as necessary.

[0012] When referring to one embodiment below, it means that the present invention can be applied to any one of the embodiments described below or a combination of two or more embodiments, and is not limited to a specific embodiment.

[0013] First Embodiment A remote driving system according to a first embodiment will be described below. The remote driving system is configured to remotely drive a mobile object based on the results of monitoring the mobile object and the environment surrounding the mobile object.

[0014] 1 is a diagram schematically illustrating the configuration of a remote driving system according to one embodiment. The remote driving system 1000 is configured to remotely drive a mobile object 10 based on the results of monitoring the mobile object 10 and the results of monitoring the external environment of the mobile object 10. The remote driving system 1000 includes a mobile object monitoring device 1, a remote command device 2, and an operation control device 3.

[0015] The following description will be given taking the case where the moving body 10 is a vehicle as an example. Fig. 2 is a diagram showing an example where the moving body is a vehicle.

[0016] The mobile object monitoring device 1 monitors the mobile object 10 and the environment surrounding the mobile object 10. The mobile object monitoring device 1 monitors the movement state of the mobile object 10, such as the position, direction of travel, speed, and acceleration of the mobile object 10. For example, the mobile object monitoring device 1 may be a device provided outside the mobile object 10.

[0017] If the moving object 10 is a vehicle, the moving object monitoring device 1 may be, for example, a camera installed along the road on which the moving object 10 travels. In this case, the camera constituting the moving object monitoring device 1 outputs monitoring information MON including video or images of the moving object 10 to the remote command device 2. The camera may be of various types, such as a camera that captures visible light images or a camera that captures infrared images. The camera may also be a general PTZ camera that captures a rectangular area, or a camera with various imaging ranges, such as a surveillance camera with a fisheye lens.

[0018] Furthermore, the mobile object monitoring device 1 may be any of various sensors that detect mobile objects using radio waves, such as a sensor embedded in a road, a sensor installed on a road, or a sensor installed at the side of a road. The means used by the sensor to monitor the mobile object 10 is not limited to radio waves, and various means capable of detecting the mobile object 10, such as infrared rays or laser light, may also be used. The mobile object monitoring device 1 outputs monitoring information MON including the monitoring results of the mobile object 10 to the remote command device 2.

[0019] The remote command device 2 generates a driving command INS for causing the mobile object 10 to move in a desired direction when operated by an operator who is a remote driver, based on the monitoring information MON received from the mobile object monitoring device 1. Then, the remote command device 2 outputs the driving command INS to the mobile object 10 by various communication means, for example, wireless communication.

[0020] 3 is a block diagram showing a schematic configuration of the remote command device 2. The remote command device 2 has a communication unit 21, a processing unit 22, a display unit 23, and a command input unit 24.

[0021] The communication unit 21 receives monitoring information MON from the moving object monitoring device 1 and outputs a driving command INS to the driving control device 3 .

[0022] The processing unit 22 generates, from the monitoring information MON, information INF1 about the moving object 10 and the surrounding environment of the moving object 10 to be displayed on the display unit 23. The processing unit 22 outputs the generated display information INF1 to the display unit 23.

[0023] The display unit 23 displays display information INF1 indicating the moving object 10 and the surrounding environment of the moving object 10 to the operator of the remote command device 2. This allows the operator to understand the moving state of the moving object 10 and the situation of the moving route.

[0024] The display unit 23 may display, for example, an overhead view of the vicinity of the mobile object 10 based on the monitoring information MON. Fig. 4 is an example of an overhead view of the vicinity of the mobile object. This overhead view displays the road 11 on which the mobile object 10 is traveling, as well as the position and direction of travel of the mobile object 10. The overhead view also displays the situation in the vicinity of the mobile object 10. In Fig. 4, as an example, a passerby walking on the sidewalk is displayed.

[0025] The display unit 23 may display, for example, a video image from the viewpoint of the moving object 10, which is generated based on the monitoring information MON. Fig. 5 is a diagram showing an example of a video image from the viewpoint of the moving object. The video image from the viewpoint of the moving object may be a pseudo video image such as a 3D polygon generated from the video and image included in the monitoring information MON.

[0026] The command input unit 24 receives a driving command INS to be given to the mobile object 10, which the operator has determined based on the information displayed on the display unit 23. When the mobile object 10 is a vehicle, the command input unit 24 may include various devices provided in the driver's seat of the vehicle, such as a steering wheel and pedals. This allows the operator to remotely drive the mobile object 10 in the same way as driving a normal vehicle.

[0027] The operation control device 3 is installed or carried on the moving body 10, and receives the operation command INS output from the remote command device 2. Then, the operation control device 3 provides a control signal CON to the moving body 10 based on the operation command INS.

[0028] In the following description, the driving control device 3 is assumed to be installed in the mobile body 10. The driving control device 3 is attached to a dashboard or the like inside the vehicle cabin of the mobile body 10 and is configured to be able to communicate with the remote command device 2 through a windshield or the like. In this case, the driving control device 3 may be incorporated into, for example, an external car navigation device or a drive recorder device. The driving control device 3 may be connected to the mobile body 10 via a cable or the like. The driving control device 3 may also be connected to the mobile body 10 by various wireless communication means such as Bluetooth. This allows the driving control device 3 to provide the mobile body 10 with a control signal CON for controlling steering, acceleration / deceleration, etc.

[0029] The driving control device 3 may also be a communication-capable terminal device such as a dedicated terminal, smartphone, or tablet computer that can be carried by a passenger of the vehicle 10. In this case, when the passenger gets on the vehicle 10, the vehicle 10 and the driving control device 3 may be connected by a cable, or by various wireless communication means such as Bluetooth. This allows the driving control device 3 to provide the vehicle 10 with a control signal CON for controlling steering, acceleration / deceleration, etc.

[0030] This makes it possible to realize remote operation of the mobile body 10 in accordance with the monitoring results of the surrounding environment outside the mobile body 10. The operation control device 3 can be configured as a device that can be retrofitted to the mobile body 10, so by retrofitting the operation control device 3 to the mobile body 10, the remote operation system 1000 can be easily applied to the mobile body 10.

[0031] In this case, when a general-purpose terminal device such as a smartphone or tablet computer is used, the operation control device 3 can be easily realized by simply installing software to the terminal device to provide the functions of the operation control device 3. As a result, the remote operation system 1000 can be easily introduced to any mobile object 10.

[0032] As described above, the remote driving system 1000 makes it possible to realize remote driving of a moving object with a simple configuration and at low cost.

[0033] Second Embodiment A remote driving system according to a second embodiment will be described. The remote driving system according to this embodiment is configured to provide information useful for remote driving of a mobile body 10 to an operator of a remote command device. Fig. 6 is a diagram schematically showing the configuration of a remote driving system according to one embodiment. The remote driving system 2000 has a configuration in which the remote command device 2 of the remote driving system 1000 is replaced with a remote command device 4.

[0034] 7 is a block diagram showing a schematic configuration of the remote command device 4. The remote command device 4 has a communication unit 41, a processing unit 42, a display unit 43, and a command input unit 44.

[0035] The communication unit 41, like the communication unit 21, receives the monitoring information MON from the mobile object monitoring device 1. Then, the communication unit 41 outputs the received monitoring information MON to the processing unit .

[0036] The processing unit 42 generates, from the monitoring information MON, information INF1 about the moving object 10 and the surrounding environment of the moving object 10 to be displayed on the display unit 43. The processing unit 42 also generates, from the monitoring information MON, auxiliary information INF2 useful for remote driving to be provided to the operator in addition to the display information INF1. The processing unit 42 outputs the generated display information INF1 and auxiliary information INF2 to the display unit 43.

[0037] The display unit 43 displays the moving object 10 and the surrounding environment of the moving object 10 indicated by the display information INF1 to the operator of the remote command device 4. This allows the operator to understand the current state of the moving object 10 and the status of the moving route.

[0038] In addition, the display unit 43 displays information useful for remotely driving the mobile body 10 to the operator based on the auxiliary information INF2.

[0039] The auxiliary information INF2 will be described below. The auxiliary information INF2 may include information about obstacles on the travel route of the mobile object 10. FIG. 8 is a diagram showing a first example of a display using auxiliary information. For example, the display unit 43 may highlight other vehicles 12 parked on the road 11 as obstacles to the mobile object 10 based on the auxiliary information INF2. In this case, the operator of the remote command device 4 can remotely drive the mobile object 10 so as to avoid the vehicles 12 that are obstacles.

[0040] Here, the highlighting may be performed in various ways, such as thickening the outline of the object, changing the color, or flashing the object. Furthermore, in addition to the outline, the color of the object itself may be changed or flashed. Furthermore, text may be displayed to alert the operator.

[0041] FIG. 9 is a diagram illustrating a second example of a display using auxiliary information. The auxiliary information INF2 may include information indicating an object located in a blind spot for the mobile object 10 traveling on the road 11. For example, assume that the processing unit 42 detects another vehicle 15 at a position obscured by a building 14 at an intersection 13 based on the monitoring information MON. If the processing unit 42 determines that the vehicle 15 is likely to enter the intersection, the processing unit 42 may include information indicating the position and traveling direction of the vehicle 15 in the auxiliary information INF2 to notify the presence of the vehicle 15. In this case, the display unit 43 may display the vehicle 15, which is obscured by the building 14 and cannot be seen, on the screen using a dashed line diagram or a 3D polygon. The display unit 43 may also display the traveling direction of the vehicle 15 using an arrow. The display unit 43 may also display the predicted movement of the vehicle 15 using animation.

[0042] FIG. 10 illustrates a third example of a display using auxiliary information. The auxiliary information INF2 may include information about an object about to enter the road, such as a pedestrian 16 who may run out onto the road. For example, assume that the processing unit 42 detects a pedestrian 16 attempting to cross the road 11 based on the monitoring information MON. If the processing unit 42 determines that the pedestrian 16 may run out onto the road 11, the processing unit 42 may include information indicating the position and direction of travel of the pedestrian 16 in the auxiliary information INF2 to alert the pedestrian 16. In this case, the processing unit 42 may display the pedestrian 16 on the screen using a dashed line diagram or a 3D polygon. The processing unit 42 may also display the pedestrian 16's direction of travel using an arrow. The processing unit 42 may also display the predicted movement of the pedestrian 16 using animation.

[0043] As described above, according to the remote driving system 2000, by providing the operator with information useful for remote driving based on the auxiliary information INF2, the mobile body 10 can be safely remotely driven in accordance with the surrounding environment of the mobile body 10.

[0044] In the above-described embodiment, the remote driving system has been described as controlling a single vehicle 10. However, the remote driving system may also control a plurality of vehicles. Fig. 11 is a diagram showing an example in which the remote driving system according to one embodiment controls a plurality of vehicles.

[0045] In this example, it is assumed that three moving bodies 10A to 10C are traveling on a road 11. Here, it is assumed that the moving bodies 10A to 10C are in an automatic driving state. The moving body 10A is the vehicle traveling at the front. The moving body 10B is the vehicle following the moving body 10A and traveling at a slower speed than the moving body 10A. Therefore, the distance between the moving body 10A and the moving body 10B is increasing. The moving body 10C is the vehicle following the moving body 10B and traveling at a higher speed than the moving body 10B. Therefore, the distance between the moving body 10B and the moving body 10C is decreasing.

[0046] 11 , by appropriately positioning the mobile object monitoring device 1, it is possible to collectively monitor the status of mobile objects 10A to 10C traveling on a road 11. Therefore, the operator can comprehensively grasp the traveling status and inter-vehicle distance of the mobile objects 10A to 10C through the remote command device 2. Therefore, if there is a change in the inter-vehicle distance between the mobile objects 10A to 10C, the operator may intervene in the automatic driving of the mobile objects 10A to 10C as appropriate to maintain the inter-vehicle distance at a predetermined distance.

[0047] 11, because the distance between moving body 10A and moving body 10B is increasing, the operator may intervene in the automatic driving of moving body 10B and remotely drive moving body 10B to accelerate until the inter-vehicle distance becomes a predetermined distance. Also, because the distance between moving body 10B and moving body 10C is decreasing, the operator may intervene in the automatic driving of moving body 10C and remotely drive moving body 10C to decelerate until the inter-vehicle distance becomes a predetermined distance.

[0048] It is also possible that the vehicle 10B may be switched from automatic driving to manual driving by the passenger, for example. In this case, the operator may intervene in the automatic driving of the vehicle 10C as appropriate depending on the behavior of the manually driven vehicle 10B.

[0049] In this embodiment, for the sake of simplicity, an example in which multiple vehicles are traveling in a single lane has been described. However, multiple lanes may exist. In this case, the operator may intervene in the automated driving of each vehicle depending on the driving conditions of the vehicles traveling in each of the multiple lanes. The operator may also intervene in the automated driving of each vehicle, including vehicles traveling in the oncoming lane.

[0050] As described above, the automated driving system according to the present embodiment can appropriately remotely drive each of the multiple moving bodies depending on the moving conditions of the moving bodies. In particular, when a situation that cannot be avoided by automated driving or manual driving is predicted, the system can intervene in the driving of the moving bodies and remotely drive them, thereby making it possible to avoid situations in which the moving bodies encounter accidents, etc.

[0051] Other Embodiments The present disclosure has been described above with reference to the embodiments, but the present disclosure is not limited to the above-described embodiments. Various modifications that can be understood by those skilled in the art can be made to the configuration and details of the present disclosure within the scope of the present disclosure. Furthermore, each embodiment can be combined with other embodiments as appropriate.

[0052] In the above embodiment, the mobile body 10 has been described as a vehicle that travels on roads, but the mobile body 10 is not limited to this. The mobile body 10 may be a vehicle that travels off roads. The mobile body 10 may also be any mobile body that travels on land, such as an electric cart.

[0053] The mobile body 10 may also be a mobile body that moves on a surface other than land. The mobile body 10 may be, for example, an aircraft, an airship, a balloon, or any other type of mobile body that moves through the air. The mobile body 10 may also be a ship or any other type of mobile body that moves on or under the sea.

[0054] Furthermore, the mobile body 10 may be any of various unmanned mobile bodies that move on land, in the air, on the sea, or underwater, such as so-called drones.

[0055] The driving control device may output information related to remote driving obtained from the mobile object, such as steering angle and speed, to the remote command device. The remote command device may compare the remote driving status of the mobile object obtained from the monitoring information with the remote driving status obtained from the driving control device and calibrate the driving commands. Since the dimensions, weight, and movement characteristics vary depending on the mobile object, calibration can be performed to adjust the remote driving status to the desired state.

[0056] The information on the surrounding environment of the mobile body may include various types of information other than that described in the above embodiment. For example, the information on the surrounding environment of the mobile body may include information indicating the effects of weather such as rainfall or snowfall, the occurrence of traffic congestion, the presence of broken-down vehicles, construction sites, traffic restrictions, the approach of emergency vehicles, etc.

[0057] Communication between the remote control device and the operation control device may use any one or a combination of various wireless communication means, such as 4G communication, 5G communication, V2x (Vehicle to X), optical communication, BLE (Bluetooth Low Energy), and Wi-Fi (Wireless Fidelity).

[0058] The remote operation system may be configured by appropriately combining two or more remote operation systems according to the embodiments.

[0059] For example, in the third embodiment, the remote operation system according to either the first or second embodiment, or a remote operation system that combines the remote operation systems according to two or more embodiments may be applied.

[0060] The mobile object monitoring device may include various devices capable of acquiring an image of the environment surrounding the mobile object, such as a camera provided in the driving control device.

[0061] In the above-described embodiments, the remote command device and operation control device according to the present disclosure have been described primarily as hardware configurations, but this is not limited thereto. It is also possible to realize one or both of the remote command device and operation control device according to the present disclosure by having a computer execute a computer program to perform any desired processing. These processes may be realized by having a computer including at least one processor (e.g., a microprocessor, CPU, GPU, MPU, DSP (Digital Signal Processor)) execute the program. Specifically, one or more programs including instructions for causing a computer to perform these algorithms related to transmission signal processing or reception signal processing may be created, and the programs may be supplied to the computer.

[0062] A computer program can be stored and provided to a computer using various types of non-transitory computer-readable media. Non-transitory computer-readable media include various types of tangible storage media. Examples of non-transitory computer-readable media include magnetic recording media (e.g., flexible disks, magnetic tapes, hard disk drives), magneto-optical recording media (e.g., magneto-optical disks), CD-ROMs (Read Only Memory), CD-Rs, CD-R / Ws, and semiconductor memories (e.g., mask ROMs, PROMs (Programmable ROMs), EPROMs (Erasable PROMs), flash ROMs, and RAMs (Random Access Memory)). The program may be provided to the computer by various types of transient computer-readable media. Examples of transient computer-readable media include electrical signals, optical signals, and electromagnetic waves. The transient computer-readable media can provide the program to the computer via a wired communication path such as an electrical wire or optical fiber, or via a wireless communication path.

[0063] An example of the configuration of a computer for realizing one or both of the remote command device and the operation control device is shown below. FIG. 12 is a diagram showing an example of the configuration of a computer for realizing one or both of the remote command device and the operation control device. One or both of the remote command device and the operation control device can be realized by a computer 9000 such as a dedicated computer or a personal computer (PC). However, the computer does not need to be physically single, and may be multiple when performing distributed processing. As shown in FIG. 12, the computer 9000 has, for example, a processor 9001, a ROM (Read Only Memory) 9002, a RAM (Random Access Memory) 9003, a storage unit 9004, a communication interface 9005, and a user interface 9006.

[0064] The processor 9001, ROM 9002, RAM 9003, storage unit 9004, communication interface 9005, and user interface 9006 are connected to each other so as to be able to communicate with each other via a bus 9007. Note that although explanation of the OS software for operating the computer is omitted, it is also installed in the computer 9000 as appropriate.

[0065] The ROM 9002 is configured by, for example, a nonvolatile semiconductor memory device, etc. The ROM 9002 stores information such as various programs used by the computer 9000.

[0066] The storage unit 9004 is configured with various storage devices such as a hard disk, a solid state disk, etc. Furthermore, the storage unit 9004 is not limited to a storage device installed in the computer 9000, but may be a storage device external to the computer 9000. The external storage device may be a cloud storage device connected to the computer 9000 via various communication means, for example, a network. The storage unit 9004 stores information such as various programs and data used by the computer 9000.

[0067] The RAM 9003 is configured by a volatile semiconductor memory device, etc. Programs, data, and other information used by the processor 9001 are loaded into the RAM 9003 from one or both of the ROM 9002 and the storage unit 9004 as appropriate.

[0068] The processor 9001 may be configured with, for example, a CPU (Central Processing Unit). Furthermore, the processor 9001 may include not only a CPU but also a GPU (Graphics Processing Unit). A GPU is suitable for performing routine processing in parallel, and by applying it to neural network processing, for example, it is possible to improve processing speed compared to a CPU. The processor 9001 executes various processes based on various programs stored in the ROM 9002 or various programs and data held in the RAM 9003, as appropriate. Furthermore, the processor 9001 may store data generated by the processing in the RAM 9003 or the storage unit 9004, as appropriate.

[0069] The communication interface 9005 is an interface that connects the computer 9000 to a communication network such as the Internet or an intranet via various wired communication means or wireless communication means, etc. This allows the computer 9000 to communicate with other devices, systems, sensors, etc. that are connected to the communication network.

[0070] The user interface 9006 includes, for example, a display unit that provides information so that the user can recognize it using a display device or the like, and an audio output unit that outputs audio. The user interface 9006 also includes an input unit that allows the user to input information to the computer 9000 by operating it, such as a keyboard, a mouse, or a touch panel. The user interface 9006 may also include devices such as sensors that obtain information useful to the user.

[0071] Although the computer 9000 has been described as a single device here, this is merely an example. The computer 9000 may be composed of multiple physically separated devices. Some of the multiple devices may be portable devices, and other devices may be stationary devices.

[0072] Each drawing is merely an example for describing one or more embodiments. Each drawing may not relate to only one particular embodiment, but may also relate to one or more other embodiments. As will be understood by those skilled in the art, various features or steps described with reference to any one drawing can be combined with features or steps shown in one or more other drawings to create, for example, an embodiment not explicitly shown or described. Not all features or steps shown in any one drawing are necessary to describe an exemplary embodiment, and some features or steps may be omitted. The order of steps described in any drawing may be changed as appropriate.

[0073] A part or all of the above-described embodiments can be described as, but not limited to, the following supplementary notes.

[0074] (Supplementary Note 1) A remote driving system comprising: a monitoring means for monitoring the running state of a mobile body and the surrounding environment of the mobile body, and outputting monitoring information indicating the monitoring results; a remote command means for providing a remote driver with information for remotely driving the mobile body based on the monitoring information, and outputting driving commands generated based on the information for the remote driver to remotely drive the mobile body; and a driving control means provided on the mobile body, for controlling the remote driving of the mobile body in accordance with the driving commands.

[0075] (Appendix 2) The remote driving system described in Appendix 1, wherein the driving control means is a terminal device that is physically separated from the moving body and can be carried by a passenger of the moving body.

[0076] (Appendix 3) A remote driving system as described in Appendix 1 or 2, wherein the remote command means displays information indicating the moving body and the surrounding environment of the moving body in a manner that is visible to the remote driver based on the monitoring information.

[0077] (Appendix 4) A remote driving system described in any one of Appendices 1 to 3, wherein the remote command means displays a bird's-eye view showing the moving body and the surrounding environment of the moving body, generated based on the monitoring information, as information indicating the moving body and the surrounding environment of the moving body.

[0078] (Appendix 5) A remote driving system described in any one of Appendices 1 to 4, wherein the remote command means displays a diagram showing the moving body and the surrounding environment of the moving body from the viewpoint of the moving body, generated based on the monitoring information, as information showing the moving body and the surrounding environment of the moving body.

[0079] (Appendix 6) A remote driving system described in any one of Appendices 1 to 5, wherein the remote command means notifies the remote driver of information about an object that is an obstacle to the movement of the moving body when the object is an obstacle.

[0080] (Appendix 7) A remote driving system described in any one of Appendices 3 to 5, wherein the remote command means displays an object that is an obstacle to the movement of the moving body to the remote driver if there is such an object.

[0081] (Appendix 8) A remote driving system described in any one of Appendices 1 to 5, wherein the monitoring means monitors multiple moving bodies, and when the multiple moving bodies are not being remotely driven, the remote driver outputs the driving command to each of the multiple moving bodies based on the monitoring information of the multiple moving bodies displayed on the remote command means, and the driving control means of each of the multiple moving bodies controls the remote driving of the moving body based on the driving command.

[0082] (Appendix 9) A remote driving system described in any one of Appendices 1 to 8, wherein the driving control means acquires information indicating the driving state from the mobile body and outputs it to the remote command means, and the remote command means compares the information indicating the driving state of the mobile body acquired from the monitoring information with the information indicating the driving state of the mobile body by the driving control means, and calibrates the driving command to be given to the mobile body based on the comparison result.

[0083] (Supplementary Note 10) A remote driving system according to any one of Supplementary Notes 1 to 9, wherein the monitoring means includes a detection means for the moving body provided in the surrounding environment of the moving body.

[0084] (Appendix 11) A remote driving system described in any one of Appendices 1 to 10, wherein the monitoring means includes a means provided in the driving control means capable of collecting information about the surrounding environment of the moving body.

[0085] (Appendix 12) A remote operation system described in any one of Appendices 1 to 11, wherein the remote command means and the operation control means are communicably connected by wireless communication means.

[0086] (Appendix 13) The remote operation system described in Appendix 12, wherein the remote command means and the operation control means are communicatively connected by combining a plurality of the wireless communication means.

[0087] (Appendix 14) A remote driving method comprising: monitoring the running state of a mobile body and the surrounding environment of the mobile body; outputting monitoring information indicating the monitoring results; providing a remote driver with information for remotely driving the mobile body based on the monitoring information; outputting driving commands generated based on the information for the remote driver to remotely drive the mobile body; and controlling the remote driving of the mobile body in accordance with the driving commands by a driving control means provided in the mobile body.

[0088] (Appendix 15) A remote driving method according to appendix 14, wherein the driving control means is a terminal device that is physically separated from the moving body and can be carried by a passenger of the moving body.

[0089] (Supplementary Note 16) A remote driving method as described in Supplementary Note 14 or 15, in which information indicating the moving body and the surrounding environment of the moving body is displayed so as to be visible to the remote driver based on the monitoring information.

[0090] (Appendix 17) A remote driving method described in any one of Appendices 14 to 16, in which a bird's-eye view showing the moving body and the surrounding environment of the moving body, generated based on the monitoring information, is displayed as information showing the moving body and the surrounding environment of the moving body.

[0091] (Appendix 18) A remote driving method described in any one of Appendices 14 to 17, in which a diagram showing the moving body and the surrounding environment of the moving body from the viewpoint of the moving body, generated based on the monitoring information, is displayed as information showing the moving body and the surrounding environment of the moving body.

[0092] (Appendix 19) A remote driving method described in any one of Appendices 14 to 18, wherein, if there is an object that obstructs the movement of the moving body, information about the obstructing object is notified to the remote driver.

[0093] (Supplementary Note 20) A remote driving method described in any one of Supplementary Notes 16 to 18, wherein if there is an object that is an obstacle to the movement of the moving body, the obstacle object is displayed to the remote driver.

[0094] REFERENCE SIGNS LIST 1 Mobile object monitoring device 2, 4 Remote command device 3 Operation control device 10, 10A to 10C Mobile object 11 Road 12 Vehicle 13 Intersection 14 Building 15 Vehicle 16 Pedestrian 21, 41 Communication unit 22, 42 Processing unit 23, 43 Display unit 24, 44 Command input unit 1000, 2000 Remote operation system 9000 Computer 9001 Processor 9002 ROM 9003 RAM 9004 Storage unit 9005 Communication interface 9006 User interface 9007 Bus CON Control signal INF1 Display information INF2 Auxiliary information INS Operation command MON Monitoring information

Claims

1. A remote driving system comprising: a monitoring means for monitoring the running state of a mobile body and the surrounding environment of the mobile body and outputting monitoring information indicating the monitoring results; a remote command means for providing a remote driver with information for remotely driving the mobile body based on the monitoring information and outputting driving commands generated based on the information for the remote driver to remotely drive the mobile body; and a driving control means provided on the mobile body for controlling the remote driving of the mobile body in accordance with the driving commands.

2. The remote driving system according to claim 1, wherein the driving control means is a terminal device that is physically separated from the vehicle and can be carried by a passenger of the vehicle.

3. The remote driving system according to claim 1 or 2, wherein the remote command means displays information indicating the moving body and the surrounding environment of the moving body in a manner that is visible to the remote driver based on the monitoring information.

4. The remote driving system according to claim 1 or 2, wherein the remote command means displays a bird's-eye view showing the moving body and its surrounding environment, generated based on the monitoring information, as information showing the moving body and its surrounding environment.

5. The remote driving system according to claim 1 or 2, wherein the remote command means displays a diagram showing the moving body and its surrounding environment from the viewpoint of the moving body, generated based on the monitoring information, as information showing the moving body and its surrounding environment.

6. The remote driving system according to claim 1 or 2, wherein, when an object that obstructs the movement of the mobile body is present, the remote command means notifies the remote driver of information about the obstructing object.

7. The remote driving system according to claim 3, wherein, when there is an object that is an obstacle to the movement of the mobile body, the remote command means displays the obstacle to the remote driver.

8. A remote driving system as described in claim 1 or 2, wherein the monitoring means monitors a plurality of moving bodies, and when the plurality of moving bodies are not being remotely driven, the remote driver outputs the driving command to each of the plurality of moving bodies based on the monitoring information of the plurality of moving bodies displayed on the remote command means, and the driving control means of each of the plurality of moving bodies controls the remote driving of the moving body based on the driving command.

9. A remote driving system as described in claim 1 or 2, wherein the driving control means acquires information indicating the driving state from the mobile body and outputs it to the remote command means, and the remote command means compares the information indicating the driving state of the mobile body acquired from the monitoring information with the information indicating the driving state of the mobile body from the driving control means, and calibrates the driving commands to be given to the mobile body based on the comparison result.

10. The remote driving system according to claim 1 or 2, wherein the monitoring means includes a detection means for the moving object provided in the surrounding environment of the moving object.

11. The remote driving system according to claim 1 or 2, wherein the monitoring means includes means provided in the driving control means that is capable of collecting information about the surrounding environment of the moving object.

12. The remote operation system according to claim 1 or 2, wherein the remote command means and the operation control means are communicably connected by wireless communication means.

13. The remote operation system according to claim 12, wherein the remote command means and the operation control means are communicably connected by combining a plurality of the wireless communication means.

14. A remote driving method comprising: monitoring the running state of a mobile body and the environment surrounding the mobile body; outputting monitoring information indicating the monitoring results; providing a remote driver with information for remotely driving the mobile body based on the monitoring information; outputting driving commands generated based on the information for the remote driver to remotely drive the mobile body; and controlling the remote driving of the mobile body in accordance with the driving commands using driving control means provided in the mobile body.

15. The remote driving method according to claim 14, wherein the driving control means is a terminal device that is physically separated from the moving body and can be carried by a passenger of the moving body.

16. The remote driving method according to claim 14 or 15, further comprising displaying information indicating the mobile object and the surrounding environment of the mobile object in a manner that is visible to the remote driver based on the monitoring information.

17. A remote driving method according to claim 14 or 15, wherein a bird's-eye view showing the moving body and its surrounding environment, generated based on the monitoring information, is displayed as information showing the moving body and its surrounding environment.

18. A remote driving method as described in claim 14 or 15, wherein a diagram showing the moving body and the surrounding environment of the moving body from the viewpoint of the moving body, generated based on the monitoring information, is displayed as information showing the moving body and the surrounding environment of the moving body.

19. The remote driving method according to claim 14 or 15, wherein, when an object that obstructs the movement of the mobile body is present, information about the obstructing object is notified to the remote driver.

20. The remote driving method according to claim 16, wherein, if there is an object that is an obstacle to the movement of the mobile object, the obstacle is displayed to the remote driver.

Images