Remote control device and remote control method

Abstract

The present invention includes: an acquisition unit (11) for acquiring information relating to other remote operating devices (1) capable of remotely operating a mobile body via communication; a delay time detection unit (12) for detecting communication delay time in communication with the mobile body; a delay time determination unit (131) for determining whether the communication delay time detected by the delay time detection unit (12) exceeds a delay time determination threshold; a switching target determination unit (131) for determining whether a switching target remote operating device is selected from the other remote operating devices (1) based on the information relating to other remote operating devices (1) acquired by the acquisition unit (11) and whether a switching determination condition is met; and a switching control unit (13) for outputting switching instruction information for remotely operating the mobile body to the switching target remote operating device selected by the switching target determination unit (132).

Description

Technical Field

[0001] This invention relates to a remote operation device and a remote operation method for remotely operating a mobile body. Background Technology

[0002] In recent years, the development of remote driving assistance services (hereinafter referred to as "remote driving") for remote operation of mobile vehicles has been highly anticipated. Remote driving requires maintaining a normal operating state.

[0003] As a technology for maintaining remote driving when the operator of a mobile body becomes unable to continue operating the mobile body during operation, for example, Patent Document 1 discloses a remote operating system that acquires the operator's state during operation and switches the operator of the mobile body to another operator based on the acquired state.

[0004] Existing technical documents

[0005] Patent documents

[0006]

Patent Document 1

[0007] Japanese Patent Application Publication No. 2006-301723 Summary of the Invention

[0008] The technical problem that the invention aims to solve

[0009] Remote operation of a mobile vehicle is performed through communication. While there is a fixed communication delay in the communication used for remote operation of a mobile vehicle, this delay can potentially become so large that it cannot maintain normal remote driving.

[0010] The technology disclosed in Patent Document 1 does not take into account the possibility of communication delays large enough to prevent the maintenance of normal remote driving in communications used for remote operation of a mobile body. Therefore, the technology disclosed in Patent Document 1 still does not solve the aforementioned problem.

[0011] The present invention was made to solve the above-mentioned problems, and its purpose is to provide a situation in which the communication delay in the communication used for remote operation of the mobile body becomes so large that it cannot maintain the normal operation of remote driving.

[0012] Technical means for solving technical problems

[0013] The remote operation device involved in this invention is a remote operation device capable of remotely operating a mobile body via communication. It is characterized by comprising: an acquisition unit that acquires information related to other remote operation devices capable of remotely operating the mobile body via communication; a delay time detection unit that detects communication delay time in communication with the mobile body; a delay time determination unit that determines whether the communication delay time detected by the delay time detection unit exceeds a delay time determination threshold; a switching target determination unit that, when the delay time determination unit determines that the communication delay time exceeds the delay time determination threshold, determines a switching target remote operation device from among the other remote operation devices based on the information related to other remote operation devices acquired by the acquisition unit and according to whether a switching determination condition is met; and a switching control unit that outputs switching instruction information to the switching target remote operation device determined by the switching target determination unit, enabling it to remotely operate the mobile body.

[0014] Invention Effects

[0015] According to the present invention, it is possible to prevent a situation where the communication delay present in the communication used for remote operation of the mobile body becomes so large that it cannot maintain the state of normal remote driving. Attached Figure Description

[0016] Figure 1 This is a diagram illustrating a structural example of a remote operating system that includes the remote operating device according to Embodiment 1.

[0017] Figure 2 This is a diagram illustrating a structural example of the remote operation device according to Embodiment 1.

[0018] Figure 3 is a diagram illustrating the information acquired by the acquisition unit of the remote operating device in the remote operating system according to Embodiment 1. Figure 3A This diagram illustrates the information acquired by the acquisition unit in the state prior to the switching of the remote operation device. Figure 3B It is used to explain from Figure 3A The diagram shown illustrates the information acquired by the acquisition unit when the state changes to the state of the remote operation device 1. Figure 3C It is used to explain from Figure 3B The diagram shown illustrates the information acquired by the acquisition unit in a state where the remote operation device has been switched.

[0019] Figure 4 This is a diagram showing the image of the remote operating device for switching targets in Embodiment 1, where the target switching determination unit determines whether the switching determination conditions are met.

[0020] Figure 5This is a diagram illustrating an example of the process in Embodiment 1 where the switching control unit determines that the switching timing has been reached based on the differential information output from the differential calculation unit.

[0021] Figure 6 This is a flowchart illustrating the operation of the remote operating device according to Embodiment 1, which is used to explain the operation of the remote operating device as a switching source.

[0022] Figure 7 This is used to explain the implementation method 1. Figure 6 Detailed flowchart of the steps in ST505

[0023] Figure 8 This is used to illustrate that in implementation method 1, such as Figure 7 This is an example diagram of an image showing the switching process when switching a remote control device for remotely operating a vehicle, as illustrated in the flowchart.

[0024] Figure 9 This is a flowchart illustrating the operation of the remote operating device according to Embodiment 1, which is used as a candidate remote operating device.

[0025] Figure 10 It is a timing diagram showing the actions in Embodiment 1, from the time the switching source remote operating device determines the switching target remote operating device until the switching target remote operating device is switched to the remote operating device of the remote operating vehicle.

[0026] Figure 11 This is an example diagram illustrating, in Embodiment 1, the switching of a remote control device for remotely operating a vehicle based on the communication delay time in communication with the vehicle.

[0027] Figure 12A , Figure 12B This is a diagram illustrating an example of the hardware structure of the remote operation device according to Embodiment 1. Detailed Implementation

[0028] Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings.

[0029] Implementation method 1.

[0030] Figure 1 This is a diagram illustrating a structural example of a remote operating system including the remote operating device 1 according to Embodiment 1.

[0031] A remote operating system is a system in which an operator (hereinafter referred to as "remote operator") remotely operates vehicle 2 as needed and is able to provide services such as driving vehicle 2 to its destination.

[0032] The remote operating system includes vehicle 2 and remote operating device 1. Although Figure 1 Only one vehicle 2 is shown, but this is just an example; the remote operating system may include more than one vehicle 2. Furthermore, although... Figure 1 Three remote operating devices 1 are shown, but this is only an example. The remote operating system includes multiple remote operating devices 1. The multiple remote operating devices 1 can be connected to a vehicle 2. In the following embodiment 1, the multiple remote operating devices 1 are also simply referred to as "remote operating devices 1".

[0033] The remote operating device 1 can remotely operate the vehicle 2 via communication. Specifically, the remote operator operates the vehicle 2 remotely by operating the remote operating device 1, and remotely drives the vehicle 2. The technology for remotely operating the vehicle 2 via communication is existing technology, therefore detailed description is omitted. The remote operating device 1 can use existing remote operating technology to remotely operate the vehicle 2. The remote operating device 1 is located in a suitable location, such as an operation center, along the driving path of the vehicle 2. However, for a single vehicle 2, only one remote operating device 1 is used for remote operation of that vehicle 2 at a time.

[0034] In the remote operating system, it is appropriately determined which of the multiple remote operating devices 1 will initially remotely operate the vehicle 2. For example, the remote operating device 1 located closest to the current position of the vehicle 2 may be the first to remotely operate the vehicle 2, or if there are passengers in the vehicle 2, the passengers may specify the remote operating device 1 to remotely operate the vehicle 2.

[0035] Furthermore, in the remote operating system, multiple remote operating devices 1 capable of communicating with vehicle 2 can identify which remote operating device 1 is remotely operating vehicle 2 when deciding to remotely operate vehicle 2. Specifically, for example, when deciding to remotely operate vehicle 2, the determined remote operating device 1 notifies other remote operating devices 1 capable of communicating with vehicle 2 of its intention to remotely operate vehicle 2. The other remote operating devices 1 then know which remote operating device 1 is in the process of remotely operating vehicle 2 through this notification. The remote operating devices 1 are interconnected via a network.

[0036] When remote operating device 1 determines that a communication delay has occurred between remote operating device 1 and vehicle 2 during remote operation of vehicle 2, requiring switching of remote operating device 1, it switches to another remote operating device 1 to remotely operate vehicle 2. In embodiment 1, the communication delay requiring switching of remote operating device 1 refers to a communication delay to the extent that the remote driving of vehicle 2 cannot be maintained normally.

[0037] When switching remote operating device 1, the other remote operating device 1 remotely operates vehicle 2. Specifically, the remote operator of the other remote operating device 1 operates the other remote operating device 1, thereby remotely operating vehicle 2.

[0038] The remote operating device 1 used in the process of remotely operating vehicle 2 is also referred to as a "switching source remote operating device". Furthermore, in Embodiment 1 below, when the "switching source remote operating device" cannot maintain the normal operation of remote driving of vehicle 2, one or more other remote operating devices 1 that can switch from the "switching source remote operating device" to become the remote operating device 1 for remotely operating vehicle 2 are also referred to as "standby remote operating devices". Additionally, in Embodiment 1 below, the standby remote operating device that switches from the "switching source remote operating device" to remotely operate vehicle 2 is also referred to as a "switching target remote operating device".

[0039] In addition, when there are multiple candidate remote operating devices, the switching source remote operating device decides which candidate remote operating device to use as the switching target remote operating device.

[0040] Figure 2 This is a diagram illustrating a structural example of the remote operation device 1 according to Embodiment 1.

[0041] In Implementation 1, the remote operation device 1 has the same structure in principle whether it is a switching source remote operation device or a backup remote operation device.

[0042] The remote operating device 1 is connected to the vehicle 2 and other remote operating devices 1 via a communication unit (not shown).

[0043] The remote operation device 1 includes an acquisition unit 11, a delay time detection unit 12, a switching control unit 13, an operation control unit 14, a detection unit 15, and an output unit 16.

[0044] The acquisition unit 11 includes a device information acquisition unit 111, an operator information acquisition unit 112, a vehicle information acquisition unit 113, a preparation operation information acquisition unit 114, and a switching control information acquisition unit 115.

[0045] The switching control unit 13 includes a delay time determination unit 131, a switching target determination unit 132, and a differential calculation unit 133.

[0046] The detection unit 15 includes a device information detection unit 151 and an operator information detection unit 152.

[0047] The output unit 16 includes a device information output unit 161, an operator information output unit 162, a switching control information output unit 163, an operation information output unit 164, and a preparation operation information output unit 165.

[0048] Department 11 acquires various types of information.

[0049] For example, when remote operating device 1 is a switching source remote operating device, acquisition unit 11 acquires information related to the candidate remote operating device from the candidate remote operating device. When remote operating device 1 is a switching source remote operating device, the information related to the candidate remote operating device acquired by acquisition unit 11 includes information related to the status of the candidate remote operating device (hereinafter referred to as "device status information") or information related to the status of the remote operator operating the candidate remote operating device (hereinafter referred to as "operator status information"). Specifically, device information acquisition unit 111 of acquisition unit 11 acquires device status information, and operator information acquisition unit 112 of acquisition unit 11 acquires operator status information.

[0050] The device status information includes, for example, information related to the communication delay time in the communication between the remote operating device 1 and the vehicle 2, information on whether the remote operating device 1 is operated by a remote operator, or information on whether the remote operating device 1 is in a standby state. In Embodiment 1, the term "remote operating device 1 in a standby state" means, for example, that the remote operating device 1 is in a state where it can operate at any time as long as there is a remote operator capable of operating the remote operating device 1.

[0051] The device information acquisition unit 111 can acquire the aforementioned device status information of the candidate remote operating device from the candidate remote operating device using an appropriate method. For example, the device information acquisition unit 111 can request device status information from the candidate remote operating device at a preset period and acquire the device status information output from the candidate remote operating device according to the request. Alternatively, for example, the candidate remote operating device can output device status information when its status changes, and the device information acquisition unit 111 can acquire the device status information output by the candidate remote operating device.

[0052] In addition, the backup remote operating device detects the aforementioned device status information. Specifically, in the backup remote operating device, the device information detection unit 151 of the detection unit 15 detects the device status information. Details of the device information detection unit 151 will be described later.

[0053] The operator status information acquired by the operator information acquisition unit 112 includes, for example, information on whether the remote operator operating the remote operating device 1 is seated facing the remote operating device 1, information on whether the remote operator operating the remote operating device 1 is qualified to remotely operate the vehicle 2, or information related to the health status of the remote operator operating the remote operating device 1.

[0054] The operator information acquisition unit 112 can acquire the aforementioned device status information related to the remote operator of the candidate remote operating device from the candidate remote operating device using an appropriate method. For example, the operator information acquisition unit 112 can request operator status information from the candidate remote operating device at a preset period and acquire the operator status information output from the candidate remote operating device according to the request. Alternatively, for example, the candidate remote operating device can output operator status information when the status of the remote operator of the candidate remote operating device changes, and the operator information acquisition unit 112 can acquire the operator status information output by the candidate remote operating device.

[0055] In addition, the backup remote operating device detects the aforementioned operator status information. Specifically, in the backup remote operating device, the operator information detection unit 152 of the detection unit 15 detects the operator status information. Details of the operator information detection unit 152 will be described later.

[0056] Additionally, for example, the acquisition unit 11 acquires information related to vehicle 2 (hereinafter referred to as "vehicle information") from vehicle 2. Specifically, the vehicle information acquisition unit 113 of the acquisition unit 11 acquires vehicle information. The vehicle information includes information related to the vehicle 2's travel speed, distance to other vehicles 2, current position of vehicle 2, destination of vehicle 2, or images of the surrounding area of ​​vehicle 2 captured by a camera mounted on vehicle 2. Furthermore, the vehicle information acquisition unit 113 functions whether the remote operation device 1 is a switching source remote operation device or a backup remote operation device.

[0057] Additionally, for example, when the remote operating device 1 is a switching source remote operating device, the acquisition unit 11 acquires preparation operation information from the switching target remote operating device. Specifically, when the remote operating device 1 is a switching source remote operating device, the preparation operation information acquisition unit 114 of the acquisition unit 11 acquires preparation operation information from the switching target remote operating device. The preparation operation information is operation information that is generated in the switching target remote operating device based on the switching preparation instruction information output by the switching source remote operating device when the remote operating device 1 of the remote operating device 2 of the remote operating device 2 is switched from the switching source remote operating device to the switching target remote operating device, and is used for remotely operating the vehicle 2. Details regarding the operation information will be explained later.

[0058] Additionally, for example, when the remote operating device 1 is a switching source remote operating device, the acquisition unit 11 acquires switching control information from the switching source remote operating device. Specifically, when the remote operating device 1 is a switching source remote operating device, the switching control information acquisition unit 115 of the acquisition unit 11 acquires switching control information from the switching source remote operating device. The switching control information is information output from the switching source remote operating device when switching is performed on the remote operating device 1 of the remote operating device 2. In Embodiment 1, the switching control information includes switching preparation instruction information that instructs the target remote operating device to prepare for remotely operating the vehicle 2; and switching instruction information that instructs to perform a switch so as to actually remotely operate the vehicle 2 as the switching source remote operating device.

[0059] Here, FIG3 is a diagram illustrating the information acquired by the acquisition unit of the remote operating device 1 in the remote operating system according to Embodiment 1. In FIG3, as an example, the remote operating system includes three remote operating devices 1: remote operating device A (1A), remote operating device B (1B), and remote operating device C (1C). Figure 3A This indicates that remote operating device A (1A) is the switching source remote operating device, and remote operating device B (1B) and remote operating device C (1C) are candidate remote operating devices and are in a state before the switching occurs. Figure 3B Indicates from Figure 3A The state shown indicates a switch of remote operating device 1, and remote operating device A (1A), which is the source remote operating device, determines remote operating device B (1B) as the target remote operating device for the switch. Additionally, in Figure 3B For ease of explanation, the diagram of the remote operating device C (1C) is omitted. Figure 3C Indicates from Figure 3B The state shown is the state where the remote operating device 1 has been switched and the remote operating device B (1B) is used as the switching source to remotely operate the vehicle 2.

[0060] exist Figure 3A In the state shown, the acquisition units 11 of remote operating devices A (1A) to C (1C) acquire vehicle information from vehicle 2, respectively. In addition, the acquisition unit 11 of remote operating device A (1A), which is a switching source remote operating device, acquires device status information and operator status information from remote operating devices B (1B) and C (1C), which are candidate remote operating devices.

[0061] exist Figure 3B In the shown state, the acquisition units 11 of remote operating devices A (1A) and B (1B) respectively acquire vehicle information from vehicle 2. Furthermore, the acquisition unit 11 of remote operating device B (1B), which is the target remote operating device, acquires switching control information from remote operating device A (1A), which is the source remote operating device. Additionally, the acquisition unit 11 of remote operating device A (1A), which is the source remote operating device, acquires preparation operation control information from remote operating device B (1B), which is the target remote operating device. Furthermore, in Figure 3B In the state shown, until the switch is completed, the acquisition unit 11 of the remote operating device A (1A) acquires device status information and operator status information from the remote operating devices B (1B) and C (1C). Figure 3B For ease of explanation, illustrations of the device status information and operator status information have been omitted.

[0062] exist Figure 3C In the state shown, the acquisition units 11 of remote operating devices A (1A) to C (1C) acquire vehicle information from vehicle 2, respectively. In addition, the acquisition unit 11 of remote operating device B (1B), which is a switching source remote operating device, acquires device status information and operator status information from remote operating devices A (1A) and C (1C), which are candidate remote operating devices.

[0063] Additionally, although the illustration is omitted in Figure 3, the remote operation device, which is the switching source for remote operation of the vehicle, sends operation information for remotely operating the vehicle 2 to the vehicle 2.

[0064] The acquisition unit 11 outputs the acquired information to the delay time detection unit 12, the switching control unit 13, and the operation control unit 14.

[0065] The delay detection unit 12 detects the communication delay time in the communication between the remote operating device 1 and the vehicle 2. Furthermore, the delay detection unit 12 functions whether the remote operating device 1 is the switching source remote operating device or the backup remote operating device.

[0066] The delay time detection unit 12 detects the communication delay time, for example, based on one-way information sent from vehicle 2 to remote operating device 1. Specifically, for example, the delay time detection unit 12 obtains the time measured in vehicle 2 (hereinafter referred to as "vehicle time") from vehicle 2, and sets the difference between the vehicle time and the time when the vehicle time is obtained through remote operating device 1 (hereinafter referred to as "device time") as the communication delay time. For example, the vehicle time information is included in vehicle-related information and can be obtained by vehicle information acquisition unit 113. The delay time detection unit 12 sets the time when vehicle information acquisition unit 113 obtains the vehicle time as the device time.

[0067] In addition, the remote operating device 1 and the vehicle 2 are equipped with clocks that can measure high-precision times such as GPS (Global Positioning System) time.

[0068] The communication delay detection method described above by the delay detection unit 12 is merely an example. The delay detection unit 12 only needs to detect a communication delay in the communication between the remote operating device 1 and the vehicle 2 that is sufficient to prevent the remote driving of the vehicle 2 from proceeding normally.

[0069] The delay time detection unit 12 outputs information related to the detected communication delay time to the switching control unit 13 and the detection unit 15.

[0070] When the remote operating device 1 is the switching source remote operating device, the switching control unit 13 determines whether to switch the remote operating device 1 that remotely operates the vehicle 2. If the switch is determined to be performed, the switching target remote operating device is selected from the candidate remote operating devices, and the switching control information for remotely operating the vehicle 2 is output to the selected switching target remote operating device.

[0071] Specifically, the delay time determination unit 131 of the switching control unit 13 determines whether the communication delay time detected by the delay time detection unit 12 exceeds a preset threshold (hereinafter referred to as "delay time determination threshold").

[0072] When the communication delay time exceeds the delay time determination threshold, the delay time determination unit 131 determines that the remote operation device 1 for remote operation of the vehicle 2 should be switched, and outputs delay existence information indicating that the communication delay time exceeds the delay time determination threshold to the switching target determination unit 132 of the switching control unit 13.

[0073] When the delay time determination unit 131 determines that the communication delay time exceeds the delay time determination threshold, the switching target determination unit 132 determines the switching target remote operating device from the candidate remote operating devices based on the information related to the candidate remote operating devices acquired by the acquisition unit 11, according to whether the switching determination conditions are met.

[0074] Pre-set switching criteria. Below are some examples to illustrate the specifics of these switching criteria.

[0075] <Example 1 of switching judgment conditions>

[0076] For example, the switching determination condition is that the communication delay between the candidate remote operating device and vehicle 2 is below a preset threshold (hereinafter referred to as the "other device delay time determination threshold"). The other device delay time determination threshold has a preset value below the delay time determination threshold. The other device delay time determination threshold can be the same value as the delay time determination threshold.

[0077] The target switching decision unit 132 determines the communication delay time between the candidate remote operation device and the vehicle 2 based on the device status information obtained by the device information acquisition unit 111.

[0078] The switching target determination unit 132 determines the candidate remote operating device as the switching target remote operating device if the communication delay time between the candidate remote operating device and the vehicle 2 is below the threshold for determining the delay time of other devices.

[0079] <Example 2 of switching judgment conditions>

[0080] For example, the switching determination condition is that the remote operator operating the candidate remote operating device is in a state where he / she is able to operate the candidate remote operating device, and the communication delay time between the candidate remote operating device and vehicle 2 is below the threshold used for determining the delay time of other devices.

[0081] The target switching determination unit 132 can determine the status of the remote operator operating the candidate remote operating device based on the operator status information obtained by the operator information acquisition unit 112. The status of the remote operator operating the candidate remote operating device is the status of being able to operate the candidate remote operating device, which refers to, for example, the status of (1) to (3) below.

[0082] (1) The remote operator is seated facing the backup remote operating device.

[0083] (2) The status that the remote operator is qualified to remotely operate vehicle 2.

[0084] (3) The health status of vehicle 2, which allows the remote operator to operate it normally.

[0085] The health status of a remote operator being able to remotely operate vehicle 2 normally refers to, for example, a state of not being under the influence of alcohol, a state of sobriety of the remote operator being above a predetermined level, or a state of proficiency of the remote operator in remotely operating vehicle 2 being above a predetermined level.

[0086] The switching target determination unit 132 determines the switching target remote operating device as the one whose status of all remote operators operating the candidate remote operating device meets the above (1) to (3) and whose communication delay time between the candidate remote operating device and the vehicle 2 is below the threshold for determining the delay time of other devices.

[0087] Furthermore, (1) to (3) above are merely examples. States other than (1) to (3) above can also be defined when the remote operator is able to operate the backup remote operating device. When the remote operator is able to operate the backup remote operating device, the state of the remote operator can be defined. As the state of the remote operator, when the remote operator in this state operates the remote operating device 1, the vehicle 2 can be remotely driven normally.

[0088] Here, Figure 4 This is a diagram illustrating how, in Embodiment 1, the switching target determination unit 132 determines the switching target remote operating device based on whether the switching determination conditions are met. Figure 4 In the example above, the switching condition is <Switching Condition Example 2>. Figure 4 In this system, there are four candidate remote operating devices, and the states of the four candidate remote operating devices and the states of the remote operators operating the four candidate remote operating devices are represented by 301 to 304. The target switching determination unit 132 determines the target remote operating device to switch from the four candidate remote operating devices.

[0089] exist Figure 4 In the state shown in 301, the remote operator of the candidate remote operating device is not seated. That is to say, the remote operator of the candidate remote operating device does not meet the above (1).

[0090] exist Figure 4 In the state shown in 302, the remote operator is not qualified to remotely operate vehicle 2. That is, the remote operator of the candidate remote operation device does not meet the above (2).

[0091] exist Figure 4In the state shown in 303, the remote operator of the candidate remote operating device is seated, qualified to remotely operate vehicle 2, and in good health. That is, the remote operator of the candidate remote operating device meets all the conditions (1) to (3) above. However, the communication delay time between the candidate remote operating device and vehicle 2 exceeds the threshold for determining the delay time of other devices.

[0092] Therefore, if the status of the candidate remote operating device and the status of the remote operator operating the candidate remote operating device are as shown in 301 to 303, the switching target determination unit 132 determines that the candidate remote operating device does not meet the above-mentioned <Switching Determination Condition Example 2>.

[0093] On the other hand, Figure 4 In the state shown in section 304, the candidate remote operating device satisfies all of the above (1) to (3), and the communication delay time between the candidate remote operating device and vehicle 2 is below the threshold for determining the delay time of other devices. That is, the candidate remote operating device satisfies the above <Switching Determination Condition Example 2>.

[0094] Therefore, the target switching determination unit 132 determines the candidate remote operating device in the state shown in 304 as the target remote operating device.

[0095] When there are multiple candidate remote operating devices that meet the switching determination conditions, the switching target determination unit 132 can select one switching target remote operating device from the candidate remote operating devices that meet the switching determination conditions according to appropriate determination conditions.

[0096] For example, suppose there are multiple candidate remote operating devices that meet the switching determination condition <Switching Determination Condition Example 1> described above and whose communication delay time with vehicle 2 is below the threshold for determining the delay time of other devices. In this case, the switching target determination unit 132, for example, determines the candidate remote operating device with the shortest communication delay time between the candidate remote operating device and vehicle 2 as the switching target remote operating device.

[0097] Alternatively, for example, the target selection unit 132 may select the candidate remote operating device that is located on the path to the destination of vehicle 2 and is closest to the current location as the target remote operating device. Furthermore, the target selection unit 132 can obtain information about the path to the destination of vehicle 2 and the current location from the vehicle information acquisition unit 113. Additionally, the target selection unit 132 can obtain information related to the location where the candidate remote operating device exists from the device information acquisition unit 111. The device information acquisition unit 111 obtains the location information of the candidate remote operating device as device status information.

[0098] Alternatively, the switching target determination unit 132 can also determine the candidate remote operating device whose communication delay time is below the threshold for determining delay time of other devices for the longest duration as the switching target remote operating device. The switching target determination unit 132 can obtain information related to the aforementioned duration from the device information acquisition unit 111. The device information acquisition unit 111 obtains the aforementioned duration-related information as device status information.

[0099] Furthermore, if no candidate remote operating device meets the handover determination criteria, the handover target determination unit 132 does not determine the handover target remote operating device. In this case, the handover source remote operating device generates operation information to stop the vehicle 2 and outputs the generated operation information to the vehicle 2.

[0100] When deciding on the target remote operating device, the target switching unit 132 outputs information related to the determined target remote operating device to the switching control unit 13.

[0101] When the switching control unit 13 outputs information related to the switching target remote operating device from the switching target determination unit 132, it outputs switching preparation instruction information to the switching target remote operating device. Additionally, at this time, the switching control unit 13 provides vehicle information and information that can determine the switching source remote operating device to the switching preparation instruction information and outputs it to the switching target remote operating device.

[0102] In the remote operating device for the target of the handover, handover preparation instruction information is acquired and preparation operation information is generated. Specifically, in the remote operating device for the target of the handover, the operation control unit 14 acquires the handover preparation instruction information and generates preparation operation information. Details of the operation control unit 14 will be described later.

[0103] In Implementation 1, as described above, the operation information generated by the remote operation device for switching targets upon receiving the switching preparation instruction information is referred to as "preparation operation information".

[0104] The differential calculation unit 133 of the switching control unit 13 calculates the difference between the operation information generated by the operation control unit 14 in the switching source remote operation device and the preparation operation information generated by the switching target remote operation device. Additionally, the preparation operation information acquisition unit 114 acquires the preparation operation information generated by the switching target remote operation device.

[0105] Here, the operation information and preparation information will be explained in more detail.

[0106] As described above, the operation information is information used for remotely operating vehicle 2. The operation information includes one or more types of information used for remotely operating vehicle 2. Specifically, for example, the operation information includes information related to the speed of vehicle 2, the control amount of the steering wheel angle, or the control amount of the throttle opening.

[0107] The operation control unit 14 generates operation information. For example, the operation control unit 14 generates operation information based on vehicle information acquired by the vehicle information acquisition unit 113.

[0108] When the operation information generated by the operation control unit 14 is output to the vehicle 2, the vehicle 2 performs remote driving based on the operation information.

[0109] The preparation operation information refers to the operation information generated by the operation control unit 14 when the switching control information acquisition unit 115 of the target remote operation device obtains the switching preparation instruction information from the source remote operation device. The preparation operation information is also used for remotely operating the vehicle 2, but it is not actually output to the vehicle 2. When generating the preparation operation information, the operation control unit 14 provides information that it knows is preparation operation information in advance.

[0110] When the remote operating device 1 is the target remote operating device for switching, the operation control unit 14 generates preparation operation information based on the vehicle information provided with the switching preparation instruction information.

[0111] Return to the explanation of the difference calculation section 133.

[0112] For example, the operation information and preparation operation information include information related to the steering wheel angle and information related to the speed of vehicle 2.

[0113] The differential calculation unit 133 calculates the difference between the information related to the steering wheel angle contained in the operation information and the information related to the steering wheel angle contained in the preparation operation information, as well as the difference between the information related to the speed of the vehicle 2 contained in the operation information and the information related to the speed of the vehicle 2 contained in the preparation operation information.

[0114] The differential calculation unit 133 outputs information related to the difference between the calculated operation information and the preparation operation information (hereinafter referred to as "differential information") to the switching control unit 13. In the above example, the differential calculation unit 133 outputs the differential information between the information related to the steering wheel angle contained in the operation information and the information related to the steering wheel angle contained in the preparation operation information, as well as the differential information between the information related to the speed of vehicle 2 contained in the operation information and the information related to the speed of vehicle 2 contained in the preparation operation information, to the switching control unit 13.

[0115] When differential information is output from the differential calculation unit 133, the switching control unit 13 determines whether the difference between the operation information and the preparation operation information is within a preset threshold (hereinafter referred to as the "differential determination threshold"). The differential determination threshold is set with values ​​that are allowed as the difference between the operation information and the preparation operation information. Specifically, the differential determination threshold is set with a difference value between the operation information and the preparation operation information, which is, for example, the degree to which, when there is a passenger in the vehicle 2, the passenger can continue to ride without any sense of inconvenience even when switching from the state of remotely operating the vehicle 2 based on the operation information to the state of remotely operating the vehicle 2 based on the preparation operation information.

[0116] Furthermore, thresholds for differential determination are set according to each type of information contained in the operation information or preparation operation information. In the example above, thresholds for differential determination are set separately for information related to the steering wheel angle and information related to the speed of vehicle 2.

[0117] When it is determined that the difference between the operation information and the preparation operation information is within the difference determination threshold, the switching control unit 13 determines whether the state in which the difference between the operation information and the preparation operation information is within the difference determination threshold continues for a preset time (hereinafter referred to as "difference duration determination time"). When difference information is output from the difference calculation unit 133, the switching control unit 13 stores the difference information in a storage unit (not shown) in association with the output date and time of the difference information. The switching control unit 13 refers to the storage unit and determines whether the state in which the difference between the operation information and the preparation operation information is within the difference determination threshold continues for the difference duration determination time.

[0118] If the difference between the operation information and the preparation operation information remains within the difference determination threshold for a period of time, the switching control unit 13 in the switching source remote operation device determines that the preparation operation information generated in the switching target remote operation device can be used as operation information for actually performing remote operation on the vehicle 2. In other words, the switching control unit 13 determines that the switching timing for switching the remote operation device 1 that can perform remote operation on the vehicle 2 from the switching source remote operation device to the switching target remote operation device has been reached.

[0119] In the remote operation device for switching sources, if the switching control unit 13 determines that the switching time has been reached, it outputs switching instruction information to the remote operation device for switching targets, and does not cause the operation control unit 14 of the remote operation device for switching sources to generate operation information. Furthermore, the switching control unit 13 outputs the switching instruction information to the remote operation device for switching targets via the output unit 16 of the remote operation device for switching sources. The switching control unit 13 also provides vehicle information in the switching command information.

[0120] In the target remote operating device that has acquired the switching instruction information output by the switching source remote operating device, vehicle information is obtained from vehicle 2, and the operation control unit 14 generates operation information based on this vehicle information. Thereafter, the target remote operating device switches to the switching source remote operating device and outputs the generated operation information to vehicle 2 for remote operation of vehicle 2. This performs the switching of remote operating device 1.

[0121] Figure 5 This is a diagram illustrating an example of the process by which the switching control unit 13 determines that the switching timing has been reached based on the differential information output from the differential calculation unit 133 in Embodiment 1.

[0122] exist Figure 5 As in the example above, the operation information and preparation operation information in the image shown contain information related to the control amount of the steering wheel angle and the speed of vehicle 2.

[0123] exist Figure 5 In the diagram, 401 and 404 represent temporal transfers of operational information currently used for remote operation of vehicle 2, generated by the switching source remote operation device. Specifically, 401 represents a temporal transfer of the control amount of the steering wheel angle, and 404 represents a temporal transfer of information related to the speed of vehicle 2.

[0124] exist Figure 5 In the diagram, 402 and 405 represent time-varying operational information generated by the switching source remote operating device that is not used for remote operation of vehicle 2. Specifically, 402 represents a time-varying control amount of the steering wheel angle, and 405 represents a time-varying information related to the speed of vehicle 2.

[0125] Figure 5 In this context, 403 represents the differential information between the control quantity of the steering wheel angle shown in 401 and the control quantity of the steering wheel angle shown in 402, which are output from the differential calculation unit 133. 406 represents the differential information between the information related to the speed of the vehicle 2 shown in 403 and the information related to the speed of the vehicle 2 shown in 404, which are output from the differential calculation unit 133.

[0126] exist Figure 5 In the process, the difference in the control amount of the steering wheel angle is within the threshold for difference judgment within the time shown in A and B, but the state of the difference in the control amount of the steering wheel angle being within the threshold for difference judgment does not continue for the duration of difference judgment.

[0127] In addition, the speed difference of vehicle 2 is within the threshold for difference determination within the time periods shown in D and E, but the state in which the speed difference of vehicle 2 is within the threshold for difference determination does not continue for the duration of difference determination.

[0128] During the time period shown in C, the difference in the control amount of the steering wheel angle becomes the differential determination threshold, and the state in which the difference in the control amount of the steering wheel angle is within the differential determination threshold continues for the differential persistence determination time. However, at the moment when the time shown in C has elapsed, the state in which the difference in the speed of vehicle 2 is within the differential determination threshold has not yet continued for the differential persistence determination time. Therefore, at the moment when the time shown in C has elapsed, the switching control unit 13 does not determine that the switching timing has been reached.

[0129] At the moment indicated by F, the difference between the control quantity of the steering wheel angle and the difference between the speed of vehicle 2 are both within the difference determination threshold and have remained within the difference determination duration. When the time indicated by F has elapsed, the switching control unit 13 determines that the switching timing has been reached.

[0130] Back Figure 2 Explanation.

[0131] The operation control unit 14 generates the above-mentioned operation information or the above-mentioned preparation operation information.

[0132] When remote operating device 1 is a switching source remote operating device, the operation control unit 14 generates operation information. When remote operating device 1 is a switching target remote operating device, the operation control unit 14 generates preparation operation information.

[0133] The operation control unit 14 outputs the generated operation information to the vehicle 2 via the output unit 16. However, if the generated operation information is preparation operation information, the operation control unit 14 outputs the preparation operation information to the switching source remote operation device via the output unit 16.

[0134] As described above, when remote operating device 1 is determined to be the target remote operating device for switching, preparation operation information is generated by the target remote operating device for switching and is not output to vehicle 2 until switching instruction information is output from the source remote operating device for switching. Since the preparation operation information is not output to vehicle 2, vehicle 2 is not actually remotely operated by the preparation operation information.

[0135] When remote operating device 1 is a candidate remote operating device, the detection unit 15 detects information related to remote operating device 1. Information related to remote operating device 1 may include device status information or operator status information.

[0136] Device information of detection unit 15: Detection device status information of detection unit 151.

[0137] As described above, the device status information includes, for example, information on whether the remote operating device 1 is operated by a remote operator, information on whether the remote operating device 1 is in standby mode, or information related to communication delay time. When the remote operating device 1 is a candidate remote operating device, the device information detection unit 151 can detect whether the candidate remote operating device is operated by a remote operator, for example, by obtaining information about the steering wheel angle from a steering wheel angle sensor (not shown). Alternatively, the device information detection unit 151 can detect whether the candidate remote operating device is in standby mode, for example, by detecting whether the candidate remote operating device is connected to the vehicle 2 via communication. Furthermore, the device information detection unit 151 can obtain the communication delay time from the delay time detection unit 12.

[0138] Furthermore, the above-described method for detecting device status information is merely an example. The device information detection unit 151 detects device status information using an appropriate method.

[0139] The device information detection unit 151 outputs device status information to the switching source remote operation device via the output unit 16.

[0140] The device information detection unit 151 can detect device status information at a preset period, or it can detect device status information when requested from the remote operation device of the switching source. Furthermore, the device information detection unit 151 can output device status information to the remote operation device of the switching source only when the device status information changes.

[0141] The switching source remote operating device acquires device status information output from the device information detection unit 151 of the candidate remote operating device. Specifically, in the switching source remote operating device, the device information acquisition unit 111 acquires the device status information output by the device information detection unit 151.

[0142] Operator information of inspection department 15; inspection department 152 inspection operator status information.

[0143] As described above, the operator status information specifically includes, for example, information on whether the remote operator operating the remote operating device 1 is seated facing the remote operating device 1, information on whether the remote operator operating the remote operating device 1 is qualified to remotely operate the vehicle 2, or information related to the health status of the remote operator operating the remote operating device 1.

[0144] When remote operating device 1 is a candidate remote operating device, for example, operator information detection unit 152 can detect whether the remote operator is seated facing the candidate remote operating device by detecting that the OK button has been pressed. For example, operator information detection unit 152 can detect whether the remote operator is qualified to remotely operate vehicle 2 by acquiring a photographic image of the remote operator taken by a photographic device (not shown) and comparing the remote operator information extracted from the acquired photographic image with the operator information pre-registered in an authentication server (not shown). Operator information detection unit 152 can extract the remote operator information from the photographic image using known image recognition technology.

[0145] Operator information detection unit 152 can detect the operator's health status from captured images, for example, using known image recognition technology. Operator information detection unit 152 can use known techniques (for example, see Japanese Patent Application Laid-Open No. 2018-077649) to determine the remote operator's level of alertness or proficiency, and detect the determined level of alertness or proficiency as the remote operator's health status. Furthermore, for example, operator information detection unit 152 can use known techniques to detect whether the remote operator is intoxicated.

[0146] Furthermore, the above-described method for detecting operator status information is merely an example. The operator information detection unit 152 detects operator status information using an appropriate method.

[0147] The operator information detection unit 152 outputs the operator status information to the switching source remote operation device via the output unit 16.

[0148] The operator information detection unit 152 can detect remote operator status information at a preset period, or it can detect operator status information when requested from the remote operating device of the switching source. Furthermore, the operator information detection unit 152 can output operator status information to the remote operating device of the switching source only when the operator status information changes.

[0149] The switching source remote operating device acquires operator status information output from the operator information detection unit 152 of the candidate remote operating device. Specifically, in the switching source remote operating device, the operator information acquisition unit 112 acquires the operator status information output by the operator information detection unit 152.

[0150] Output section 16 acquires various information.

[0151] When the remote operating device 1 is a backup remote operating device, the device information output unit 161 of the output unit 16 outputs the device status information output by the device information detection unit 151 to the switching source remote operating device.

[0152] When the remote operating device 1 is a backup remote operating device, the operator information output unit 162 of the output unit 16 outputs the operator status information output by the operator information detection unit 152 to the switching source remote operating device.

[0153] When the remote operating device 1 is a backup remote operating device, the preparation operation information output unit 165 of the output unit 16 outputs the preparation operation information generated by the operation control unit 14 to the switching source remote operating device. Furthermore, since the preparation operation information provides information indicating that it is preparation operation information, the preparation operation information output unit 165 can determine, based on this information, that the operation information output from the operation control unit 14 is preparation operation information.

[0154] When the remote operating device 1 is a switching source remote operating device, the switching control information output unit 163 of the output unit 16 outputs the switching preparation instruction information or switching instruction information output by the switching control unit 13 to the switching target remote operating device.

[0155] When the remote operating device 1 is a switching source remote operating device, the operation information output unit 164 of the output unit 16 outputs the operation information generated by the operation control unit 14 to the vehicle 2.

[0156] The operation of the remote operation device 1 according to Embodiment 1 will be described.

[0157] In the following description of the actions, flowcharts are used to illustrate the actions when remote operating device 1 is a switching source remote operating device, and the actions when remote operating device 1 is a remote operating device other than the switching source remote operating device.

[0158] Figure 6 This is a flowchart illustrating the operation of the remote operating device 1 according to Embodiment 1, which is used to explain the operation of the remote operating device as a switching source.

[0159] The delay time detection unit 12 detects the communication delay time in the communication between the switching source remote operation device and the vehicle 2 (step ST501).

[0160] The delay time detection unit 12 outputs information related to the detected communication delay time to the switching control unit 13 and the detection unit 15.

[0161] Department 11 acquires various types of information.

[0162] Specifically, for example, the device information acquisition unit 111 of the acquisition unit 11 acquires device status information from the candidate remote operating device, and the operator information acquisition unit 112 of the acquisition unit 11 acquires operator status information from the candidate remote operating device (step ST502).

[0163] The acquisition unit 11 outputs the acquired information to the delay time detection unit 12, the switching control unit 13, and the operation control unit 14.

[0164] In addition, the operation of the remote operating device 1 is performed in the order of steps ST501 and ST502, but it is not limited to this. The operation of the remote operating device 1 can also be performed in the order of steps ST502 and ST501.

[0165] The delay time determination unit 131 of the switching control unit 13 determines whether the communication delay time detected by the delay time detection unit 12 in step ST501 exceeds the delay time determination threshold (step ST503).

[0166] If the delay time determination unit 131 determines in step ST503 that the communication delay time does not exceed the delay time determination threshold (the "No" case in step ST503), the operation of the remote operation device 1 returns to step ST501 and proceeds to the operation after step ST501.

[0167] In step ST503, when it is determined that the communication delay time exceeds the delay time determination threshold (the case of "yes" in step ST503), the delay time determination unit 131 determines that the remote operation device 1 for remote operation of vehicle 2 should be switched, and outputs delay existence information indicating that the communication delay time exceeds the delay time determination threshold to the switching target determination unit 132 of the switching control unit 13.

[0168] The switching target determination unit 132 determines the switching target remote operating device from the candidate remote operating devices based on the information related to the candidate remote operating devices acquired by the acquisition unit 11 in step ST502, according to whether the switching determination conditions are met (step ST504).

[0169] When deciding on the target remote operating device, the target switching unit 132 outputs information related to the determined target remote operating device to the switching control unit 13.

[0170] Furthermore, in step ST504, if there is no candidate remote operating device that meets the switching determination conditions and the switching target determination unit 132 cannot determine the switching target remote operating device, the operation control unit 14 generates operation information to stop the vehicle 2 and outputs the generated operation information to the vehicle 2 via the output unit 16. Then, the remote operating device 1 terminates. Figure 6 The actions are shown in the flowchart.

[0171] The switching control unit 13 switches the remote operating device 1 that remotely operates the vehicle 2 to the target remote operating device determined by the switching target determination unit 132 in step ST504 (step ST505). Specifically, the switching control unit 13 outputs switching instruction information to the target remote operating device to enable it to remotely operate the vehicle 2.

[0172] Here, Figure 7 This is used to explain the implementation method 1. Figure 6 The flowchart shows the detailed actions in step ST505.

[0173] exist Figure 6 In step ST504, when the handover target determination unit 132 outputs information related to the handover target remote operation device, the handover control unit 13 outputs handover preparation instruction information to the handover target remote operation device (step ST601).

[0174] The differential calculation unit 133 of the switching control unit 13 calculates the operation information generated by the operation control unit 14 of the switching source remote operation device and the preparation operation information generated by the switching target remote operation device (see below). Figure 9 The difference between step ST804 and step ST602.

[0175] The differential calculation unit 133 outputs differential information to the switching control unit 13.

[0176] When differential information is output from the differential calculation unit 133 in step ST602, the switching control unit 13 determines whether the state of the difference between the operation information and the preparation operation information within the differential determination threshold has continued for the differential duration determination time (step ST603).

[0177] If, in step ST603, the switching control unit 13 determines that the state of the difference between the operation information and the preparation operation information is within the difference determination threshold and the difference determination time is not continued (the case of "No" in step ST603), the operation of the remote operation device 1 returns to step ST602 and performs the operation after step ST602.

[0178] In step ST603, if the state in which the difference between the operation information and the preparation operation information is within the difference determination threshold has persisted for the difference persistence determination time (the case of "Yes" in step ST603), the switching control unit 13 determines that the time has been reached to switch to the operation information. Figure 6 In step ST504, the switching timing for remote operation of vehicle 2 is determined by the switching target determination unit 132 to the switching target remote operation device.

[0179] If the switching control unit 13 determines that the switching time has been reached, it outputs switching instruction information to the remote operating device of the switching target and does not cause the operation control unit 14 of the remote operating device of the switching source to generate operation information.

[0180] Therefore, the remote control device 1 for remotely operating vehicle 2 is switched.

[0181] Figure 8 This is used to illustrate that in implementation method 1, such as Figure 7 This is an example diagram of an image showing the switching process when switching the remote control device 1 that remotely operates vehicle 2, as illustrated in the flowchart.

[0182] In the remote operation device for switching sources, if it is determined that the difference between the operation information and the preparation operation information exceeds the threshold for difference determination, vehicle 2 continues to perform remote operation based on the operation information output from the remote operation device for switching sources (see reference). Figure 8 (of 701).

[0183] Subsequently, in the remote operation device for switching sources, even if it is determined that the difference between the operation information and the preparation operation information is within the difference determination threshold, if the state that the difference is within the difference determination threshold does not continue for the difference duration determination time, vehicle 2 continues to perform remote operation based on the operation information output from the remote operation device for switching sources (see reference). Figure 8 702).

[0184] In the source switching remote operation device, when it is determined that the difference between the operation information and the preparation operation information is within the difference determination threshold and the difference persistence determination time has been maintained, vehicle 2 performs remote operation based on the operation information output from the target switching remote operation device (see reference). Figure 8 703).

[0185] Figure 9 This is a flowchart illustrating the operation of the remote operating device 1 according to Embodiment 1, which is used as a candidate remote operating device.

[0186] The detection unit 15 detects information related to the remote operation device 1, or in other words, the candidate remote operation device. Specifically, the device information detection unit 151 detects device status information. In addition, the operator information detection unit 152 detects operator status information (step ST801).

[0187] The output unit 16 outputs the information related to the candidate remote operating device detected by the detection unit 15 in step ST801 to the switching source remote operating device (step ST802).

[0188] The switching control information acquisition unit 115 checks whether it has acquired the switching preparation indication information output by the switching source remote operating device from the switching source remote operating device (refer to...). Figure 7 The determination is made in step ST601 (step ST803).

[0189] In step ST803, if the switching control information acquisition unit 115 does not acquire the switching preparation indication information (the case of "No" in step ST803), the operation of the remote operation device 1 returns to step ST801 and performs the operation after step ST801.

[0190] In step ST803, if the switching control information acquisition unit 115 acquires the switching preparation instruction information (if "Yes" is true in step ST803), the operation control unit 14 generates preparation operation information. Then, the operation control unit 14 outputs the generated preparation operation information to the switching source remote operation device via the output unit 16 (step ST804).

[0191] The switching control information acquisition unit 115 checks whether it has acquired the switching instruction information output by the switching source remote operating device from the switching source remote operating device (refer to...). Figure 7 The determination is made in step ST601 (step ST805).

[0192] In step ST805, if the switching control information acquisition unit 115 does not acquire the switching instruction information (the case of "No" in step ST805), the operation of the remote operation device 1 returns to step ST804 and performs the operation after step ST804.

[0193] In step ST805, if the switching control information acquisition unit 115 acquires switching instruction information (if "Yes" is true in step ST805), the operation control unit 14 generates operation information. The operation control unit 14 outputs the generated operation information to the vehicle 2 via the output unit 16 (step ST806).

[0194] When operation information is output in step ST806, the remote operating device 1 operates as a remote operating device for remotely operating the vehicle 2 (see reference). Figure 6 , Figure 7 ).

[0195] Figure 10 It is a timing diagram showing the actions in Embodiment 1, from the time the switching source remote operating device determines the switching target remote operating device until the switching target remote operating device is switched to the remote operating device 1 that remotely operates the vehicle 2.

[0196] Figure 10 Indicates use Figure 7 The flowchart illustrates the actions of the remote control device for switching sources, and in Figure 9 The actions of switching the target remote operating device are described in steps ST803 to ST806.

[0197] When the remote operating device of the switching source determines the remote operating device of the switching target, the remote operating device of the switching source outputs a switching preparation instruction information to the remote operating device of the switching target (step ST901).

[0198] The target remote operating device generates preparation operation information based on the vehicle information provided in the handover preparation instruction information. The target remote operating device outputs the generated preparation operation information to the source remote operating device (step ST902).

[0199] In addition, the preparation operation information is not output to vehicle 2. Based on the operation information generated by the switching source remote operation device based on the vehicle information obtained from vehicle 2, vehicle 2 is remotely operated (steps ST903 to ST904).

[0200] The switching source remote operating device calculates the difference between the operation information and the preparation operation information obtained from the switching target remote operating device (step ST905). Then, the switching source remote operating device determines whether the state where the calculated difference is within the difference determination threshold has been maintained for the difference duration determination time.

[0201] In the remote operation device for switching sources, steps ST902 to ST905 are repeated until the state in which the difference calculated in step ST905 is within the threshold for difference determination continues for the difference determination time.

[0202] If the remote operating device for switching sources determines that the state where the difference calculated in step ST905 is within the difference determination threshold has persisted for the difference persistence determination time, it outputs switching instruction information to the remote operating device for switching targets (step ST906). At this time, the remote operating device for switching sources stops generating operating information. That is, operating information is no longer output from the remote operating device for switching sources to vehicle 2.

[0203] If the target remote operating device receives the switching instruction information output in step ST906, it generates operating information based on the vehicle information obtained from vehicle 2. The target remote operating device outputs the generated operating information to vehicle 2 (steps ST907-ST908). At this time, the source remote operating device stops preparing to generate operating information.

[0204] Subsequently, the target remote operating device is switched to remote operating device 1 for remotely operating vehicle 2. Thus, remote operating device 1 for remotely operating vehicle 2 is switched.

[0205] As described above, in the process of remotely operating the vehicle 2, if the communication delay time in communication with the vehicle 2 exceeds a delay determination threshold, the remote operating device 1 according to Embodiment 1 determines to switch the remote operating device 1 that is remotely operating the vehicle 2. Then, based on information related to candidate remote operating devices, the remote operating device 1 determines the target remote operating device to switch from the candidate remote operating devices according to whether the switching determination condition is met. The remote operating device 1 switches the remote operating device 1 that is remotely operating the vehicle 2 by outputting switching instruction information to the determined target remote operating device.

[0206] Specifically, if the remote operating device 1, or in other words, the switching source remote operating device, decides to switch to the target remote operating device, it first outputs a switching preparation instruction to the target remote operating device. The switching source remote operating device obtains the preparation operation information generated based on the switching preparation instruction from the target remote operating device, and calculates the difference between the operation information to be output to vehicle 2 for remote operation of vehicle 2 and the preparation operation information. Furthermore, if the difference between the operation information and the preparation operation information remains within the difference determination threshold for a period of time, the switching source remote operating device outputs a switching instruction to the target remote operating device, and does not generate operation information in the switching source remote operating device.

[0207] In the remote operating device that receives the switching instruction information, operation information is generated and output to vehicle 2. The remote operating device is then switched to remote operating device 1 that remotely operates vehicle 2.

[0208] As described above, in the process of remotely operating the vehicle 2, if the communication delay time in the communication with the vehicle 2 exceeds the delay determination threshold, the remote operating device 1 of Embodiment 1 determines to switch the remote operating device 1. Therefore, it can prevent the state of not being able to maintain normal remote driving due to the increased communication delay in the communication for remote operation of the vehicle 2.

[0209] Here, Figure 11 This is an example diagram showing an image of a remote operating device 1 switching to remotely operate a vehicle 2 based on a communication delay time in communication with the vehicle 2 in Embodiment 1.

[0210] exist Figure 11In the process, remote operating device 1 (1001), remote operating device 1 (1002) and remote operating device 1 (1003) are installed on the path of vehicle 2.

[0211] Initially, it is assumed that the remote operating device 1 (1002) located at the location closest to the current position of vehicle 2 is determined to be the remote operating device 1 for remotely operating vehicle 2.

[0212] Subsequently, as vehicle 2 moves, the distance between vehicle 2 and remote operating device 1 (1002) increases. Consequently, the communication delay time in the communication between vehicle 2 and remote operating device 1 (1002) also increases, eventually exceeding the delay time determination threshold.

[0213] When the communication delay between vehicle 2 and remote operating device 1 (1002) exceeds the delay time determination threshold, remote operating device 1 (1002) determines to switch remote operating device 1. Then, remote operating device 1 (1002) determines remote operating device 1 (1003) whose communication delay with vehicle 2 is shorter than that of remote operating device 1 (1002) as the target remote operating device to switch.

[0214] When a switching instruction is received from the remote operating device 1 (1002), the remote operating device 1 (1003) that is determined to be the target remote operating device for switching performs remote operation on the vehicle 2.

[0215] Furthermore, when the remote operating device 1 according to Embodiment 1 is a switching source remote operating device, as described above, if a target remote operating device is determined to be switched, preparation operation information is first generated for the target remote operating device. The switching source remote operating device obtains the preparation operation information from the target remote operating device. When the difference between the operation information and the preparation operation information is within a difference determination threshold and the difference persistence determination time continues, the remote operating device 1 is switched so that the target remote operating device operates as a remote operating device 1 for remotely operating the vehicle 2.

[0216] When remote operating device 1 is actually switched, and after the switch, operating information is output from remote operating device 1, which is the source of the switch, to vehicle 2, remote operating device 1 has already been operated by the remote operator. That is, during the period when the remote operator operates remote operating device 1, which has become the source of the switch after the switch, remote operating device 1 does not need to stop vehicle 2. As a result, the driving efficiency of vehicle 2 can be improved. Specifically, for example, when vehicle 2 is a taxi, bus, or truck, remote operating device 1 does not need to stop vehicle 2, thereby improving the operating rate of vehicle 2 itself. In addition, remote operating device 1 does not need to stop vehicle 2, thereby enabling vehicle 2 to reach its destination quickly.

[0217] Furthermore, under normal circumstances, the likelihood of an abnormal event occurring increases when vehicle 2 stops or starts. When remote operating device 1 is a switching source remote operating device, switching remote operating device 1 to remotely operate vehicle 2 does not require stopping vehicle 2, thus reducing the likelihood of an abnormal event occurring during the switching process.

[0218] Furthermore, even if the remote control device 1 of the remotely operated vehicle 2 changes, in other words, even if the remote operator who operates the remote control device 1 of the remotely operated vehicle 2 changes, the remote control device 1 can still reduce the sudden and large changes in the control quantity of the vehicle 2.

[0219] In Embodiment 1 described above, the remote operating device 1 and the vehicle 2 can be connected, for example, solely via wireless communication. Alternatively, the remote operating device 1 and the vehicle 2 can be connected via communication methods including wired and wireless communication, such as public networks like mobile communication networks or the Internet, or dedicated lines. The remote operating device 1 and the vehicle 2 are connected via communication methods including wireless or wired communication, and the remote operating device 1 remotely operates the vehicle 2 through this communication.

[0220] The communication delay between remote operating device 1 and vehicle 2 includes not only the wireless communication delay between remote operating device 1 and vehicle 2, but also the wired communication delay between remote operating device 1 and vehicle 2. If, during remote operation of vehicle 2, the communication delay exceeds a delay determination threshold, remote operating device 1 determines to switch to another device. This prevents the communication delay during remote operation of vehicle 2 from becoming so large that it cannot maintain normal remote driving.

[0221] In addition, in the above embodiment 1, the remote operation device 1 includes a differential calculation unit 133, but the remote operation device 1 does not necessarily need to include a differential calculation unit 133.

[0222] If the remote operating device 1 does not have a differential calculation unit 133, and the target switching unit 132 decides to switch to a target remote operating device, then the target switching unit 132 outputs switching instruction information to the target remote operating device.

[0223] In this case, the remote operation device 1 may be configured to not include the preparation operation information acquisition unit 114 and the preparation operation information output unit 165.

[0224] Furthermore, in Embodiment 1 above, the object remotely operated by the remote operating device 1 is set as vehicle 2, but this is only an example. The remote operating device 1 can apply Embodiment 1 above to all mobile bodies that can be remotely operated, and perform remote operation on those mobile bodies.

[0225] Figure 12A , Figure 12B This is a diagram illustrating an example of the hardware structure of the remote operation device 1 according to Embodiment 1.

[0226] In Embodiment 1, the functions of the acquisition unit 11, the delay time detection unit 12, the switching control unit 13, the operation control unit 14, the detection unit 15, and the output unit 16 are implemented by the processing circuit 1101. That is, the remote operation device 1 includes the processing circuit 1101, which is used to switch the control of the remote operation device 1 that performs remote operation on the vehicle 2 when the communication delay time between the remote operation device 1 and the vehicle 2 exceeds the delay time determination threshold during the remote operation of the vehicle 2.

[0227] Processing circuit 1101 can be as follows Figure 12A What is shown is dedicated hardware, and it can also be like... Figure 12B The CPU (Central Processing Unit) 1105 shown is executing the program stored in memory 1106.

[0228] When the processing circuit 1101 is dedicated hardware, the processing circuit 1101 corresponds, for example, to a single circuit, a composite circuit, a programmed processor, a parallel programmed processor, an ASIC (Application Specific Integrated Circuit), an FPGA (Field-Programmable Gate Array), or a combination thereof.

[0229] When the processing circuit 1101 is a CPU 1105, the functions of the acquisition unit 11, delay time detection unit 12, switching control unit 13, operation control unit 14, detection unit 15, and output unit 16 are implemented by software, firmware, or a combination of software and firmware. That is, the acquisition unit 11, delay time detection unit 12, switching control unit 13, operation control unit 14, detection unit 15, and output unit 16 are implemented by a processing circuit such as a CPU 1105 or a system LSI (Large-Scale Integration) that executes programs stored in the memory 1106, etc. Furthermore, the programs stored in the memory 1106, etc., are naturally programs that cause the computer to execute the steps and methods of the acquisition unit 11, delay time detection unit 12, switching control unit 13, operation control unit 14, detection unit 15, and output unit 16. Here, memory 1106 is, for example, a non-volatile or volatile semiconductor memory such as RAM (Random Access Memory), ROM (Read Only Memory), flash memory, EPROM (Erasable Programmable Read Only Memory), and EEPROM (Electrically Erasable Programmable Read-Only Memory), as well as a hard disk, optical disk, etc.

[0230] Furthermore, the functions of the acquisition unit 11, delay time detection unit 12, switching control unit 13, operation control unit 14, detection unit 15, and output unit 16 can be partially implemented using dedicated hardware and partially implemented using software or firmware. For example, the functions of the acquisition unit 11 and output unit 16 can be implemented using the processing circuit 1101, which is dedicated hardware. The functions of the delay time detection unit 12, switching control unit 13, operation control unit 14, and detection unit 15 can be implemented by the processing circuit 1101 reading and executing the program stored in the memory 1106.

[0231] Additionally, the remote operating device 1 has an input interface device 1103 and an output interface device 1104 for communicating with the vehicle 2 or other remote operating devices 1. The communication unit, which is omitted from the illustration, consists of an input interface device and an output interface device.

[0232] As described above, the remote operation device 1 according to Embodiment 1 is a remote operation device 1 capable of remotely operating a mobile body via communication. The remote operation device 1 includes: an acquisition unit 11, which acquires information related to other remote operation devices 1 capable of remotely operating a mobile body via communication; a delay time detection unit 12, which detects the communication delay time in communication with the mobile body; a delay time determination unit 131, which determines whether the communication delay time detected by the delay time detection unit 12 exceeds a delay time determination threshold; a switching target determination unit 132, which, when the delay time determination unit 131 determines that the communication delay time exceeds the delay time determination threshold, determines a switching target remote operation device from the other remote operation devices 1 based on the information related to other remote operation devices 1 acquired by the acquisition unit 11 and according to whether a switching determination condition is met; and a switching control unit 13, which outputs switching instruction information to the switching target remote operation device determined by the switching target determination unit 132 to enable it to remotely operate the mobile body. Therefore, the remote operation device 1 can prevent the situation where the communication delay in the communication used for remote operation of the vehicle 2 becomes so large that it cannot maintain the state of normal remote driving.

[0233] Furthermore, the remote operation device 1 according to Embodiment 1 is configured to include: an operation control unit 14 that generates operation information for remotely operating a mobile body; a preparation operation information acquisition unit 114 that acquires preparation operation information for remotely operating a mobile body generated by the target remote operation device from the target remote operation device; and a difference calculation unit 133 that calculates the difference between the operation information generated by the operation control unit 14 and the preparation operation information acquired by the preparation operation information acquisition unit 114. If the difference calculated by the difference calculation unit 133 remains within a difference determination threshold for a certain duration, the switching control unit 13 outputs switching instruction information to the target remote operation device and does not cause the operation control unit 14 to generate operation information. During the period when the remote operator operates the switched remote operation device 1 after the switch, the remote operation device 1 does not need to stop the vehicle 2. As a result, the driving efficiency of the vehicle 2 can be improved.

[0234] Furthermore, even if the remote operating device 1 of the remotely operated vehicle 2 changes, in other words, even if the remote operator who operates the remote operating device 1 of the remotely operated vehicle 2 changes, the remote operating device 1 can still reduce the sudden and large changes in the control quantity of the vehicle 2.

[0235] Furthermore, within the scope of this invention, any structural element of the embodiments can be modified, or any structural element can be omitted in the embodiments.

[0236] Industrial practicality

[0237] The remote operation device involved in this invention is configured to prevent the situation where the communication delay in the communication used for remote operation of the mobile body becomes so large that it cannot maintain the state of normal remote driving, and thus the state of normal remote driving cannot be maintained. Therefore, it can be applied to a remote operation device for remote operation of a mobile body.

[0238] Label Explanation

[0239] 1. Remote operating device

[0240] 11 Acquisition Department

[0241] 111 Device Information Acquisition Unit

[0242] 112 Operator Information Acquisition Department

[0243] 113 Vehicle Information Acquisition Department

[0244] 114 Preparation Operation Information Acquisition Department

[0245] 115 Switching Control Information Acquisition Unit

[0246] 12 Delay Time Detection Department

[0247] 13 Switching Control Unit

[0248] 131 Delay Time Determination Department

[0249] 132 Switch Target Decision Department

[0250] 133 Difference Calculation Department

[0251] 14 Operation and Control Department

[0252] 15. Testing Department

[0253] 151 Device Information Detection Department

[0254] 152 Operator Information Detection Department

[0255] 16 Output Section

[0256] 161 Device Information Output Unit

[0257] 162 Operator Information Output Department

[0258] 163 Switching control information output unit

[0259] 164 Operation Information Output Department

[0260] 165 Preparation for Operation Information Output Department

[0261] 1101 Processing Circuit

[0262] 1102 HDD

[0263] 1103 Input Interface Device

[0264] 1104 Output Interface Device

[0265] 1105 CPU

[0266] 1106 Memory.

Claims

1. A remote operation device, It is a remote operation device capable of remotely operating a moving body via communication, characterized in that... include: The acquisition unit acquires information relating to other remote operating devices capable of remotely operating the mobile body via the communication; A delay time detection unit that detects communication delay time in the communication with the mobile body; The delay time determination unit determines whether the communication delay time detected by the delay time detection unit exceeds a delay time determination threshold. The switching target determination unit, when the delay time determination unit determines that the communication delay time exceeds the delay time determination threshold, determines the switching target remote operating device from the other remote operating devices based on the information related to the other remote operating devices obtained by the acquisition unit, and whether the switching determination condition is met. A switching control unit outputs switching instruction information to the remote operating device of the switching target determined by the switching target determination unit, enabling it to remotely operate the mobile body. An operation control unit that generates operation information for remotely operating the mobile body; The preparation operation information acquisition unit acquires preparation operation information generated by the switching target remote operation device for remotely operating the mobile body from the switching target remote operation device. as well as The differential calculation unit calculates the difference between the operation information generated by the operation control unit and the preparation operation information acquired by the preparation operation information acquisition unit. When the state in which the difference calculated by the difference calculation unit is within the difference determination threshold has been maintained for the difference determination time, the switching control unit outputs the switching instruction information to the remote operation device of the switching target and prevents the operation control unit from generating the operation information.

2. The remote operation device as described in claim 1, characterized in that, The acquisition unit includes a device information acquisition unit, which acquires device status information related to the communication delay time between the other remote operating devices and the mobile body. The switching determination condition refers to the fact that the communication delay time between the other remote operating device and the mobile body, as obtained by the device information acquisition unit, is below the threshold for determining the delay time of other devices.

3. The remote operation device as described in claim 2, characterized in that, The acquisition unit includes an operator information acquisition unit, which acquires operation status information related to the status of a remote operator operating the other remote operating device. The switching determination condition refers to the state of the remote operator who operates the other remote operating device, as obtained by the operator information acquisition unit, being able to operate the other remote operating device, and the communication delay time between the other remote operating device and the mobile body, as obtained by the device information acquisition unit, being below the threshold for determining the delay time of the other device.

4. A remote operation method, It is a remote operation method performed by a remote operation device capable of remotely operating a mobile body via communication, characterized in that... include: The step of acquiring information relating to other remote operating devices capable of remotely operating the mobile body via the communication; The step of detecting communication delay time in the communication with the moving body by the delay time detection unit; The delay time determination unit determines whether the communication delay time detected by the delay time detection unit exceeds the delay time determination threshold. When the delay time determination unit determines that the communication delay time exceeds the delay time determination threshold, the switching target determination unit determines the target remote operating device from the other remote operating devices based on the information related to the other remote operating devices obtained by the acquisition unit, and whether the switching determination condition is met. The step of the switching control unit outputting switching instruction information to the remote operating device of the switching target determined by the switching target determination unit, enabling it to remotely operate the mobile body; The step of the operation control unit generating operation information for remotely operating the mobile body; The step of the preparation operation information acquisition unit acquiring preparation operation information for remotely operating the mobile body generated by the switching target remote operation device; as well as The step of the difference calculation unit calculating the difference between the operation information generated by the operation control unit and the preparation operation information acquired by the preparation operation information acquisition unit. When the state in which the difference calculated by the difference calculation unit is within the difference determination threshold has been maintained for the difference determination time, the switching control unit outputs the switching instruction information to the remote operation device of the switching target and prevents the operation control unit from generating the operation information.

Images