Remotely operated vehicle control device

Abstract

To provide a technology that can determine, with high reliability, whether or not remote operation can be started for a vehicle subject to remote operation.SOLUTION: A controller of a remotely operated vehicle obtains a target tire angle from a remote operation device and also obtains an actual tire angle of the remotely operated vehicle. The controller switches operation of the remotely operated vehicle to remote operation by the remote operation device when the target tire angle and the actual tire angle match with each other. This allows a decision to start remote operation with high reliability.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present disclosure relates to a control device for a remotely operated vehicle. [Background technology]

[0002] Patent Document 1 discloses a technology for acquiring information indicating the equipment status of a remotely driven vehicle that is the target of remote driving. An information processing device controls the remote driving device based on information indicating the equipment status of the remotely driven vehicle and information indicating the equipment status of a remote driving device that remotely drives the remotely driven vehicle. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2019-174993 A Summary of the Invention [Problem to be solved by the invention]

[0004] As shown in Patent Document 1, a technology for remotely driving a vehicle is known. A vehicle to be remotely driven is called a remotely driven vehicle or a remotely driven vehicle. The remotely driven vehicle is driven based on an operation input to a remote driving device called a remote cockpit. When a request to start remote driving is received, remote driving is started. However, if remote driving is started immediately without checking the state of the remotely driven vehicle and the remote cockpit, a malfunction may occur due to a large discrepancy between the state of the remotely driven vehicle and the state of the remote cockpit. Therefore, before starting remote driving, it is necessary to accurately determine whether or not remote driving is possible. Here, it is possible to determine whether to start remote driving in the remote cockpit. However, the remote cockpit is merely an operating system, and the occurrence rate of erroneous judgment due to a malfunction or the like is higher than that of the vehicle. In order to suppress erroneous judgment, it is preferable to determine whether to start remote driving on the vehicle side.

[0005] An object of the present disclosure is to provide a technology that can reliably determine whether or not remote driving can be started in a vehicle that is the subject of remote driving. [Means for solving the problem]

[0006] The present disclosure relates to a control device for a remotely operated vehicle. The control device includes: Obtaining a target tire angle from a remote driving device and obtaining an actual tire angle of the remote driving vehicle; In response to the match between the target tire angle and the actual tire angle, switching the driving of the remotely driven vehicle to remote driving by the remote driving device. It is characterized by: Effect of the Invention

[0007] According to the present disclosure, in a vehicle to be remotely driven, it is possible to perform a highly reliable determination as to whether remote driving can be started. By performing a highly reliable determination, the occurrence of problems when remote driving is started is suppressed. [Brief description of the drawings]

[0008] [Figure 1] FIG. 4 is a conceptual diagram for explaining a remote operation switching condition in the embodiment of the present disclosure. [Diagram 2] FIG. 4 is a conceptual diagram for explaining a remote operation switching condition in the embodiment of the present disclosure. [Diagram 3] 1 is a conceptual diagram for explaining visual support provided by a remote driving system according to an embodiment of the present disclosure. [Figure 4] 1 is a conceptual diagram for explaining visual support provided by a remote driving system according to an embodiment of the present disclosure. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] Embodiments of the present disclosure will be described with reference to the accompanying drawings.

[0010] 1. Remotely operated vehicle control device The control device according to the present embodiment is mounted on a vehicle and enables remote driving of the vehicle. The control device according to the present embodiment may be a device built into the vehicle as a vehicle component, or may be an external device that can be attached to and detached from the vehicle. The latter is called a remote driving kit. In the following description, the control device is assumed to be a remote driving kit. A vehicle equipped with a remote driving kit and capable of remote driving is called a remote driving vehicle. A remote driving device for remotely driving the remote driving vehicle is called a remote cockpit. The remote driving kit enables remote driving of the vehicle by transmitting a target control amount to the remote driving vehicle, communicating with the outside of the vehicle, and transmitting the state of the remote driving vehicle to the remote cockpit. The remote cockpit is equipped with a monitor for displaying information acquired by an on-board camera and an operation system for inputting a target control amount of the remote driving vehicle, and is operated by a remote driver. In addition, the remote driving vehicle may be simultaneously equipped with an automatic driving kit that enables automatic driving.

[0011] The remote driving vehicle starts remote driving when the start of remote driving is requested. The timing when the start of remote driving is requested is, for example, when the remote driving vehicle that was being driven automatically by the automatic driving kit approaches a place that requires assistance by remote driving, such as an intersection, or when the occupant of the remote driving vehicle requests remote driving. When remote driving of the remote driving vehicle starts and is switched to a remote driving state, it is driven according to the operation input to the remote cockpit. However, when the start of remote driving is requested, the remote driving vehicle is not necessarily in a state where it can be immediately switched to a remote driving state. For example, the tires of the remote driving vehicle may be turned to the right, and the steering wheel of the remote cockpit may be reset to the center position. If remote driving is started in such a state, a malfunction may occur in which the remote driving vehicle is not driven as intended by the remote driver.

[0012] Therefore, in this embodiment, a remote driving switching condition is set as a condition for safely switching the remote driving vehicle to the remote driving state, and the remote driving vehicle is switched to the remote driving state only when the remote driving switching condition is satisfied. For example, by setting the remote driving switching condition that the state of the operation system of the remote cockpit and the state of the control system of the remote driving vehicle match, the control of the vehicle becomes smooth when switching to the remote driving state, and the riding comfort of the passengers of the remote driving vehicle is improved. Alternatively, for example, by setting the remote driving switching condition that the amount of brake operation by the remote driver is sufficient, it is possible to prevent the remote driving vehicle from suddenly starting.

[0013] In this embodiment, when a request to start remote driving is made, the decision of whether to switch to the remote driving state is made not on the remote cockpit side but on the remote driving vehicle side on which the remote driving kit is mounted. The device mounted on the vehicle side has a lower risk of failure and is also highly expandable compared to an operation system such as a remote cockpit. Therefore, by making the decision on the vehicle side, the reliability of the decision can be greatly improved. Furthermore, by making the decision on the vehicle side, the remote cockpit can be made of general-purpose parts, and costs can be reduced. In order to further improve reliability, the decision system of the remote driving kit may be designed to be redundant.

[0014] Here, it is appropriate that the remote driving switching condition is that the state of the operation system of the remote cockpit and the state of the control system of the remotely driven vehicle have a specified correspondence relationship. For example, it is assumed that the remote driving switching condition is that the range of the shift lever of the remote cockpit and the range of the shift lever of the remotely driven vehicle match. Alternatively, for example, the remote driving switching condition may be that the acceleration amount input to the remote cockpit is an amount that does not cause sudden acceleration or deceleration by comparing the acceleration amount input to the remote cockpit with the vehicle speed of the remotely driven vehicle. Among these, one of the most important remote driving switching conditions is that the actual tire angle and the target tire angle match. The actual tire angle is the actual tire angle of the remotely driven vehicle, and the target tire angle is the value input by the remote driver operating the operation system of the remote cockpit, such as the steering wheel, converted into the tire angle of the remotely driven vehicle.

[0015] A typical operation system for controlling the tire angle of a remote-driven vehicle is a steering wheel. However, the operation system is not limited to a steering wheel, and a joystick, a touch panel, an input device in the shape of a game controller, or the like may be used as the operation system. For example, the target tire angle may be calculated by the remote driver operating a joystick left and right. When the operation system for controlling the tire angle is a steering wheel, the operation angle of the steering wheel is called the steering angle.

[0016] Here, the agreement between the actual tire angle and the target tire angle includes not only a perfect agreement but also an agreement in which the difference between the actual tire angle and the target tire angle is smaller than a certain value. Furthermore, the agreement between the actual tire angle and the target tire angle may be determined based on the tire angle, or, if the operation system is steering, based on the steering angle. In other words, the remote driving switching condition may be satisfied when the value obtained by converting the actual tire angle into the steering angle matches the steering angle of the remote cockpit.

[0017] When the remote-driven vehicle is autonomously driven, the actual tire angle is more likely to change than the state of other control systems of the remote-driven vehicle. Therefore, the actual tire angle and the target tire angle are particularly likely to differ between the remote cockpit and the remote-driven vehicle. Therefore, if the remote-driving switching condition is that the actual tire angle and the target tire angle match, the discomfort felt by the remote driver when switching to the remote driving state can be particularly reduced. In addition, the movement of the vehicle when switching to the remote driving state becomes smoother, so that both comfortable operation by the remote driver and ride comfort for the passengers can be achieved.

[0018] Therefore, the control device according to this embodiment sets the match between the actual tire angle and the target tire angle as the remote driving switching condition. That is, the remote driving kit acquires information on the actual tire angle and the target tire angle. Then, when the actual tire angle and the target tire angle match, it determines that the remote driving switching condition is satisfied and switches the remotely driven vehicle to a remote driving state.

[0019] FIG. 1 conceptually shows an example of a remote driving switching condition in this embodiment. In FIG. 1, the operation system for operating the tire angle of the remotely driven vehicle is the steering. Information on the vehicle state including the actual tire angle is notified to the remote cockpit. The remote driver operates the remote cockpit while checking the information on the vehicle state. At this time, the remote cockpit may assist the operation of the remote driver by applying a reaction force to the steering. The steering angle input by the operation of the remote driver is converted into a target tire angle and transmitted to the remotely driven vehicle. The remote driving kit compares the actual tire angle with the target tire angle, and switches the remotely driven vehicle to a remote driving state when the actual tire angle and the target tire angle match.

[0020] This flow is shown, for example, in FIG. 2. When the remote driving kit is activated, the remote driving vehicle is in a state where remote driving is possible. When the start of remote driving is requested, the remote driving kit notifies the remote cockpit of information about the vehicle state acquired from the remote driving vehicle. The remote cockpit receives the notification from the remote driving kit and notifies the remote driving kit of information about the state of the operation system. The information notified from the remote cockpit to the remote kit includes the target tire angle or the steering angle corresponding to the target tire angle. If the actual tire angle and the target tire angle match based on the notified information, the remote driving kit determines that the remote driving switching condition is satisfied, and transmits a request to switch to the remote driving state to the remote driving vehicle. When the switching is completed, a switching completion notification is sent from the remote driving vehicle to the remote cockpit through the remote driving kit, and the operation system of the remote cockpit displays that the switching is completed. After the switching, information for remote driving is transmitted between the remote driving vehicle, the remote driving kit, and the remote cockpit, and the remote driving vehicle is remotely driven.

[0021] 2.Remote Operation System The control device according to this embodiment constitutes a remote driving system together with a remote cockpit.

[0022] When the remote driving switching condition is not satisfied, it is necessary to make the actual tire angle and the target tire angle coincide with each other in order to start remote driving. There are two methods for making the actual tire angle and the target tire angle coincide with each other: a method for changing the actual tire angle and a method for changing the target tire angle. The remote driving system according to this embodiment does not change the actual tire angle. Instead, the remote driver is prompted to operate the operation system such as the steering wheel so that the target tire angle coincides with the actual tire angle, and the operation of the remote driver to make the target tire angle coincide with the actual tire angle is supported. Since the remote driver himself is required to perform the operation in order to make the target tire angle coincide with the actual tire angle, it becomes easier for the remote driver to grasp the state of the remotely driven vehicle with a more realistic feeling, and remote driving becomes smoother. Furthermore, since it is not necessary to mount a control mechanism for making the target tire angle coincide with the actual tire angle in the operation system of the remote cockpit, costs can be reduced.

[0023] The operation for matching the target tire angle with the actual tire angle and satisfying the remote driving switching condition is called initial state preparation. In a flow similar to that of Fig. 2, when the remote cockpit receives information about the vehicle state, the remote driving system judges whether the actual tire angle and the target tire angle match. If they do not match, the remote driving system issues a request to the remote driver to match the actual tire angle with the target tire angle. This request is conveyed to the remote driver, for example, by being displayed on a monitor in the remote cockpit or by being output as voice guidance from a speaker in the remote cockpit.

[0024] The remote driver who receives the request must operate the steering wheel and other operating systems himself to prepare for the initial state. The remote driving system receives a confirmation request from the remote driver who has prepared for the initial state and judges again whether the target tire angle and the actual tire angle match. If the target tire angle and the actual tire angle do not match even after the second judgment, the remote driving system again requests the remote driver to match the target tire angle and the actual tire angle. If the target tire angle and the actual tire angle match, the remote driving vehicle is switched to a remote driving state.

[0025] The remote driving system according to this embodiment supports the operation of the remote driver who is preparing for the initial state. The support includes visual support and tactile support. Here, the visual support will be explained first. The visual support is performed by visually teaching the remote driver the operation direction and operation amount of the remote cockpit operation system required to satisfy the remote driving switching conditions by displaying them on a monitor in the remote cockpit or the like.

[0026] The remote driving system acquires information on the actual tire angle and information on the target tire angle inputted to the remote cockpit. Then, the difference between the actual tire angle and the target tire angle is visually displayed. For example, as shown in FIG. 3, the predicted tire trajectory is displayed on the monitor. In the monitor, the tire trajectory is drawn by dotted lines and dashed lines superimposed on the scenery around the vehicle acquired by the vehicle-mounted camera. The tire trajectory drawn by dotted lines displays the predicted trajectory based on the actual tire angle, and the tire trajectory drawn by dashed lines displays the predicted trajectory based on the target tire angle. The remote driver can match the actual tire angle with the target tire angle by operating the operation system such as the steering wheel so that the predicted trajectory based on the actual tire angle and the predicted trajectory based on the target tire angle match. In FIG. 3, the predicted trajectory based on the actual tire angle and the predicted trajectory based on the target tire angle are displayed by dotted lines and dashed lines, but the two predicted trajectories may be distinguished by displaying them by solid lines of different colors. In addition, in the display of the predicted tire trajectory line exemplified in FIG. 3, when the remote driver operates an operating system such as the steering wheel in the direction opposite to the target, the predicted tire trajectory line may be changed to a display that emphasizes the correct operating direction by flashing or changing to a highlighted color.

[0027] The tire angle may also be displayed as a numerical value. The numerical value can be displayed above or below the monitor that projects the scenery around the vehicle, or can be superimposed on the monitor that projects the scenery around the vehicle. For example, it may be displayed as "Necessary operation amount: 50 deg left remaining (remotely driven vehicle tire angle: 150 deg left)", and the remote driver can know the required operation amount specifically as a numerical value.

[0028] Figure 4 shows another example of visual support. (1) is an example of a display using bars. The left dotted line indicates the maximum tire angle when the tires of the remotely driven vehicle are turned to the left, the right dotted line indicates the maximum tire angle when the tires of the remotely driven vehicle are turned to the right, and the middle dotted line indicates the neutral position of the tires of the remotely driven vehicle. The actual tire angle is shown as the position of the solid line of the bar, and the amount of change in the target tire angle from the neutral position is shown by the diagonal lines. The left or right end of the diagonal line bar is the target tire angle. The remote driver can operate the steering wheel and other operating systems according to the display to match the actual tire angle with the target tire angle.

[0029] FIG. 4 (2) is an example of a visual support that displays the amount of rotation. The actual tire angle is indicated by a solid line, and the target tire angle is indicated by a dashed line, and the amount of rotation is indicated by the dotted line. The dotted line indicates the neutral position of the tire. The position of the dashed line changes depending on the steering angle input to the remote cockpit. The required amount of rotation is displayed by an arrow, and the remote driver can match the actual tire angle with the target tire angle by operating the steering wheel or other control system so as to satisfy the amount of rotation displayed by the arrow. At this time, if the remote driver operates the steering wheel or other control system in the opposite direction, the display indicating the target tire angle may be changed to a flashing or highlighted color to emphasize the display.

[0030] The above-described visual supports may be selected either alone or in combination. The remote driver can easily make the actual tire angle coincide with the target tire angle by operating the steering wheel or other operating systems in accordance with the visual supports.

[0031] Next, an example of haptic support in the case where the operation system for manipulating the tire angle is a steering wheel will be described. The haptic support is provided by applying a reaction force to the steering wheel. When the remote driver operates the steering wheel in a direction that causes the target tire angle and the actual tire angle to move apart, the reaction force to the steering wheel is applied as a steering reaction force that prevents this. Conversely, when the steering operation by the remote driver is in the correct direction, that is, in a direction that brings the target tire angle and the actual tire angle closer to each other, the remote driving system does not apply a steering reaction force. This allows the remote driver to haptically know whether or not he is operating the steering wheel in the correct direction.

[0032] When the remote driving switching condition is satisfied, the initial state preparation is completed. When the initial state preparation is completed by visual support or tactile support, that is, when the target tire angle and the actual tire angle match, the remote driver may be visually or tactilely instructed. Here, when the target tire angle and the actual tire angle match, it may include when the difference between the target tire angle and the actual tire angle becomes smaller than a certain value. For example, in the case of displaying the predicted tire trajectory lines as shown in FIG. 3, when the remote driving switching condition is satisfied, the predicted trajectory lines are integrated into one, or the color of the predicted trajectory lines is changed. Alternatively, when the remote driving switching condition is satisfied, the remote driver may be instructed by applying a minute vibration to the steering wheel, the driver's seat, the brake pedal, or the like of the remote cockpit.

[0033] After the driver is notified that the initial state preparation is complete, the remotely driven vehicle is switched to a remotely driven state. If the remote driving system has a function associated with remote driving, the function associated with remote driving may be enabled at the same time when the remotely driven vehicle is switched to the remote driving state. An example of the function associated with remote driving is a virtual steering reaction force generation function. The virtual steering reaction force generation function is a function for improving the operating comfort of the remote operator. By generating a virtual steering reaction force, a reaction force simulating the state of the vehicle is applied to the remote cockpit based on information about the vehicle state, such as the vehicle speed and steering current, of the remotely driven vehicle acquired by the remote driving system.

[0034] 3. Variations As a modified example of the control device according to the present embodiment, the remote driving switching condition may be set for each control mechanism, rather than for all control mechanisms of the remotely driven vehicle. For example, the agreement between the target tire angle and the actual tire angle is set as the remote driving switching condition only for the tire control mechanism of the remotely driven vehicle. When the target tire angle and the actual tire angle match, the remote driving kit switches only the tire control mechanism of the remotely driven vehicle to the remote driving state, and causes the tire angle of the remotely driven vehicle to start following the target tire angle. Similarly, the remote driving switching condition can be set for each control mechanism other than the tire control mechanism. By setting the remote driving switching condition for each control mechanism, a smoother switch to the remote driving state can be performed.

[0035] 4.Effects As described above, the control device according to the present embodiment allows the vehicle to determine whether or not the remotely driven vehicle can be switched to a remotely driven state when a request to start remote driving is received. By performing the determination on the vehicle side, the reliability of the determination is improved, the occurrence of malfunctions is suppressed, and the vehicle can be safely switched to a remotely driven state.

Claims

1. A remote driving device for remotely driving a remote driving vehicle, The remote-controlled vehicle is configured to switch to remote control driving when the target tire angle and the actual tire angle match, The remote operation device is providing the target tire angle to the remotely driven vehicle and obtaining the actual tire angle from the remotely driven vehicle; An image acquired from an on-board camera of the remote-driven vehicle is displayed in a superimposed manner on a first trajectory line showing a predicted trajectory of the tire based on the target tire angle and a second trajectory line showing a predicted trajectory of the tire based on the actual tire angle, thereby prompting the remote driver to perform an operation to make the target tire angle and the actual tire angle coincident with each other. Remote driving device.

2. The remote driving device according to claim 1, A notification means is provided for visually or tactilely notifying that the steering device is being operated in a direction in which the first trajectory line and the second trajectory line are separated from each other. Remote driving device.

3. The remote driving device according to claim 1, a notification means for visually or tactilely notifying that the first trajectory line and the second trajectory line have substantially coincided with each other; Remote driving device.

4. A remote driving system comprising the remote driving device according to any one of claims 1 to 3 and the remote driving vehicle.

Images