Remote control device and remote control system

Abstract

To provide a remote operation device and a remote operation system that enable an operator who remotely operates a mobile body to easily and accurately grasp the actual movement of the mobile body during steering.SOLUTION: A remote operation device 20 includes: a steering unit 36 for steering a mobile body 12; an acquisition unit 24 for acquiring an amount of communication delay from an amount of steering of the steering unit 36 when the steering unit 36 is steered and an actual amount of steering of the mobile body 12; and a display unit 22 for displaying the steering information for steering the steering unit 36 based on the amount of communication delay acquired by the acquisition unit 24. A remote operation system 10 includes the mobile body 12 and the remote operation device 20 for remotely operating the mobile body 12.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to a remote control device and a remote control system. [Background technology]

[0002] 2. Description of the Related Art A remote control system including a moving body and a remote control device that controls the moving body from outside the moving body has been conventionally known (see, for example, Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2017-107374 A Summary of the Invention [Problem to be solved by the invention]

[0004] However, in a remote control system in which a moving body is remotely controlled from a cockpit installed outside the moving body, a communication delay between the cockpit and the moving body causes a discrepancy between the steering timing of the operator at the cockpit and the steering timing of the moving body that is actually traveling. Therefore, it is difficult for the operator to grasp the relationship between the steering at the cockpit and the actual movement of the moving body. As a result, the operator may make unnecessary steering corrections to the moving body, which may cause the behavior of the moving body to become unstable.

[0005] SUMMARY OF THE PRESENT DISCLOSURE An object of the present invention is to provide a remote control device and a remote control system that enable an operator who remotely controls a moving object to easily and accurately grasp the actual movement of the moving object when steering. [Means for solving the problem]

[0006] In order to achieve the above object, the present invention No. 1 Aspects ofThe remote control device includes a steering unit that steers a moving body, an acquisition unit that acquires a communication delay amount from the steering amount of the steering unit when the steering unit is steered and the actual steering amount of the moving body, and a display unit that displays steering information for steering the steering unit based on the communication delay amount acquired by the acquisition unit.

[0007] No. 1 Aspects of According to the invention, the steering information for steering the steering unit is displayed on the display unit based on the communication delay amount acquired by the acquisition unit. Therefore, even if a difference occurs between the steering timing of the operator at the cockpit and the steering timing of the moving body that is actually traveling due to a communication delay between the cockpit and the moving body, the operator who remotely operates the moving body can easily accurately grasp the actual movement of the moving body when steering by looking at the steering information displayed on the display unit.

[0008] Also, The present invention relates to 2 Aspects of The remote control device is No. 1 Aspects of In the remote control device, the steering information is a trajectory line of the moving body.

[0009] No. 2 Aspects of According to the invention, the steering information for steering the steering unit is a trajectory line, so that an operator who remotely operates the moving body can easily intuitively and accurately grasp the actual movement of the moving body when steering, compared to a case where the steering information is not linear.

[0010] The present invention also relates to No. 3 Aspects of The remote control system includes a moving body, a steering unit for steering the moving body, an acquisition unit for acquiring a communication delay amount from the steering amount of the steering unit when the steering unit is steered and the actual steering amount of the moving body, and a display unit for displaying steering information for steering the steering unit based on the communication delay amount acquired by the acquisition unit, and is equipped with a remote control device for remotely operating the moving body.

[0011] No.3 Aspects of According to the invention, a remote control device that remotely controls a moving body has a display unit that displays steering information for steering the steering unit based on the communication delay amount acquired by the acquisition unit. Therefore, even if a delay in communication between the cockpit and the moving body causes a discrepancy between the steering timing of the operator at the cockpit and the steering timing of the moving body that is actually traveling, the operator who remotely controls the moving body can easily accurately grasp the actual movement of the moving body when steering by looking at the steering information displayed on the display unit. Effect of the Invention

[0012] As described above, according to the present invention, an operator who remotely operates a moving object can easily and accurately grasp the actual movement of the moving object when steering. [Brief description of the drawings]

[0013] [Figure 1] 1 is a block diagram showing a configuration of a remote control system according to an embodiment of the present invention. [Diagram 2] 11 is an explanatory diagram showing a trajectory line displayed on a display unit of the remote control system according to the embodiment. FIG. [Diagram 3] 4 is a flowchart showing a processing process on the remote cockpit side of the remote control system according to the present embodiment. [Figure 4] 11 is a graph showing a deviation caused by a communication delay between the steering timing on the remote cockpit side and the steering timing on the vehicle side of the remote control system according to this embodiment, and the video feedback displayed on the display unit. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0014] Hereinafter, an embodiment of the present invention will be described in detail with reference to the drawings. As shown in Fig. 1, a remote control system 10 according to this embodiment includes a vehicle 12 as an example of a moving body, and a remote cockpit 20 as a remote control device having a cockpit in which a driver (operator) who operates the vehicle 12 remotely from outside the vehicle sits.

[0015] The vehicle 12 is equipped with a control unit 14 electrically connected to a drive actuator 15. That is, the control unit 14 receives operation instructions (control data) transmitted from a remote cockpit 20, and drives the drive actuator 15 based on the instructions to control actual driving, such as operation of the accelerator pedal (driving), operation of the steering wheel (steering), and operation of the brake pedal (stopping).

[0016] The vehicle 12 also includes a communication unit 16 having a receiving unit that receives operation instructions (control data) from a remote cockpit 20 via an antenna 18, and a transmitting unit that transmits steering information required for remote driving to the remote cockpit 20. The communication unit 16 is electrically connected to the control unit 14.

[0017] The vehicle 12 is also equipped with a camera 17 that captures images of the surroundings of the vehicle 12. The cameras 17 are provided at a plurality of appropriate positions on the vehicle 12. Note that the steering information required for remote driving, which is transmitted by the transmitting unit in the communication unit 16, also includes images (image data) captured by the camera 17. The camera 17 is also electrically connected to the control unit 14.

[0018] The remote cockpit 20 is equipped with a communication unit 26 having a receiving unit that receives steering information necessary for remote driving from the communication unit 16 via an antenna 28, and a transmitting unit that transmits operation instructions (control data) from the remote cockpit 20 to the communication unit 16.

[0019] The remote cockpit 20 also includes a display unit 22 that displays actual steering information received by a communication unit 26, such as images captured by a plurality of cameras 17 provided on the vehicle 12. The display unit 22 is composed of a plurality of displays and LED devices, and is also configured to display various meters that show the state of the vehicle 12, such as a speedometer 40 (see FIG. 2) that shows the speed of the vehicle 12.

[0020] That is, the display unit 22 is capable of displaying the state of the vehicle 12, such as when it is moving or when it is stopped, in addition to the display of various meters such as the speedometer 40, and is capable of visually notifying the driver of the state of the vehicle 12 that is actually moving on the roadway. The display unit 22 is also capable of displaying steering information for steering the steering wheel 36, which will be described later.

[0021] The remote cockpit 20 also includes a control unit 24 electrically connected to the display unit 22 and the communication unit 26. The control unit 24 transmits operation instructions (control data) on the remote cockpit 20 side to the vehicle 12 side, receives steering information required for remote driving from the vehicle 12 side, and controls the remote cockpit 20 side by displaying the steering information on the display unit 22, for example.

[0022] The control unit 24 is an example of an acquisition unit that acquires the amount of communication delay from the steering amount of the steering wheel 36 when the steering wheel 36 is steered and the actual steering amount of the vehicle 12. Based on the amount of communication delay acquired by the control unit 24, steering information for steering the steering wheel 36 is displayed on the display unit 22.

[0023] The control unit 24 is also electrically connected to an operating device 30 for driving the vehicle 12. The operating device 30 has an accelerator pedal 32 for the driver to operate the accelerator, a brake pedal 34 for the driver to operate the brakes, a steering wheel 36 for the driver to operate the steering wheel, and switches (SW) 38 for the driver to perform other necessary operations.

[0024] That is, the control unit 24 transmits various actual driving operations performed by the driver on the remote cockpit 20 side to the vehicle 12 side via the communication unit 26. The steering wheel 36 is an example of a steering unit that steers the vehicle 12. The steering wheel 36 uses steer-by-wire (electronically controlled steering mechanism) technology, allowing the driver to obtain steering feedback (steering reaction force).

[0025] Here, the steering information displayed on the display unit 22 will be described.

[0026] 2, the display unit 22 displays a road D on which the vehicle 12 is traveling. A trajectory line 42, which is a predicted trajectory indicating the direction in which the vehicle 12 should proceed, is superimposed and displayed at the center position in the road width direction of the road D. The trajectory line 42 is steering information for steering the steering wheel 36 based on the communication delay amount (steering presentation amount) calculated by the control unit 24.

[0027] The trajectory line 42 is a line displayed on the display unit 22 after being transformed into a straight line or a curve according to the condition of the road D, and the one shown in the figure is a solid line of a predetermined length. Note that the trajectory line 42 is not limited to the one shown in the figure, and the type, length, width, color, and shade of the line are arbitrary. Also, the type, width, color, and the like of the trajectory line 42 may be configured to be preset by the driver, or may be configured to be automatically set by the control unit 24.

[0028] The amount of communication delay is calculated by the control unit 24 by monitoring and comparing the amount of steering when the steering wheel 36 on the remote cockpit 20 side is steered with the amount of steering of the actual steering wheel on the vehicle 12 side at that time, and from at least one of the change tendency (rate of change) and the average value.

[0029] In other words, if there is a communication delay between the remote cockpit 20 and the vehicle 12, it takes a certain amount of time from when the steering wheel 36 on the remote cockpit 20 side is steered until the steering wheel on the actual vehicle 12 side is steered and the amount of steering is fed back to the remote cockpit 20 side.

[0030] The control unit 24 estimates the actual steering amount of the steering wheel on the vehicle 12 side due to steering of the steering wheel 36 on the remote cockpit 20 side, taking into account the communication delay amount, which is the specified time, calculates a steering suggestion amount, which is the steering amount that the driver should steer with the steering wheel 36, and displays a trajectory line 42 on the display unit 22 based on the steering suggestion amount.

[0031] That is, the trajectory line 42 is displayed on the display unit 22 in a state where the difference between the steering timing of the steering wheel 36 on the remote cockpit 20 side and the steering timing of the steering wheel on the actual vehicle 12 side is absorbed. Therefore, the driver can remotely control the vehicle 12 by simply driving while trusting the trajectory line 42.

[0032] Next, the operation of the remote cockpit 20 and the remote control system 10 according to this embodiment configured as above will be described.

[0033] In Fig. 4, the steering amount of the steering wheel 36 on the remote cockpit 20 side is shown with a solid line versus time, and the steering amount of the steering wheel on the vehicle 12 side is shown with a dashed line versus time. Furthermore, in Fig. 4, the video feedback of the steering wheel on the vehicle 12 side is shown with a dashed line. The video feedback is displayed on the display unit 22.

[0034] 4(A), due to a communication delay between the remote cockpit 20 and the vehicle 12, the steering timing from the neutral position of the steering wheel on the vehicle 12 side is delayed by time T1 compared to the steering timing from the neutral position of the steering wheel 36 on the remote cockpit 20 side. In other words, time T1 is the time until the steering of the steering wheel 36 on the remote cockpit 20 side is reflected in the steering of the steering wheel on the vehicle 12 side.

[0035] And, because the image data has a larger capacity than the control data, the communication delay causes the image feedback of the steering wheel of the vehicle 12 (the timing of steering from the neutral position) displayed on the display unit 22 of the remote cockpit 20 to be delayed by time T2 compared to the timing of steering from the neutral position of the steering wheel of the vehicle 12. In other words, time T2 is the time until the steering of the steering wheel of the vehicle 12 is reflected in the image feedback.

[0036] Therefore, even if the steering of the steering wheel 36 on the remote cockpit 20 side is completed (even if the steering wheel 36 on the remote cockpit 20 side reaches the steering amount shown at time Te), the steering of the steering wheel on the vehicle 12 side in the video feedback displayed on the display unit 22 is in an incomplete state (the steering amount of the steering wheel on the vehicle 12 side in the video feedback shown at time Te has not reached the steering amount of the steering wheel 36 on the remote cockpit 20 side).

[0037] In other words, it takes time T1+time T2 for the steering of the steering wheel 36 on the remote cockpit 20 side to be reflected in the video feedback. Therefore, the driver on the remote cockpit 20 side may want to further steer the steering wheel 36 when he or she sees the video feedback displayed on the display unit 22, which may cause the behavior of the actual vehicle 12 to become unstable.

[0038] The same is true when returning the steering wheel 36 on the remote cockpit 20 side to the neutral position. That is, as shown in Fig. 4(B), due to a communication delay between the remote cockpit 20 and the vehicle 12, the timing of steering the steering wheel on the vehicle 12 side to the neutral position is delayed by a time T1 compared to the timing of steering the steering wheel 36 on the remote cockpit 20 side to the neutral position.

[0039] Furthermore, since image data has a larger capacity than control data, the communication delay causes the image feedback of the steering wheel on the vehicle 12 side (the timing of steering to the neutral position) displayed on the display unit 22 on the remote cockpit 20 side to be delayed by a time T2 compared to the actual timing of steering the steering wheel on the vehicle 12 side to the neutral position.

[0040] Therefore, even if the steering wheel 36 on the remote cockpit 20 side has been completely steered to the neutral position (even if the steering wheel 36 on the remote cockpit 20 side has reached the neutral position shown at time Tc), the steering of the steering wheel on the vehicle 12 side in the video feedback displayed on the display unit 22 is in an incomplete state (the steering amount of the steering wheel on the vehicle 12 side in the video feedback shown at time Tc has not yet reached the neutral position).

[0041] That is, in this case, it takes time T1+time T2 for the steering of the steering wheel 36 on the remote cockpit 20 side to be reflected in the video feedback. Therefore, the driver on the remote cockpit 20 side may want to further steer the steering wheel 36 when he or she sees the video feedback displayed on the display unit 22, which may cause the behavior of the actual vehicle 12 to become unstable.

[0042] Therefore, in the remote cockpit 20 and remote operation system 10 of this embodiment, as described above, the display unit 22 displays the trajectory line 42, which is steering information for steering the steering wheel 36, so that the driver can easily and accurately visually grasp the actual state (movement) of the vehicle 12.

[0043] 3, in the remote cockpit 20, the control unit 24 first acquires the steering amount of the steering wheel 36 in the remote cockpit 20 (step S1). That is, it is determined whether the steering wheel 36 in the remote cockpit 20 has been steered from the neutral position (step S2).

[0044] If it is determined that the steering wheel 36 on the remote cockpit 20 side has not been steered from the neutral position (no change), the current trajectory line 42 continues to be displayed on the display unit 22 (step S3), and the process returns to step S1 again.

[0045] When it is determined that the steering wheel 36 on the remote cockpit 20 side has been steered from the neutral position (changed), the amount of steering from the neutral position of the steering wheel on the vehicle 12 side is acquired (step S4). Then, the communication delay amount from when the steering wheel 36 on the remote cockpit 20 side is steered to when the steering wheel on the vehicle 12 side is steered and the amount of steering is fed back to the remote cockpit 20 side is measured (step S5).

[0046] In addition, this communication delay amount may be calculated in advance by the control unit 24 from at least one of the change tendency (rate of change) and average value by monitoring and comparing the steering amount when the steering wheel 36 on the remote cockpit 20 side is steered and the actual steering amount of the steering wheel on the vehicle 12 side at that time.

[0047] When the communication delay is, for example, less than 200 ms, there is no difference between the steering amount when the steering wheel 36 on the remote cockpit 20 side is steered and the steering amount on the vehicle 12 side at that time, so the steering amount of the steering wheel 36 on the remote cockpit 20 side becomes the steering amount of the steering wheel on the vehicle 12 side as it is, and the steering suggested amount is calculated based on that (step S7). Then, the trajectory line 42 is displayed with the calculated steering suggested amount (step S8).

[0048] When the communication delay is, for example, 200 ms or more, there is a difference between the steering amount when the steering wheel 36 on the remote cockpit 20 side is steered and the steering amount of the steering wheel on the vehicle 12 side at that time, so the steering amount of the steering wheel on the vehicle 12 side is estimated according to the communication delay amount, and a steering suggestion amount is calculated (step S9). Then, the trajectory line 42 is displayed with the calculated steering suggestion amount (step S8). In this embodiment, the above steps are repeatedly executed, and the trajectory line 42 is always displayed on the display unit 22.

[0049] Therefore, even if a communication delay between the remote cockpit 20 and the vehicle 12 causes a discrepancy between the steering timing (steering amount) of the driver at the remote cockpit 20 and the steering timing (steering amount) of the vehicle 12 that is actually traveling, the driver can properly drive the actual vehicle 12 by looking at the trajectory line 42 displayed on the display unit 22 (trusting the trajectory line 42) and steering the steering wheel 36.

[0050] That is, even if a communication delay occurs between the driver who remotely operates the vehicle 12 and the actual vehicle 12, the driver can easily and accurately grasp the movement of the vehicle 12 from the trajectory line 42. Therefore, the driver can be restrained or prevented from making unnecessary steering corrections to the vehicle 12, and the behavior of the vehicle 12 can be restrained or prevented from becoming unstable.

[0051] In addition, as shown in FIG. 2, since the steering information for steering the steering wheel 36 is in the form of a trajectory line 42, the driver who remotely operates and drives the vehicle 12 can intuitively, easily, and accurately grasp the actual movement of the vehicle 12 when steering the steering wheel 36, compared to a case in which the steering information is not linear.

[0052] In this way, according to the remote cockpit 20 and remote operation system 10 of this embodiment, the driver on the remote cockpit 20 side can easily, accurately, and even intuitively grasp the state (movement) of the vehicle 12 visually from the display unit 22, regardless of communication delays (delay or stop of the image, reduced resolution, or reduced frame rate).

[0053] Although the remote cockpit 20 and the remote control system 10 as the remote control device according to the present embodiment have been described above with reference to the drawings, the remote cockpit 20 and the remote control system 10 according to the present embodiment are not limited to those shown in the drawings, and can be appropriately modified in design within the scope of the present invention. For example, the moving body is not limited to the vehicle 12.

[0054] 3, the communication delay threshold is not limited to 200 ms. Generally, if the communication delay threshold is 500 ms or more, remote driving becomes impossible, so the communication delay threshold may be a value smaller than that. Furthermore, the steering information displayed on the display unit 22 is not limited to the trajectory line 42. The steering information displayed on the display unit 22 may be, for example, numbers, etc.

[0055] That is, the display unit 22 may be configured to display 0 when the road D is straight (the steering wheel 36 is in the neutral position), and display numbers 1 to 5 according to the degree of curvature of the road D. In this case, it is preferable that the number 1 to 5 is followed by the letter R when turning right, and the number 1 to 5 is followed by the letter L when turning left. [Explanation of symbols]

[0056] 10 Remote Control System 12 Vehicles (moving objects) 20 Remote cockpit (remote control device) 22 Display section 24 Control unit (acquisition unit) 36 Steering wheel (steering section) 42 Locus line

Claims

1. A steering unit for steering a remote moving object; an acquisition unit that calculates and acquires a communication delay amount from at least one of a rate of change and an average value between a steering amount of the steering unit when the steering unit is steered and an actual steering amount of the moving body; a display unit that displays steering information for steering the steering unit based on the communication delay amount acquired by the acquisition unit; Equipped with A remote control device in which the steering information is a trajectory line that indicates the direction in which the moving body should move as a predicted trajectory while absorbing the difference between the steering timing of the steering section and the actual steering timing of the moving body.

2. A remote moving object; a remote control device for remotely controlling the moving body, the remote control device comprising: a steering unit for steering the moving body; an acquisition unit for calculating and acquiring a communication delay amount from at least one of a rate of change and an average value of a steering amount of the steering unit when the steering unit is steered and an actual steering amount of the moving body; and a display unit for displaying steering information for steering the steering unit based on the communication delay amount acquired by the acquisition unit; Equipped with A remote control system in which the steering information is a trajectory line that indicates the direction in which the moving body should move as a predicted trajectory while absorbing the difference between the steering timing of the steering unit and the actual steering timing of the moving body.

Images