Remote support system and mobile unit

Abstract

To provide a technology capable of achieving both convenience of remote support using a mobile terminal and suppression of unintended behavior of a mobile object when an abnormality occurs in an operating system using the mobile terminal.SOLUTION: A remote support system is provided with a mobile object, a mobile terminal, and a security device. The mobile terminal wirelessly transmits a control signal to the mobile object based on an operation input from an operator performing remote support of the mobile object. A safety device is physically connected to the mobile terminal and exchanges diagnostic signals with the mobile object through the mobile terminal. When no abnormality is detected in the diagnostic signal received from the safety device via the mobile terminal, the mobile object performs movement control of the mobile object based on the control signal from the mobile terminal, and does not perform movement control when an abnormality is detected.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present disclosure relates to a technique for remotely supporting a moving body using a mobile terminal.

Background Art

[0002] Patent Document 1 discloses a remote operation system that remotely operates an industrial vehicle using a mobile terminal as a remote operation device.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In the technique described in Patent Document 1, when an abnormality occurs in the operation system using the mobile terminal, unintended behavior may occur in the industrial vehicle (moving body). In order to avoid such a situation, it is conceivable to provide an emergency stop switch on the moving body. However, as a result, the operator is required to continue to be near the moving body during the execution of the remote operation, even though the mobile terminal is being used for remote operation. This is not preferable in terms of the convenience of remote support using the mobile terminal.

[0005] The present disclosure has been made in view of the above problems, and an object thereof is to provide a technique that can achieve both the convenience of remote support using a mobile terminal and the suppression of unintended behavior of a moving body when an abnormality occurs in the operation system using the mobile terminal.

Means for Solving the Problems

[0006] The remote support system described herein comprises a mobile unit, a portable terminal, and a safety device. The portable terminal wirelessly transmits control signals to the mobile unit based on operations input by an operator providing remote support to the mobile unit. The safety device is physically connected to the portable terminal and exchanges diagnostic signals with the mobile unit via the portable terminal. If the mobile unit does not detect an abnormality in the diagnostic signals received from the safety device via the portable terminal, it executes driving control of the mobile unit based on the control signals from the portable terminal; if an abnormality is detected, it does not execute driving control.

[0007] The mobile body relating to this disclosure is remotely supported based on control signals wirelessly transmitted from a mobile terminal operated by an operator. A safety device is physically connected to the mobile terminal. The safety device exchanges diagnostic signals with the mobile body via the mobile terminal. The mobile body comprises one or more processors. If the one or more processors do not detect an abnormality in the diagnostic signals received from the safety device via the mobile terminal, they execute driving control of the mobile body based on the control signals from the mobile terminal; if they detect an abnormality, they do not execute driving control. [Effects of the Invention]

[0008] According to this disclosure, a safety device that exchanges diagnostic signals with a mobile device via a mobile terminal is physically connected to the mobile terminal. The mobile device is configured not to execute driving control based on control signals from the mobile terminal if it detects an abnormality in the diagnostic signals received from the safety device via the mobile terminal. This makes it possible to achieve both the convenience of remote support using a mobile terminal and the suppression of unintended behavior of the mobile device in the event of a malfunction in the operating system using the mobile terminal. [Brief explanation of the drawing]

[0009] [Figure 1] This is a schematic diagram showing an example configuration of a remote support system according to an embodiment. [Figure 2] This diagram illustrates the exchange of information (signals) between a mobile device and a vehicle during remote support. [Figure 3] This flowchart shows a specific example of a process related to the management of vehicle driving control using a mobile terminal according to the embodiment. [Modes for carrying out the invention]

[0010] Embodiments of this disclosure will be described with reference to the attached drawings.

[0011] 1. Configuration of the remote support system Let's consider remote support for mobile objects. Remote support is a concept that includes remote monitoring, remote assistance, and remote driving. Examples of mobile objects include vehicles and robots. Vehicles may be autonomous vehicles or vehicles driven by a driver. Examples of robots include logistics robots. As an example, in the following explanation, we will consider the case where the mobile object being remotely supported is a vehicle. When generalizing, replace "vehicle" with "mobile object" in the following explanation.

[0012] Figure 1 is a schematic diagram showing an example configuration of a remote support system 1 according to an embodiment. The remote support system 1 includes a mobile terminal (or simply a terminal) 10, a dongle 20, and a vehicle 30. The vehicle 30 receives remote support based on signals wirelessly transmitted from the terminal 10.

[0013] Terminal 10 is operated by an operator who provides remote support for the vehicle 30. Remote support using terminal 10 includes remote operation of the vehicle 30 using the tilting motion of terminal 10. In the example of remote operation, the control signal S0 based on the operation input from the operator to terminal 10 for vehicle 30 driving control (motion control) corresponds to the above signal wirelessly transmitted from terminal 10 to vehicle 30. Terminal 10 is, for example, a smartphone. Alternatively, terminal 10 may be, for example, a tablet terminal or a notebook computer.

[0014] The terminal 10 includes, for example, a touch panel 11, a communication device 12, a processor 13, a storage device 14, and sensors 15. The touch panel 11 is formed on one surface of the terminal 10 and is configured to display a desired image. The touch panel 11 includes a touch sensor. The touch sensor is configured to detect whether or not an operator is touching the touch panel 11. The communication device 12 is configured to communicate with the vehicle 30 via a wireless communication network 2.

[0015] The processor 13 performs various processes for remote support, including the remote operation of the vehicle 30. The storage device 14 stores various information necessary for the processing by the processor 13. More specifically, the processor 13 performs various processes using various programs related to remote support. These programs may be stored in the storage device 14 or recorded on a computer-readable recording medium. The sensors 15 include, for example, a tilt angle sensor and a position sensor. The tilt angle sensor detects the tilt angle (attitude) of the terminal 10. The tilt angle sensor is configured to include, for example, a 6-axis gyro sensor. The tilt angle of the terminal 10 is used for the remote operation of the vehicle 30 using the action of tilting the terminal 10. The position sensor includes a GNSS (Global Navigation Satellite System) receiver and detects the position and orientation of the terminal 10.

[0016] The dongle 20 is an example of a "safety device" as described in this disclosure. The dongle 20 is used when physically connected to the terminal 10 during remote driving (remote support). The dongle 20 includes a processor that performs the "predetermined calculations" described later. More specifically, the dongle 20 is a small device that is physically connected to the terminal 10 using a connection method such as a Universal Serial Bus (USB) connection, rather than a wireless connection. For example, when remote driving is not being performed, the dongle 20 is mounted on the vehicle 30. Then, when remote driving is performed, the dongle 20 is removed from the vehicle 30 and attached to the terminal 10.

[0017] Vehicle 30 includes a communication device 31, a traveling device 32, sensors 33, and an in-vehicle electronic control unit (in-vehicle ECU) 34. The communication device 31 communicates with the outside of the vehicle 30. For example, the communication device 31 communicates with the terminal 10 via the wireless communication network 2. The traveling device 32 includes a steering device, a driving device, and a braking device. The steering device includes an electric motor for steering the wheels. The driving device includes one or both of an electric motor and an internal combustion engine for driving the vehicle 30. The braking device includes a brake actuator for braking the vehicle 30.

[0018] The sensors 33 include recognition sensors, vehicle state sensors, and position sensors. The recognition sensors recognize the situation around the vehicle 30. Examples of the recognition sensors include cameras, LIDAR (Laser Imaging Detection and Ranging), radars, etc. The vehicle state sensors detect the state of the vehicle 30. The vehicle state sensors include a speed sensor, an acceleration sensor, a yaw rate sensor, a steering angle sensor, etc. The position sensors detect the position and orientation of the vehicle 30. For example, the position sensors include a GNSS receiver.

[0019] The in-vehicle ECU 34 is a computer that controls the vehicle 30. The in-vehicle ECU 34 includes one or more processors 35 (hereinafter simply referred to as the processor 35) and one or more storage devices 36 (hereinafter simply referred to as the storage device 36). The processor 35 executes various processes related to the control of the vehicle 30. The storage device 36 stores various information necessary for the processes by the processor 35.

[0020] 2. Management of Vehicle Travel Control Using a Portable Terminal The portable terminal 10 is used to remotely control the travel of the vehicle 30. By using the operation system of the vehicle 30 with such a terminal 10, the operator can remotely control the vehicle travel by operating the terminal 10 inside or outside the vehicle 30.

[0021] FIG. 2 is a diagram for explaining the exchange of information (signals) between the mobile terminal 10 and the vehicle 30 during the execution of remote support. In order to realize remote support, the terminal 10 and the vehicle 30 exchange information. The information exchanged includes vehicle information and terminal information.

[0022] The vehicle information is various information related to remote support and is obtained by the sensors 33. The in-vehicle ECU 34 transmits the vehicle information to the terminal 10 via the communication device 31.

[0023] The terminal information is also various information related to remote support. The terminal information includes a control signal based on an operation input from the operator. In the example of remote driving, the "control signal based on an operation input from the operator" corresponds to a control signal S0 of the vehicle 30 based on an operation of the operator tilting the terminal 10 (that is, an operation directly input from the operator to the terminal 10). More specifically, the control signal S0 is, for example, a target control amount that is a target value of the control amount of at least one of the longitudinal and lateral movements of the vehicle 30. The target control amount is, for example, any one or more of a target vehicle speed, a target (longitudinal) acceleration, a target steering angle, a target yaw rate, and a target lateral acceleration. Further, the control signal S0 may be basic information for calculating the target control amount. The basic information is, for example, information on the tilt angle of the terminal 10 corresponding to the operation of tilting the terminal 10. The terminal 10 transmits the terminal information including the control signal S0 to the vehicle 30 (in-vehicle ECU 34) via the communication device 12.

[0024] In the example where the control signal S0 included in the terminal information received from the terminal 10 is a target control amount, the in-vehicle ECU 34 outputs (commands) the target control amount to the running device 32 to control the running (movement) of the vehicle 30. Alternatively, in the example where the control signal S0 is basic information for the target control amount, the in-vehicle ECU 34 calculates the target control amount based on the basic information and outputs the calculated target control amount to the running device 32 to control the running of the vehicle 30.

[0025] If an abnormality occurs in the operating system using terminal 10, unintended vehicle behavior may occur. To avoid such a situation, it is conceivable to install an emergency stop switch on vehicle 30. However, with such a measure, the operator would be required to remain near vehicle 30 while remote support is being provided, even though they are using the mobile terminal 10 for remote support. This is undesirable in terms of the convenience of remote support using terminal 10. Note that the abnormality of the "operating system using terminal 10" as referred to here includes abnormalities in terminal 10 itself, such as malfunctions, and abnormalities in communication from terminal 10 to vehicle 30.

[0026] In order to realize the emergency stop function of the vehicle 30 in view of the above-mentioned problems, the remote support system 1 according to this embodiment is equipped with a dongle 20 that is used when physically connected to the terminal 10. In the remote support system 1, driving control of the vehicle 30 using the terminal 10 is permitted on the premise that the dongle 20 is attached to the terminal 10.

[0027] The in-vehicle ECU 34 exchanges "diagnostic signals" with the dongle 20 via the terminal 10. The diagnostic signals are used for diagnosing abnormalities in the "operation system using the terminal 10". Specifically, as shown in Figure 2, the diagnostic signals include a first diagnostic signal S1 as an inquiry and a second diagnostic signal S2 as a response. The inquiry is sent from the vehicle 30 to the dongle 20 via the terminal 10. The response is sent (replied) from the dongle 20, which has received the first diagnostic signal S1, to the vehicle 30 via the terminal 10.

[0028] More specifically, the dongle 20 is associated one-to-one with the in-vehicle ECU 34. As described above, the in-vehicle ECU 34 sends an inquiry (first diagnostic signal S1) to the dongle 20 via the terminal 10. The dongle 20 generates a response (second diagnostic signal S2) by performing a "predetermined calculation" in response to the received inquiry. The dongle 20 then sends the generated response to the vehicle 30 via the terminal 10. The in-vehicle ECU 34 verifies the received response and detects an abnormality in the response (second diagnostic signal S2) based on the verification result. A specific example of the "predetermined calculation" will be described later with reference to Figure 3.

[0029] If no abnormality is detected in the second diagnostic signal S2 received from the dongle 20 via terminal 10 (i.e., the response from dongle 20 is appropriate), the in-vehicle ECU 34 (vehicle 30) executes driving control of vehicle 30 based on the control signal S0 from terminal 10. In other words, the in-vehicle ECU 34 permits driving (operation) of vehicle 30 using terminal 10. On the other hand, if an abnormality is detected (i.e., the response from dongle 20 is inappropriate), the in-vehicle ECU 34 does not execute driving control. In other words, the in-vehicle ECU 34 does not permit driving control.

[0030] Figure 3 is a flowchart illustrating a specific example of the process for managing the driving control of a vehicle 30 using a mobile terminal 10 according to an embodiment. The process in this flowchart is repeatedly executed by the in-vehicle ECU 34 (processor 35). The process in this flowchart is started, for example, when the vehicle 30 receives terminal information from the terminal 10 equipped with the dongle 20.

[0031] In step S100, the in-vehicle ECU 34 executes a process to output (send) an inquiry (first diagnostic signal S1) to the dongle 20. The dongle 20, having received the inquiry via the terminal 10, performs a "predetermined calculation" to generate a response (second diagnostic signal S2) and sends the generated response to the vehicle 30 via the terminal 10. This exchange of inquiries and responses is performed between the in-vehicle ECU 34 and the dongle 20 without being identified by the terminal 10.

[0032] The calculations performed by the dongle 20 are predetermined between the in-vehicle ECUs 34. The transmission of these calculation details from the in-vehicle ECUs 34 to the dongle 20 can be carried out, for example, as follows: The calculation details are registered in the dongle 20 when the in-vehicle ECUs 34 are shipped, or when the vehicle 30 equipped with the in-vehicle ECUs 34 is shipped. Alternatively, the calculation details may be registered in the dongle 20 when a software update is performed on the vehicle 30. Furthermore, the calculation details may be notified to the dongle 20 by the terminal 10 reading a QR (Quick Response) code (registered trademark) output to the monitor of the vehicle 30 equipped with the in-vehicle ECUs 34.

[0033] The calculation performed by the dongle 20 is executed as follows, for example: The in-vehicle ECU 34 presents a numerical value N1 as a query to the dongle 20. The dongle 20 performs a calculation on the numerical value N1 presented by the in-vehicle ECU 34 and returns the result of the calculation (numerical value N3) to the in-vehicle ECU 34. An example of the calculation content agreed upon between the in-vehicle ECU 34 and the dongle 20 is as follows: The dongle 20 calculates and returns the numerical value N3 (=N1+N2) obtained by adding a predetermined numerical value N2 to the numerical value N1 as a query.

[0034] The interaction between the in-vehicle ECU 34 and the dongle 20 is performed, for example, at a certain frequency. The dongle 20 is required to calculate and output an appropriate response to the queries from the in-vehicle ECU 34 at a certain frequency. As an example, this interaction is performed when the in-vehicle ECU 34 sequentially presents three different numerical values ​​N1 within a predetermined period, as shown below. Here, let's assume that the numerical value N2 is, for example, 3.

[0035] If the numerical value N1 in the first query is 5, and the numerical value N3 calculated by the dongle 20 is 8 (=5+3), then the response (second diagnostic signal S2) can be said to be normal. Next, if the numerical value N1 in the second query is 6, and the calculated numerical value N3 is 9 (=6+3), then the response (second diagnostic signal S2) can be said to be normal. Next, if the numerical value N1 in the third query is 7, and the calculated numerical value N3 is, for example, 11 (=7+4), then the response (second diagnostic signal S2) can be said to be abnormal because it differs from the correct answer of 10 (=7+3). Reasons why the response from the dongle 20 does not show the intended numerical value include, for example, that the value of the response is corrupted when it passes through the terminal 10, or that the response is stuck.

[0036] In step S102, following step S100, the in-vehicle ECU 34 determines whether or not it has received a response (second diagnostic signal S2) from the dongle 20 regarding the exchange described above. In cases where the exchange occurs at a certain frequency, it is determined whether or not all responses have been received. If the in-vehicle ECU 34 has not received a response (step S102; No), the process proceeds to step S104.

[0037] In step S104, the in-vehicle ECU 34 determines whether the state of no response from the dongle 20 has continued for a predetermined time or longer. If the predetermined time has not elapsed without a response (step S104; No), the process returns to step S102.

[0038] On the other hand, if the in-vehicle ECU 34 receives a response from the dongle 20 (step S102; Yes), the process proceeds to step S106. In step S106, the in-vehicle ECU 34 determines whether the response is normal or not. More specifically, in the example where the exchange is performed at a certain frequency as described above, if all responses are normal, the result of the determination is Yes. On the other hand, if one or more responses are abnormal, the result of the determination is No.

[0039] If the response is normal (step S106; Yes), in step S108, the in-vehicle ECU 34 determines that the "operation system using terminal 10" is normal. Next, in step S110, the in-vehicle ECU 34 outputs the target control amount based on the control signal S0 included in the terminal information to the driving device 32.

[0040] On the other hand, if the response is abnormal (step S106; No), that is, if the response is inappropriate, the process proceeds to step S112. Also, if the state of no response continues for a predetermined time or longer (step S104; Yes), the response is inappropriate, and the process proceeds to step S112. Reasons for the state of no response continuing for a predetermined time or longer include, for example, that communication from terminal 10 to vehicle 30 is interrupted, or that the dongle 20 is removed from terminal 10 while remote driving is being performed.

[0041] In step S112, the onboard ECU 34 determines that the "operation system using terminal 10" is abnormal. Then, in step S114, the onboard ECU 34 instructs the running gear 32 to perform an emergency stop of the vehicle 30. Alternatively, the onboard ECU 34 outputs a target control amount for the emergency stop of the vehicle 30 to the running gear 32, without relying on the control signal S0 included in the terminal information.

[0042] As described above, according to this embodiment, remote operation of the vehicle 30 is performed using a terminal 10 to which a dongle 20, which serves as an emergency stop means for the vehicle 30, is physically connected. During remote operation, the onboard ECU 34 (vehicle 30) exchanges diagnostic signals with the dongle 20 via the terminal 10. The onboard ECU 34 then permits driving control of the vehicle 30 using the terminal 10 only if the response from the dongle 20 (second diagnostic signal S2) is appropriate. This makes it possible to detect an abnormality in the operating system using the terminal 10 and bring the vehicle 30 to an emergency stop without restricting the location and movement of the operator during remote operation, compared to an example where an emergency stop switch is provided in the vehicle.

[0043] Thus, according to this embodiment, it is possible to suitably achieve both the convenience of remote driving (remote support) using the mobile terminal 10 and the suppression of unintended vehicle 30 (mobile body) behavior in the event of an abnormality in the operating system using the mobile terminal 10. [Explanation of Symbols]

[0044] 1 Remote support system, 2 Wireless communication network, 10 Mobile terminals, 20 Dongles (safety devices), 30 Vehicles (mobile devices), 34 In-vehicle ECUs, 35 Processors

Claims

1. Mobile and A portable terminal that wirelessly transmits control signals to the mobile unit based on operations input by an operator providing remote support to the mobile unit, A safety device that is physically connected to the mobile terminal and exchanges diagnostic signals with the mobile body via the mobile terminal, Equipped with, The aforementioned moving body is If no abnormality is detected in the diagnostic signal received from the safety device via the mobile terminal, the mobile body's driving control is executed based on the control signal from the mobile terminal. If the aforementioned abnormality is detected, the driving control will not be executed. The aforementioned diagnostic signal is A first diagnostic signal transmitted from the mobile device to the safety device via the mobile terminal, A second diagnostic signal transmitted from the safety device to the mobile body via the mobile terminal, Includes, The safety device generates a second diagnostic signal by performing a predetermined calculation on the received first diagnostic signal. The mobile unit verifies the received second diagnostic signal and detects the abnormality based on the result of the verification. A remote support system characterized by the following features.

2. If the mobile unit does not receive the diagnostic signal from the safety device via the mobile terminal, it will not perform the driving control. The remote support system according to feature 1.

3. The mobile terminal generates the control signal based on the operation directly input to the mobile terminal by the operator. The remote support system according to feature 1.

Images