Remote support system and mobile object

Abstract

To provide a technology capable of achieving both convenience of remote support using a mobile terminal and suppression of unintended behavior of a mobile object when an abnormality occurs in an operating system using the mobile terminal.SOLUTION: A remote support system is provided with a mobile object, a mobile terminal, and a security device. The mobile terminal wirelessly transmits a control signal to the mobile object based on an operation input from an operator performing remote support of the mobile object. A safety device is physically connected to the mobile terminal and exchanges diagnostic signals with the mobile object through the mobile terminal. When no abnormality is detected in the diagnostic signal received from the safety device via the mobile terminal, the mobile object performs movement control of the mobile object based on the control signal from the mobile terminal, and does not perform movement control when an abnormality is detected.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present disclosure relates to a technique for remotely supporting a moving object using a mobile terminal. [Background technology]

[0002] Patent Document 1 discloses a remote control system that uses a mobile terminal as a remote control device to remotely control an industrial vehicle. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2019-077528 A Summary of the Invention [Problem to be solved by the invention]

[0004] In the technology described in Patent Document 1, when an abnormality occurs in the operation system using the mobile terminal, unintended behavior may occur in the industrial vehicle (mobile body). To avoid such a situation, it is possible to provide an emergency stop switch on the mobile body. However, as a result, the operator is required to remain near the mobile body while the remote operation is being performed, even though the operator is using the mobile terminal for remote operation. This is not desirable in terms of the convenience of remote support using a mobile terminal.

[0005] The present disclosure has been made in consideration of the above-mentioned problems, and aims to provide technology that can achieve both the convenience of remote support using a mobile terminal and the suppression of unintended behavior of a moving object when an abnormality occurs in the operation system using the mobile terminal. [Means for solving the problem]

[0006] The remote support system according to the present disclosure includes a moving body, a mobile terminal, and a safety device. The mobile terminal wirelessly transmits a control signal to the moving body based on an operation input by an operator who remotely supports the moving body. The safety device is physically connected to the mobile terminal and exchanges a diagnostic signal with the moving body via the mobile terminal. When the moving body does not detect an abnormality in the diagnostic signal received from the safety device via the mobile terminal, the moving body executes travel control of the moving body based on the control signal from the mobile terminal, and when the mobile body detects an abnormality, the traveling control is not executed.

[0007] A mobile body according to the present disclosure is remotely supported based on a control signal wirelessly transmitted from a mobile terminal operated by an operator. A safety device is physically connected to the mobile terminal. The safety device exchanges diagnostic signals with the mobile body via the mobile terminal. The mobile body includes one or more processors. When the one or more processors do not detect an abnormality in the diagnostic signal received from the safety device via the mobile terminal, the one or more processors execute driving control of the mobile body based on the control signal from the mobile terminal, and when an abnormality is detected, the one or more processors do not execute driving control. Effect of the Invention

[0008] According to the present disclosure, a safety device that exchanges diagnostic signals with a mobile body via a mobile terminal is physically connected to the mobile terminal. The mobile body is configured not to execute travel control of the mobile body based on a control signal from the mobile terminal when an abnormality is detected in the diagnostic signal received from the safety device via the mobile terminal. This makes it possible to achieve both the convenience of remote support using the mobile terminal and the suppression of unintended behavior of the mobile body when an abnormality occurs in an operation system using the mobile terminal. [Brief description of the drawings]

[0009] [Figure 1] 1 is a schematic diagram illustrating a configuration example of a remote support system according to an embodiment; [Diagram 2] FIG. 2 is a diagram for explaining the exchange of information (signals) between a mobile terminal and a vehicle while remote support is being performed. [Diagram 3] 10 is a flowchart showing a specific example of a process related to management of vehicle driving control using a mobile terminal according to an embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] Embodiments of the present disclosure will be described with reference to the accompanying drawings.

[0011] 1. Configuration of remote support system Consider remote support of a moving object. Remote support is a concept including remote monitoring, remote assistance, and remote driving. Examples of moving objects include vehicles and robots. The vehicle may be an autonomous vehicle or a vehicle driven by a driver. Examples of robots include logistics robots. As an example, in the following description, consider a case where the moving object to be remotely supported is a vehicle. When generalizing, the word "vehicle" in the following description should be read as "moving object".

[0012] 1 is a schematic diagram showing a configuration example of a remote support system 1 according to an embodiment. The remote support system 1 includes a mobile terminal (or simply a terminal) 10, a dongle 20, and a vehicle 30. The vehicle 30 receives remote support based on a signal wirelessly transmitted from the terminal 10.

[0013] The terminal 10 is operated by an operator who provides remote support for the vehicle 30. Remote support using the terminal 10 includes remote driving of the vehicle 30 using an action of tilting the terminal 10. In an example of remote driving, a control signal S0 based on an operation input from the operator to the terminal 10 for driving control (motion control) of the vehicle 30 corresponds to the above-mentioned signal wirelessly transmitted from the terminal 10 to the vehicle 30. The terminal 10 is, for example, a smartphone. Alternatively, the terminal 10 may be, for example, a tablet terminal or a notebook terminal.

[0014] The terminal 10 includes, for example, a touch panel 11, a communication device 12, a processor 13, a storage device 14, and sensors 15. The touch panel 11 is formed on one plate surface of the terminal 10 and is configured to be capable of displaying a desired image. The touch panel 11 includes a touch sensor. The touch sensor is configured to be capable of detecting whether or not an operator has touched the touch panel 11. The communication device 12 is configured to be capable of communicating with the vehicle 30 via the wireless communication network 2.

[0015] The processor 13 executes various processes for remote support including remote driving of the vehicle 30. The storage device 14 stores various information necessary for processing by the processor 13. More specifically, the processor 13 executes various processes using various programs related to remote support. The various programs may be stored in the storage device 14, or may be recorded in a computer-readable recording medium. The sensors 15 include, for example, an inclination angle sensor and a position sensor. The inclination angle sensor detects the inclination angle (attitude) of the terminal 10. The inclination angle sensor includes, for example, a six-axis gyro sensor. The inclination angle of the terminal 10 is used for remote driving of the vehicle 30 using the action of tilting the terminal 10. The position sensor includes a GNSS (Global Navigation Satellite System) receiver and detects the position and orientation of the terminal 10.

[0016] The dongle 20 corresponds to an example of a "safety device" according to the present disclosure. The dongle 20 is used in a state where it is physically connected to the terminal 10 during remote driving (remote support). The dongle 20 includes a processor that executes "predetermined calculations" described below. More specifically, the dongle 20 is a small device that is physically connected to the terminal 10 using a connection method such as a universal serial bus (USB) connection, rather than a wireless connection. For example, the dongle 20 is mounted on the vehicle 30 while remote driving is not being performed. Then, during remote driving, the dongle 20 is removed from the vehicle 30 and attached to the terminal 10.

[0017] The vehicle 30 includes a communication device 31, a driving device 32, sensors 33, and an on-vehicle electronic control unit (on-vehicle ECU) 34. The communication device 31 communicates with the outside of the vehicle 30. For example, the communication device 31 communicates with the terminal 10 via the wireless communication network 2. The driving device 32 includes a steering device, a drive device, and a braking device. The steering device includes an electric motor that steers the wheels. The drive device includes one or both of an electric motor and an internal combustion engine for driving the vehicle 30. The braking device includes a brake actuator for braking the vehicle 30.

[0018] The sensors 33 include a recognition sensor, a vehicle state sensor, and a position sensor. The recognition sensor recognizes the situation around the vehicle 30. Examples of the recognition sensor include a camera, a LIDAR (Laser Imaging Detection and Ranging), a radar, and the like. The vehicle state sensor detects the state of the vehicle 30. The vehicle state sensor includes a speed sensor, an acceleration sensor, a yaw rate sensor, a steering angle sensor, and the like. The position sensor detects the position and orientation of the vehicle 30. For example, the position sensor includes a GNSS receiver.

[0019] The on-vehicle ECU 34 is a computer that controls the vehicle 30. The on-vehicle ECU 34 includes one or more processors 35 (hereinafter simply referred to as the processor 35) and one or more storage devices 36 (hereinafter simply referred to as the storage devices 36). The processor 35 executes various processes related to the control of the vehicle 30. The storage device 36 stores various information required for the processes by the processor 35.

[0020] 2. Managing vehicle driving control using mobile devices The mobile terminal 10 is used to remotely control the traveling of the vehicle 30. By utilizing the operation system of the vehicle 30 using such a terminal 10, an operator can operate the terminal 10 inside or outside the vehicle 30 to remotely control the traveling of the vehicle.

[0021] 2 is a diagram for explaining the exchange of information (signals) between the mobile terminal 10 and the vehicle 30 during remote support. To realize the remote support, the terminal 10 and the vehicle 30 exchange information. The exchanged information includes vehicle information and terminal information.

[0022] The vehicle information is various types of information related to remote support, and is obtained by the sensors 33. The in-vehicle ECU 34 transmits the vehicle information to the terminal 10 via the communication device 31.

[0023] The terminal information is also various information related to remote support. The terminal information includes a control signal based on an operation input by an operator. In the example of remote driving, the "control signal based on an operation input by an operator" corresponds to a control signal S0 of the vehicle 30 based on an operation of the operator tilting the terminal 10 (that is, an operation directly input to the terminal 10 by the operator). More specifically, the control signal S0 is, for example, a target control amount that is a target value of a control amount of at least one of the longitudinal and lateral movements of the vehicle 30. The target control amount is, for example, any one or more of a target vehicle speed, a target (longitudinal) acceleration, a target steering angle, a target yaw rate, and a target lateral acceleration. The control signal S0 may also be basic information for calculating the target control amount. The basic information is, for example, information on the tilt angle of the terminal 10 according to the operation of tilting the terminal 10. The terminal 10 transmits the terminal information including the control signal S0 to the vehicle 30 (the on-board ECU 34) via the communication device 12.

[0024] In an example where the control signal S0 included in the terminal information received from the terminal 10 is a target control amount, the in-vehicle ECU 34 outputs (commands) the target control amount to the traveling device 32 to control the traveling (motion) of the vehicle 30. Alternatively, in an example where the control signal S0 is basic information of the target control amount, the in-vehicle ECU 34 calculates the target control amount based on the basic information, and outputs the calculated target control amount to the traveling device 32 to control the traveling of the vehicle 30.

[0025] When an abnormality occurs in the operation system using the terminal 10, unintended vehicle behavior may occur. To avoid such a situation, it is possible to provide an emergency stop switch in the vehicle 30. However, with such a measure, the operator is required to remain near the vehicle 30 while the remote support is being performed, even though the operator is using the mobile terminal 10 for the remote support. This is not preferable in terms of the convenience of the remote support using the terminal 10. Note that the abnormality of the "operation system using the terminal 10" here includes an abnormality of the terminal 10 itself, such as a breakdown, and an abnormality in communication from the terminal 10 to the vehicle 30.

[0026] In order to realize an emergency stop function of the vehicle 30 in consideration of the above-mentioned problems, the remote support system 1 according to this embodiment includes a dongle 20 that is used in a state where it is physically connected to the terminal 10. In the remote support system 1, driving control of the vehicle 30 using the terminal 10 is permitted on the premise that the dongle 20 is attached to the terminal 10.

[0027] The in-vehicle ECU 34 exchanges a "diagnostic signal" with the dongle 20 via the terminal 10. The diagnostic signal is a signal used for diagnosing an abnormality in the "operation system using the terminal 10." Specifically, as shown in FIG. 2, the diagnostic signal includes a first diagnostic signal S1 as an inquiry and a second diagnostic signal S2 as an answer. The inquiry is transmitted from the vehicle 30 to the dongle 20 via the terminal 10. The answer is transmitted (returned) from the dongle 20, which has received the first diagnostic signal S1, to the vehicle 30 via the terminal 10.

[0028] More specifically, dongle 20 is in one-to-one correspondence with in-vehicle ECU 34. As described above, in-vehicle ECU 34 transmits an inquiry (first diagnostic signal S1) to dongle 20 via terminal 10. Dongle 20 generates a response (second diagnostic signal S2) by performing a "predetermined calculation" on the received inquiry. Then, dongle 20 transmits the generated response to vehicle 30 via terminal 10. In-vehicle ECU 34 verifies the received response and detects an abnormality in the response (second diagnostic signal S2) based on the result of the verification. A specific example of the "predetermined calculation" will be described later with reference to FIG. 3.

[0029] If no abnormality is detected in the second diagnosis signal S2 received from the dongle 20 via the terminal 10 (i.e., the response from the dongle 20 is appropriate), the in-vehicle ECU 34 (vehicle 30) executes driving control of the vehicle 30 based on the control signal S0 from the terminal 10. In other words, the in-vehicle ECU 34 permits driving (operation) of the vehicle 30 using the terminal 10. On the other hand, if the abnormality is detected (i.e., the response from the dongle 20 is inappropriate), the in-vehicle ECU 34 does not execute driving control. In other words, the in-vehicle ECU 34 does not permit driving control.

[0030] 3 is a flowchart showing a specific example of a process related to management of driving control of the vehicle 30 using the mobile terminal 10 according to the embodiment. The process of this flowchart is repeatedly executed by the in-vehicle ECU 34 (processor 35). The process of the flowchart is started, for example, when the vehicle 30 receives terminal information from the terminal 10 equipped with the dongle 20.

[0031] In step S100, the in-vehicle ECU 34 executes a process of outputting (transmitting) an inquiry (first diagnostic signal S1) to the dongle 20. The dongle 20, which receives the inquiry via the terminal 10, performs a "predetermined calculation" to generate a response (second diagnostic signal S2), and transmits the generated response to the vehicle 30 via the terminal 10. Such exchange of inquiry and response is executed between the in-vehicle ECU 34 and the dongle 20, without being identified by the terminal 10.

[0032] The content of the calculation performed by the dongle 20 is determined in advance between the in-vehicle ECU 34 and the dongle 20. The content of the calculation can be transmitted from the in-vehicle ECU 34 to the dongle 20, for example, as follows. That is, the content of the calculation is registered in the dongle 20 when the in-vehicle ECU 34 is shipped, or when the vehicle 30 mounting the in-vehicle ECU 34 is shipped. Alternatively, the content of the calculation may be registered in the dongle 20 when a software update of the vehicle 30 is performed. Furthermore, the content of the calculation may be notified to the dongle 20 by the terminal 10 reading a QR (Quick Response) code (registered trademark) outputted on a monitor of the vehicle 30 mounting the in-vehicle ECU 34.

[0033] The calculation by the dongle 20 is executed, for example, as follows. That is, the in-vehicle ECU 34 presents a numerical value N1 as an inquiry to the dongle 20. The dongle 20 performs a calculation on the numerical value N1 presented by the in-vehicle ECU 34, and returns the result of the calculation (numerical value N3) to the in-vehicle ECU 34. An example of the calculation content agreed upon between the in-vehicle ECU 34 and the dongle 20 is as follows. That is, the dongle 20 calculates and returns a numerical value N3 (=N1+N2) obtained by adding a predetermined numerical value N2 to the numerical value N1 as an inquiry.

[0034] The communication between the in-vehicle ECU 34 and the dongle 20 is performed, for example, at a fixed frequency. The dongle 20 is required to calculate and output an appropriate response to inquiries from the in-vehicle ECU 34 at a fixed frequency. As an example, the communication is performed by the in-vehicle ECU 34 presenting three numerical values ​​N1 in sequence within a predetermined period as shown below. Here, the numerical value N2 is assumed to be 3, for example.

[0035] If the value N1 of the first inquiry is 5 and the value N3 calculated by the dongle 20 is 8 (=5+3), the answer (second diagnostic signal S2) can be said to be normal. Next, if the value N1 of the second inquiry is 6 and the value N3 calculated is 9 (=6+3), the answer (second diagnostic signal S2) can be said to be normal. Next, if the value N1 of the third inquiry is 7 and the value N3 calculated is, for example, 11 (=7+4), the answer (second diagnostic signal S2) can be said to be abnormal because it is different from the correct answer of 10 (=7+3). The reason why the answer from the dongle 20 does not show the intended value in this way can be, for example, that the value of the answer becomes distorted when the answer passes through the terminal 10, or that the answer is fixed.

[0036] In step S102 following step S100, the in-vehicle ECU 34 determines whether or not it has received a response (second diagnosis signal S2) from the dongle 20 regarding the above-mentioned exchange. In an example in which the exchange is performed at a fixed frequency, it determines whether or not all the responses have been received. If the in-vehicle ECU 34 has not received a response (step S102; No), the process proceeds to step S104.

[0037] In step S104, the in-vehicle ECU 34 determines whether or not a predetermined time period has elapsed without a response from the dongle 20. If the predetermined time period has not elapsed without a response (step S104; No), the process returns to step S102.

[0038] On the other hand, if the in-vehicle ECU 34 receives a response from the dongle 20 (step S102; Yes), the process proceeds to step S106. In step S106, the in-vehicle ECU 34 determines whether the response is normal or not. More specifically, in the example in which exchanges are performed at a fixed frequency as described above, if all the responses are normal, the determination result is Yes. On the other hand, if one or more of the responses are abnormal, the determination result is No.

[0039] If the answer is normal (step S106; Yes), in step S108, the in-vehicle ECU 34 determines that the "operation system using the terminal 10" is normal. Next, in step S110, the in-vehicle ECU 34 outputs to the traveling device 32 a target control amount based on the control signal S0 included in the terminal information.

[0040] On the other hand, if the answer is abnormal (step S106; No), that is, if the answer is inappropriate, the process proceeds to step S112. Also, if the state of no answer continues for a predetermined time or more (step S104; Yes), the answer is also inappropriate, and the process proceeds to step S112. Reasons for the state of no answer continuing for a predetermined time or more include, for example, a loss of communication from the terminal 10 to the vehicle 30, or the dongle 20 being removed from the terminal 10 while remote driving is being performed.

[0041] In step S112, the in-vehicle ECU 34 determines that the "operation system using the terminal 10" is abnormal. Next, in step S114, the in-vehicle ECU 34 instructs the traveling device 32 to make an emergency stop of the vehicle 30. Alternatively, the in-vehicle ECU 34 outputs a target control amount for an emergency stop of the vehicle 30 to the traveling device 32, regardless of the control signal S0 included in the terminal information.

[0042] As described above, according to this embodiment, remote driving of the vehicle 30 is performed using the terminal 10 to which the dongle 20, which serves as an emergency stop means for the vehicle 30, is physically connected. During remote driving, the in-vehicle ECU 34 (vehicle 30) exchanges diagnostic signals with the dongle 20 via the terminal 10. Then, the in-vehicle ECU 34 permits driving control of the vehicle 30 using the terminal 10 only when the response (second diagnostic signal S2) from the dongle 20 is appropriate. This makes it possible to detect an abnormality in the operation system using the terminal 10 and bring the vehicle 30 to an emergency stop without restricting the location and movement of the operator during remote driving, as compared to an example in which an emergency stop switch is provided in the vehicle.

[0043] In this way, according to this embodiment, it is possible to optimally achieve both the convenience of remote driving (remote support) using the mobile terminal 10 and the suppression of unintended behavior of the vehicle 30 (moving body) when an abnormality occurs in the operation system using the mobile terminal 10. [Explanation of symbols]

[0044] 1 Remote support system, 2 Wireless communication network, 10 Mobile terminal, 20 Dongle (safety device), 30 Vehicle (mobile body), 34 Vehicle ECU, 35 Processor

Claims

1. A moving body, a mobile terminal that wirelessly transmits to the mobile object a control signal based on an operation input by an operator who remotely supports the mobile object; a safety device that is physically connected to the mobile terminal and that exchanges diagnostic signals with the mobile object via the mobile terminal; Equipped with The moving body is When no abnormality is detected in the diagnostic signal received from the safety device via the mobile terminal, the vehicle travel control is executed based on the control signal from the mobile terminal. When the abnormality is detected, the driving control is not executed. A remote support system comprising:

2. When the moving body does not receive the diagnostic signal from the safety device via the mobile terminal, the moving body does not execute the driving control.

2. The remote support system according to claim 1.

3. The diagnostic signal is A first diagnostic signal transmitted from the moving object to the safety device via the mobile terminal; a second diagnostic signal transmitted from the safety device to the mobile object via the mobile terminal; Including, the safety device generates the second diagnostic signal by performing a predetermined calculation on the received first diagnostic signal; The mobile unit verifies the received second diagnostic signal and detects the abnormality based on a result of the verification.

2. The remote support system according to claim 1.

4. The mobile terminal generates the control signal based on an operation directly input to the mobile terminal by the operator.

2. The remote support system according to claim 1.

5. A mobile object that is remotely supported based on a control signal wirelessly transmitted from a mobile terminal operated by an operator, a security device is physically connected to the mobile terminal; The safety device exchanges diagnostic signals with the mobile object via the mobile terminal; The mobile entity comprises one or more processors; The one or more processors: When no abnormality is detected in the diagnostic signal received from the safety device via the mobile terminal, the vehicle travel control is executed based on the control signal from the mobile terminal. When the abnormality is detected, the driving control is not executed. A moving object characterized by:

Images