Remote operation system for multiple vehicles

Abstract

A remote operation system is provided with: an imaging device for imaging a plurality of vehicles; a remote operation monitor that outputs the image captured by the imaging device and is configured to be capable of selecting a vehicle to be operated among the plurality of vehicles by receiving a touch operation; a communication device that is communicatively connected to a selected vehicle, which is a vehicle selected by a touch operation among the plurality of vehicles; the remote operation input device is used for instructing the selected vehicle which is in communication connection to act; a first control unit that is mounted on each of the plurality of vehicles, and that exhibits a connection state with the communication device by using a light emission mode of a light emitting device provided in the vehicle; and a second control unit, the second control unit having: an acquisition unit that acquires a light emission pattern in the image; a conversion unit that converts the acquired light emission mode to connection state information; and an output unit that outputs the converted connection state information to a remote operation monitor in association with the vehicle.

Description

Technical Field

[0001] This disclosure relates to remote operating systems for multiple vehicles. Background Technology

[0002] Patent Document 1 discloses a system for remotely operating a vehicle. In this system, the operation modes of the touch panel are configured to not interfere with each other, thereby suppressing accidental operation. Existing technical documents Patent documents

[0003] Patent Document 1: Japanese Patent Application Publication No. 2019-077527 Summary of the Invention The problem that the invention aims to solve

[0004] However, the technology disclosed in Patent Document 1 does not consider the possibility of incorrect selection of the target vehicle when there are multiple target vehicles. There is room for improvement in this regard. Technical means for solving problems

[0005] This disclosure can be implemented in the following ways.

[0006] (1) According to one aspect of this disclosure, a remote operating system for multiple vehicles is provided. The system comprises: a camera that captures images of the multiple vehicles; a remote operation monitor that outputs the images captured by the camera and is configured to allow selection of a vehicle as the target of operation from the multiple vehicles by receiving a touch operation; a communication device that communicates with the vehicle selected by the touch operation, i.e., the selected vehicle; a remote operation input device that instructs an action to the selected vehicle with which it is communicatively connected; a first control unit mounted on each of the multiple vehicles, which uses the light emission pattern of a light emission device of the vehicle to indicate the connection status with the communication device; and a second control unit comprising: an acquisition unit that acquires the light emission pattern in the images; a conversion unit that converts the acquired light emission pattern into connection status information; and an output unit that outputs the converted connection status information in association with the vehicle to the remote operation monitor. According to this system, a first control unit mounted on each of the multiple vehicles uses a light-emitting pattern to indicate the connection status between the vehicle and the communication device. An acquisition unit acquires the light-emitting pattern, a conversion unit converts the light-emitting pattern into connection status information, and an output unit outputs the connection status information in association with the vehicle to a remote operation monitor. Therefore, operators can visually confirm the vehicle connected to the communication device by viewing the connection status information output to the remote operation monitor. This prevents situations where a vehicle not intended for operation is mistakenly selected and remotely operated by the remote operation input device. (2) Based on the system described above, the light emission mode may also be set inherently for each vehicle. According to this system, since the illumination pattern is inherently set for each vehicle, there is no need to prepare additional markers for vehicle identification; vehicles can be identified simply by using the illumination pattern. (3) Based on the above-mentioned system, the light emission mode may also include the color of the light and the flashing period. According to this system, since the light emission pattern includes both the color of the light and the flicker period, it can produce a wider variety of light emission patterns compared to a system where the light emission pattern is only composed of either the color of the light or the flicker period.

[0007] This disclosure can be implemented in various ways other than remote operating systems. For example, it can be implemented as a method for outputting connection status information, a program for executing that method, etc. Attached Figure Description

[0008] Figure 1 This is a simplified diagram illustrating a remote operating system for a vehicle in one aspect of this disclosure. Figure 2 This is a flowchart illustrating the process of outputting connection status information. Detailed Implementation

[0009] A. Implementation method: <Overall Composition> Figure 1 This is a simplified diagram illustrating a remote operating system 1 (hereinafter also referred to as System 1) for vehicles V1 and V2 in one embodiment of this disclosure. System 1 is used to operate vehicles V1 and V2 from a location away from their workspace. System 1 includes first control units C1 and C2, a camera 100, a remote operation monitor 200, a second communication device 300, a remote operation input device 400, and a second control unit 500.

[0010] In this embodiment, the first control units C1 and C2 are mounted on vehicles V1 and V2, which are the objects of operation. Vehicles V1 and V2 are, for example, forklifts used as industrial vehicles. Unless otherwise specified, vehicles V1 and V2 have the same configuration. In addition to the first control units C1 and C2, each vehicle V1 and V2 also has first communication devices A1 and A2, and light-emitting devices L1 and L2.

[0011] The first communication devices A1 and A2 communicate with the second communication device 300 (described later). Vehicles V1 and V2 receive instructions from the remote operation input device 400 (described later) via the first communication devices A1 and A2. The communication in this embodiment is wireless communication.

[0012] The light-emitting devices L1 and L2 are controlled by the first control units C1 and C2 to emit light. In this embodiment, the light-emitting devices L1 and L2 have various light-emitting modes. The "light-emitting mode" in this embodiment includes both the color of the light and the flashing period. The light-emitting devices L1 and L2 are, for example, composed of LEDs.

[0013] The first control units C1 and C2 are configured as computers equipped with a CPU and memory. The first control units C1 and C2 control the illumination modes of the light-emitting devices L1 and L2, using these modes to indicate the connection status between the vehicles V1 and V2 and the second communication device 300 (described later). In this embodiment, the "connection status" includes "connected," indicating that the vehicles V1 and V2 are connected to the second communication device 300, and "not connected," indicating that the vehicles V1 and V2 are not connected to the second communication device 300. The "connected" illumination mode is represented, for example, by emitting blue light from the light-emitting devices L1 and L2. The "not connected" illumination mode is represented, for example, by emitting red light from the light-emitting devices L1 and L2. The correspondence between the connection status and the illumination modes is pre-stored in the memory of the first control units C1 and C2.

[0014] Furthermore, in this embodiment, the light emission mode is inherently set for each vehicle V1 and V2. For example, a light emission mode with a different flashing cycle is set for each vehicle V1 and V2. In this disclosure, "flashing cycle" refers to the number of times the light emission devices L1 and L2 flash per unit time. In this embodiment, the first control unit C1 causes the light emission device L1 of vehicle V1 to flash at a cycle of 15 flashes per minute. The first control unit C2 causes the light emission device L2 of vehicle V2 to flash at a cycle of 30 flashes per minute. The correspondence between vehicles V1 and V2 and the light emission mode is pre-stored in the memory of the first control units C1 and C2.

[0015] The camera 100 captures images of vehicles V1 and V2 and transmits the captured data to a remote monitoring monitor 200 and a second control unit 500. The camera 100 is installed in the working space of vehicles V1 and V2. The camera 100 is, for example, a camera equipped with a CCD sensor or a CMOS sensor.

[0016] The remote operation monitor 200 is a device that serves as both an input device and a display device. The remote operation monitor 200 is, for example, a touch panel. The remote operation monitor 200 outputs the captured data transmitted via the imaging device 100. Furthermore, the remote operation monitor 200 is configured to accept touch operation and is configured to allow selection of vehicles V1 and V2 as the objects of operation through this touch operation. That is, the operator selects the vehicle V1 or V2 that they wish to operate from the vehicles V1 and V2 displayed on the remote operation monitor 200 by touch operation, thereby determining that vehicle V1 or V2 as the selected vehicle. In this embodiment, vehicle V2 is selected.

[0017] The second communication device 300 communicates with the selected vehicle V2 via the first communication device A2.

[0018] The remote operation input device 400 instructs the selected vehicle V2 to perform actions. These actions include, for example, moving the selected vehicle V2 forward and backward, raising or lowering the forks, etc. The instructions are transmitted to the selected vehicle V2 via the second communication device 300.

[0019] The second control unit 500 is configured as a computer equipped with a CPU 510 and a memory 520. The CPU 510 executes the program stored in the memory 520, thereby enabling the acquisition unit 511, the conversion unit 512, and the output unit 513 to perform their functions.

[0020] The acquisition unit 511 acquires the illumination patterns contained in the image transmitted from the imaging device 100. Hereinafter, the description will be based on an example where vehicle V1 is disconnected and vehicle V2 is connected. The illumination device L1 of vehicle V1 illuminates in a red illumination pattern, indicating a flashing cycle of 15 times / minute for vehicle V1 and indicating a disconnected state. The illumination device L2 of vehicle V2 illuminates in a blue illumination pattern, indicating a flashing cycle of 30 times / minute for vehicle V2 and indicating a connected state.

[0021] The conversion unit 512 converts the light emission pattern obtained by the acquisition unit 511 into connection status information. "Connection status information" indicates which vehicle V1 or V2 is in which connection state. In this embodiment, the connection status information includes vehicle V1 being disconnected and vehicle V2 being connected. The correspondence between the light emission pattern and the connection status information is pre-stored in the memory 520.

[0022] The output unit 513 outputs the connection status information converted by the conversion unit 512, associated with vehicles V1 and V2, to the remote operation monitor 200. For example... Figure 1 As shown, the output unit 513 displays "Not connected" as connection status information S1 on the display screen of vehicle V1 on the remote operation monitor 200, and displays "Connecting" as connection status information S2 on vehicle V2.

[0023] <Processing of Connection Status Information Output> Figure 2 This is a flowchart illustrating the output processing (hereinafter also referred to as output processing) of connection status information. Output processing is performed as a procedure for enabling vehicles V1 and V2 to perform operations via remote operation. Output processing is initiated and executed by system 1.

[0024] The acquisition unit 511 acquires the illumination pattern contained in the image transmitted from the imaging device 100 (step S100). Hereinafter, "step S" will be abbreviated as "S". The conversion unit 512 converts the illumination pattern acquired by the acquisition unit 511 into connection status information (S110). The output unit 513 outputs the connection status information converted by the conversion unit 512 to the remote operation monitor 200 in association with the vehicles V1 and V2 (S120). The processing from S100 to S120 is repeated until system 1 is stopped.

[0025] According to the first embodiment of system 1 described above, since the first control units C1 and C2 use the light emission patterns of the light emission devices L1 and L2 to represent the connection status between vehicles V1 and V2 and the second communication device 300, the second control unit 500 converts the light emission patterns in the image into connection status information, and displays the connection status information S1 and S2 in association with vehicles V1 and V2 on the remote operation monitor 200. Therefore, by observing the display on the remote operation monitor 200, the operator can identify which vehicle V1 or V2 is the selected vehicle. As a result, the possibility of vehicles V1 and V2 being remotely operated when they are mistakenly selected can be reduced.

[0026] Furthermore, according to System 1 of the first embodiment, since the first control units C1 and C2 use the light emission patterns of the light-emitting devices L1 and L2 to indicate the connection status between the vehicles V1 and V2 and the second communication device 300, the connection status can be transmitted simply and inexpensively compared to a configuration that uses a communication device to transmit the connection status. In addition, the connection status can be transmitted even when communication is not established.

[0027] Furthermore, according to the system 1 of the first embodiment, since the light emission pattern is inherently set for each vehicle V1 and V2, there is no need to prepare additional markers for identifying vehicles V1 and V2; vehicles V1 and V2 can be identified simply by the light emission pattern.

[0028] Furthermore, according to System 1 of the first embodiment, since the light emission pattern includes both the color of the light and the flashing period, the variety of light emission patterns is greater than that of a light emission pattern consisting only of either the color of the light or the flashing time. This increases the number of candidate vehicles that can be operated on by System 1.

[0029] B. Other implementation methods: (B1) In the above embodiments, the flashing period of the light-emitting devices L1 and L2 is inherently set for each vehicle V1 and V2, but this disclosure is not limited thereto. The light emission color of the light-emitting devices L1 and L2 can also be inherently set for each vehicle V1 and V2.

[0030] (B2) In the above embodiments, the light emission mode includes the color of the light and the flashing period, but this disclosure is not limited thereto. The light emission mode may also be only either the color of the light or the flashing period.

[0031] (B3) In the above embodiments, the light emission pattern is inherent to each vehicle V1 and V2, but this disclosure is not limited thereto. Alternatively, the light emission pattern may be common to each vehicle V1 and V2. In this case, a marker is set to distinguish between vehicles V1 and V2, and the light emission pattern is used to transmit connection status information related to the connection status of vehicles V1 and V2.

[0032] (B4) In the above embodiments, the flicker period can be any value other than 15 times / minute or 30 times / minute. Furthermore, the illumination mode can also include a flicker period shorter than or the same as the frame rate of the shooting device 100. Specifically, when the frame rate of the shooting device 100 is 60fps, the flicker period can be 60 times / minute or 80 times / minute. In this case, the light-emitting devices L1 and L2 are identified as always illuminated in the shooting device 100.

[0033] (B5) In the above embodiments, the light-emitting devices L1 and L2 may also emit any type of light. For example, the light-emitting devices L1 and L2 may also emit invisible light. In this case, as the imaging device 100, a device capable of capturing invisible light is used. Specifically, the light-emitting devices L1 and L2 may emit infrared light, and the imaging device 100 may include an infrared sensor.

[0034] (B6) In the above embodiments, the connection state includes connected and not connected, but this disclosure is not limited thereto. The connection state may also include any state of vehicles V1 and V2. For example, the connection state may further include "operating" indicating that vehicles V1 and V2 are being operated by the remote operation input device 400, and "malfunction occurring" indicating that a malfunction has occurred in vehicles V1 and V2.

[0035] (B7) In the above embodiments, the remote operation monitor 200 and the remote operation input device 400 are configured independently of each other, but this disclosure is not limited thereto. The remote operation monitor 200 and the remote operation input device 400 may also be configured as an integral unit. For example, the remote operation of vehicles V1 and V2 may also be realized using the interface displayed on the remote operation monitor 200.

[0036] (B8) In the above embodiment, the first control units C1 and C2 are mounted on vehicles V1 and V2, but this disclosure is not limited thereto. The first control units C1 and C2 can also be implemented as computers located outside the vehicles V1 and V2. In this case, the first control units C1 and C2 control the light-emitting modes of the light-emitting devices L1 and L2 by transmitting information to the first communication devices A1 and A2. In addition, the first control units C1 and C2 and the second control unit 500 can also be implemented by a single computer.

[0037] (B9) In the above embodiment, the number of vehicles is 2, but this disclosure is not limited to this. The number of vehicles can be any number more than 2. In addition, the vehicle that becomes the object of remote operation can be any vehicle other than an industrial vehicle.

[0038] This disclosure is not limited to the embodiments described above, and can be implemented in various configurations without departing from its spirit. For example, technical features in the embodiments that correspond to the technical features in the various embodiments described in the "Summary of the Invention" section can be appropriately replaced or combined to solve some or all of the above-described problems, or to achieve some or all of the above-described effects. In addition, technical features can be appropriately deleted as long as they are not described as essential parts in this specification. Explanation of reference numerals in the attached figures

[0039] 1…Remote operating system; 100…Camera; 200…Remote operating monitor; 300…Secondary communication device; 400…Remote operation input device; 500…Secondary control unit; 510…CPU; 511…Acquisition unit; 512…Conversion unit; 513…Output unit; 520…Memory; A1, A2…First communication device; C1, C2…First control unit; L1, L2…Light-emitting device; S1, S2…Connection status information; V1, V2…Vehicle.

Claims

1. A system that is a remote operation system of a plurality of vehicles, comprising: a photographing device that photographs the plurality of vehicles; a remote operation monitor that outputs an image photographed by the photographing device and is configured to be able to select a vehicle as an operation target among the plurality of vehicles by accepting a touch operation; a communication device that communicatively connects with a selected vehicle that is selected among the plurality of vehicles by the touch operation; a remote operation input device that instructs an action to the selected vehicle that has been communicatively connected; a first control section that is mounted on each of the plurality of vehicles and indicates a connection state with the communication device using a light emission pattern of a light emission device possessed by the vehicle; and a second control section, the second control section having: an acquisition section that acquires the light emission pattern in the image; a conversion section that converts the acquired light emission pattern into connection state information; and an output section that outputs the converted connection state information to the remote operation monitor in association with the vehicle.

2. The system according to claim 1, wherein the light emission pattern is set uniquely for each of the vehicles.

3. The system according to claim 1 or 2, wherein the light emission pattern includes a color of light and a flickering period. ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​

Images