Teleoperated system and communication method for a teleoperated system

Abstract

To achieve both stabilization of moving image transmission and suppression of a delay time, and thereby to accurately perform remote maneuvering of a work machine.SOLUTION: A remote maneuver system comprises a first remote maneuver unit mounted on a work machine and a second maneuver unit mounted on a remote maneuver device. The first remote maneuver unit comprises: a plurality of first encoders which encode moving image data that a plurality of imaging devices mounted on the work machine acquire respectively; and a second encoder which integrates the encoded data that the plurality of first encoders encode, and then adds error correction codes to the integrated encoded data and outputs first integrated encoded data. The second remote maneuver unit comprises: a first decoder which decodes the first integrated encoded data based upon the error correction codes, and then separates the decoded data into the respective moving image data of the plurality of imaging devices; and a plurality of second decoders which decode the respective moving image data of the plurality of imaging devices separated by the first decoder.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present disclosure relates to a remote control system for remotely controlling a work machine (hydraulic excavator, dump truck, etc.) at a civil engineering work site, mine, etc., and a communication method for the remote control system. [Background technology]

[0002] In recent years, there has been progress in the development of remote control systems for remotely controlling work machines. Work machines that are the target of remote control are usually equipped with multiple cameras that can capture images of the situation in multiple directions (forward, left, right, rear, etc.) of the work machine. Remote control systems are required to process the video data obtained from these multiple cameras and transmit it to a remote control system located tens to hundreds of kilometers away from the work site, thereby accurately controlling the work machine. To achieve this, it is necessary to transmit the video images captured by the multiple cameras accurately and with low latency.

[0003] In order to accurately transmit video data, a transmission method using error-correcting codes is used. However, if the code size of the error-correcting code is fixed, there is a problem that the delay time increases in inverse proportion to the decrease in transmission speed. In conventional technology (for example, Patent Document 1), when the transmission speed is low, packet data is cut off at a predetermined threshold time and encoding is continued even if the number of packets is insufficient, thereby suppressing the increase in delay time. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Laid-Open No. 2009-60513 Summary of the Invention [Problem to be solved by the invention]

[0005] However, with the above-mentioned technology, it is difficult to achieve both stable video transmission and reduced delay time. Therefore, there is a need for a remote control system that can achieve both stable video transmission and reduced delay time. The present disclosure provides a remote control system and communication method that achieve both stable video transmission and reduced delay time, thereby enabling accurate remote control of a work machine. [Means for solving the problem]

[0006] To solve the above problems, a remote control system according to a first aspect of the present disclosure is a remote control system for remotely controlling a work machine using a remote control device, and includes a first remote control unit mounted on the work machine and a second remote control unit mounted on the remote control device. The first remote control unit includes a plurality of first encoders that encode video data acquired by a plurality of image capture devices mounted on the work machine, and a second encoder that combines the encoded data encoded by the plurality of first encoders, adds an error correction code to the combined encoded data, and outputs first combined encoded data. The second remote control unit includes a first decoder that decodes the first combined encoded data based on the error correction code, and then separates it into video data for each of the plurality of image capture devices, and a plurality of second decoders that decode the video data for each of the plurality of image capture devices separated by the first decoder.

[0007] A remote control system according to a second aspect of the present disclosure is a remote control system for remotely controlling a work machine equipped with multiple imaging devices using a remote control device, and includes: a first remote control unit mounted on the work machine; a first network providing wireless communication at a work site where the work machine is installed; a second network providing communication between devices installed at a remote base where the remote control device is located; a gateway device connected to the first remote control unit via the first network; an inter-base network providing communication between the first network and the second network; and a second remote control unit mounted on the remote control device. The first remote control unit and the second remote control unit are capable of sending and receiving data via the first network, the gateway device, the inter-base network, and the second network. The gateway device includes a retransmission control unit that requests the first remote control unit to retransmit data when an error is found in the data received from the work machine; and an encoder that adds an error correction code to the data received from the first remote control unit and outputs encoded data. The second remote control unit includes a decoder that decodes the encoded data based on the error correction code. [Effects of the Invention]

[0008] According to the present disclosure, it is possible to provide a remote control system and a communication method that achieve both stable video transmission and reduced delay time, thereby enabling accurate remote control of a work machine. [Brief explanation of the drawings]

[0009] [Figure 1] 1 is an overall configuration diagram illustrating a remote control system according to a first embodiment. [Figure 2] 2 is a block diagram illustrating an example of the configuration of a remote control unit 105 mounted on a work machine 100. FIG. [Figure 3] 2 is a block diagram illustrating an example of the configuration of a remote control unit 145 mounted on a remote control device 140. FIG. [Figure 4] FIG. 2 is a block diagram showing an example of the hardware configuration of remote control units 105 and 145. [Figure 5] FIG. 2 is a conceptual diagram illustrating a specific example of forward error correction performed by the FEC encoder 231. [Figure 6] FIG. 2 is a conceptual diagram illustrating a specific example of forward error correction performed by the FEC encoder 231. [Figure 7] FIG. 2 is a data structure diagram showing an example of the data structure of an RTP packet. [Figure 8] FIG. 2 is a data structure diagram illustrating an example of the data structure of an FEC packet. [Figure 9] FIG. 4 is a block diagram illustrating an example of the configuration of a remote control unit 105 mounted on a work machine 100 according to a second embodiment. [Figure 10] FIG. 10 is a block diagram illustrating an example of the configuration of a remote control unit 145 mounted on a remote control device 140 according to a second embodiment. [Figure 11] FIG. 10 is an overall configuration diagram illustrating a remote control system according to a third embodiment. [Figure 12] FIG. 10 is a block diagram illustrating an example of the configuration of a remote control unit 105 mounted on a work machine 100 according to a third embodiment. [Figure 13] FIG. 11 is a block diagram illustrating an example of a configuration of a gateway device 115 according to a third embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0010] Hereinafter, the present embodiment will be described with reference to the accompanying drawings. In the accompanying drawings, functionally identical elements may be designated by the same numerals. Note that the accompanying drawings show embodiments and implementation examples according to the principles of the present disclosure, but these are for understanding the present disclosure and are not to be used to interpret the present disclosure in a limiting manner. The descriptions in this specification are merely typical examples and are not intended to limit the scope or application of the present disclosure in any way.

[0011] Although the present embodiment has been described in sufficient detail to enable those skilled in the art to implement the present disclosure, it should be understood that other implementations and forms are possible, and that changes in configuration and structure and substitutions of various elements are possible without departing from the scope and spirit of the technical ideas of the present disclosure. Therefore, the following description should not be interpreted as being limited thereto.

[0012] [First embodiment] The overall configuration of the remote control system of the first embodiment will be described with reference to Figure 1. This remote control system is broadly composed of a work machine 100, a work site network 110, a wireless communication device 111, an inter-base network 120, a remote base network 130, and a remote control device 140. The work site network 110 is a network for providing communications, including wireless sections, to various devices at the work site where the work machine 100 is located. The remote base network 130 is a network for providing communications at the remote base where the remote control device 140 is located. The inter-base network 120 is a network for connecting the work site network 110 and the remote base network 130. The inter-base network 120 is a network that connects work sites and remote bases that are several kilometers to several hundred kilometers apart, and is a network with a large capacity, but which can experience large delays in data transmission due to the communication distance.

[0013] The work machine 100 is a machine used to perform excavation, loading, transporting, and other tasks at a work site (such as a mine), and is, for example, a hydraulic excavator, a crane truck, or a dump truck. As an example, the work machine 100 is equipped with a wireless communication device 101 and a remote control unit 105 (first remote control unit). Although only one work machine 100 is shown in Fig. 1, it goes without saying that multiple work machines 100 may exist within one system (within a work site).

[0014] On the other hand, the remote control device 140 is configured to receive moving image data from the work machine 100 via the various networks 110, 120, 130 and wireless communication devices 101, 111, and to be able to control the work machine 100 located at a work site away from a remote base based on that moving image data. The remote control device 140 is equipped with a remote control unit 145 (second remote control unit).

[0015] Next, an example of the configuration of the remote control unit 105 mounted on the work machine 100 will be described with reference to the block diagram in Figure 2. The remote control unit 105 mounted on the work machine 100 is configured to include an image processing unit 200, multiple (three in Figure 2) cameras 211 (A to C), a sensor 212, a drive system 213, and a control network 222.

[0016] The cameras 211 (A-C) are installed at multiple locations on the work machine 100 and are configured to be able to capture images of the surroundings of the work machine 100 (forward, left, right, rear, etc.) as moving images. The sensors 212 are sensors for detecting objects around the work machine 100 or for detecting the position and status of the work machine 100, and include, for example, optical sensors, ultrasonic sensors, radar, etc. The drive system 213 is a system for driving the various motors, hydraulic pumps, etc. mounted on the work machine 100. The sensors 212 and drive system 213 are capable of sending and receiving measurement data and control data to the image processing unit 200 via the control network 222.

[0017] The image processing unit 200 includes a plurality of video encoders 221 (first encoders), a communication control unit 230, and an IP network interface 241. The communication control unit 230 further includes an FEC encoder 231 (second encoder) and a CAN / IP conversion unit 232. The plurality of video encoders 221 (A to C) are provided corresponding to the plurality of cameras 211, and are configured to encode video data acquired by each of the plurality of cameras 211 and convert the encoded data into a format for transmitting the video data over a network, such as packets in the Realtime Transport Protocol (RTP). The video encoders 221, the communication control unit 230, and the IP network interface 241 may be implemented as hardware or software.

[0018] The FEC encoder 231 in the communication control unit 230 integrates multiple sets of coded data obtained from the multiple video encoders 221. Then, the FEC encoder 231 treats the integrated coded data as a single unit of error correction and performs encoding on the integrated coded data using a so-called forward error correction (FEC) code as a single (single-structure or common) error correction code. Specifically, the FEC encoder 231 integrates RTP packets containing multiple sets of coded data obtained from the multiple video encoders 221 into predetermined groups and generates an FEC packet for each integrated group. The FEC packets are output to the IP network interface 241 along with the corresponding RTP packets. The CAN / IP converter 232 converts CAN data format signals of the sensor 212 and the drive system 213 received from the control network 222 into IP communication data. The converted IP communication data is transferred to the IP network interface 241 and output / transmitted in parallel with (independently of) the FEC encoded data output by the FEC encoder 231 (the IP communication data is not subject to forward error correction by the FEC encoder 231).

[0019] 3, an example of the configuration of the remote control unit 145 mounted on the remote control device 140 will be described. The remote control unit 145 mounted on the remote control device 140 can be roughly composed of an image processing unit 250, a control network 272, a display device 261, a display device 262, and a control device 263.

[0020] 3, the image processing unit 250 can be made up of, for example, an IP network interface 291, a communication control unit 280, and a video decoder 271 (271A to C). The IP network interface 291 receives data from the remote control unit 105 mounted on the work machine 100 via the wireless communication device 111, the work site network 110, the inter-base network 120, and the remote base network 130, and transfers the received data to the communication control unit 280.

[0021] As an example, the communication control unit 280 includes an FEC decoder 281 (first decoder) that applies forward error correction (FEC) to received data and decodes it, and a CAN / IP conversion unit 282 that converts IP communication data into CAN data. The FEC decoder 281 decodes received data based on data (RTP packets, FEC packets) received from the remote control unit 105 mounted on the work machine 100, and, as will be described later, separates the decoded data into the original data for each of the multiple cameras 211A-C based on the header data of the RTP packets and FEC packets and outputs it. The video decoders 271A-C (second decoders) convert the decoded data separated by the FEC decoder 281 into a video image data format that can be input to the display device 261. The display device 261 is, for example, a liquid crystal display or the like, and displays the restored video image data.

[0022] The CAN / IP conversion unit 282 converts the IP communication data received from the CAN / IP conversion unit 232 of the remote control unit 105 mounted on the work machine 100 back into CAN data. The converted CAN data (detection data from the sensor 212 and control data for the drive system 213) is transmitted to the display device 262 and control device 263 via the control network 272.

[0023] FIG. 4 shows an example of the hardware configuration of the remote control unit 105, 145. For example, the remote control unit 105 may be composed of a CPU / DSP 301, a memory unit 302, a logic circuit 303, a LAN interface 304, and a communication bus 305. The CPU / DSP 301 is an arithmetic processing unit and signal processing unit for executing various calculations and data processing in the remote control unit 105, 145. The memory unit 302 is, for example, a ROM, RAM, flash memory, or hard disk drive, and is a storage device for storing data and computer programs required for various calculations and data processing in the remote control unit 105, 145. The logic circuit 303 provides various calculations and data processing along with computer programs. The LAN interface 304 provides data communication with the outside. The communication bus 305 connects the CPU / DSP 301, the memory unit 302, the logic circuit 303, and the LAN interface 304, and provides data communication between them.

[0024] Next, a specific example of the forward error correction configuration used in the FEC encoder 231 and the FEC decoder 281 will be described with reference to FIGS. 5 and 6. FIG. 5 shows a method for generating FEC packets specified by Pro-MPEG. In this method, multiple RTP packets (410) are arranged in a matrix (e.g., M rows and N columns), then grouped by row / column, and an FEC packet (420) is generated by performing an exclusive OR operation on the grouped RTP packets. The generated FEC packet is added to the RTP packet matrix and transmitted as coded data. In the method shown in FIG. 5, RTP packets are arranged two-dimensionally and grouped in both the row and column directions to generate FEC packets, thereby improving the accuracy of error correction. Note that the forward error correction according to Pro-MPEG shown in FIG. 5 is just one example, and it goes without saying that other forward error correction methods are also possible, such as generating packets one-dimensionally and performing forward error correction on groups of multiple packets.

[0025] When forward error correction is performed, decoding of erroneous or missing RTP packets cannot be performed until all RTP packets and FEC packets included in a group have been received by the receiving device. Therefore, when the video transmission rate is low, delays in video playback may increase. Even when FEC packets are generated one-dimensionally, buffer storage delays occur in proportion to the number of PTP packets included in the group, resulting in a corresponding increase in video delay. When FEC packets are generated two-dimensionally (Pro-MPEG method) as shown in Figure 5, there is a possibility that the storage delay will increase further by a number corresponding to the number of rows in the matrix. This storage delay becomes more severe as the transmission rate of video data decreases.

[0026] Therefore, in the remote control system according to the first embodiment, the RTP packets created after encoding the image data obtained from the multiple cameras 211A-C by the video encoders 221A-C are further integrated by the FEC encoder 231 and forward error correction is applied (FEC packets are generated and added to the RTP packets). As shown in FIG. 6, the RTP header of the RTP packet contains camera identification information indicating which of the multiple cameras 211A-C the data was obtained from, and a sequence number indicating the processing procedure for the video (A-1, B-1, ..., C-4). By including such information in the RTP packet header, data separation becomes possible in the FEC decoder of the remote control device 140.

[0027] 7 shows an example of the data structure of an RTP packet. The RTP packet may include, for example, an RTP header 510 in which header information is stored, and an RTP payload portion 520 in which actual data (including image capture data from cameras 211A-C) is stored. The RTP header 510 may store, for example, timestamp information 511, sequence number information 512, and synchronization source information 513. The timestamp information 511 is information related to a timestamp indicating the time when the RTP packet was generated. The sequence number information is information related to a number indicating the data processing order in camera 211 from which the RTP packet originated. The synchronization source information 513 is information indicating which camera the actual data related to the RTP packet was obtained from.

[0028] 8 shows an example of the data structure of an FEC packet. The FEC packet includes an FEC header 610 that stores, for example, header information, and an FEC payload 620 that stores actual data (the exclusive OR of RTP packets in one column or one row of a matrix). The FEC header 610 can include, for example, direction information 611, camera number information 612, camera identification information 613, 615, 617, and SN map information 614, 616, 618.

[0029] Direction information 611 indicates in which direction (vertical or horizontal) the RTP packets in the matrix of RTP packets the FEC packet relates to. Camera number information 612 is information regarding the number of cameras 211 that generated the video data to be integrated in the FEC encoder 231. Camera identification information 613, 615, and 617 is information for identifying the camera 211. SN map information 614, 616, and 618 is information regarding a single column or row of RTP packets that were used to generate the FEC packet. If the RTP packets that were used to generate the FEC packet are a single row of RTP packets in the row direction, the SN map information 614, 616, and 618 include the first sequence number and the number of packets in the row. If the RTP packets from which the FEC packet was generated are a column-wise sequence of RTP packets, the SN map information 614, 616, 618 includes a list of the first sequence numbers of the column of RTP packets and sequence number increments.

[0030] As described above, according to the remote control system of the first embodiment, video data acquired by the multiple cameras 211A-C of the work machine 100 and encoded by the video encoders 221A-C is integrated in the FEC encoder 231, subjected to forward error correction, and transmitted. The receiving remote control device 140 decodes this data, then separates and plays it back according to the header information of the RTP packets. Because the information from the multiple cameras is transmitted in an integrated state with forward error correction applied, even if the video data rate of some cameras is low, buffer delays in the FEC decoder 281 for forward error correction are reduced without changing the code size, and even if the transmission speed drops on some communication paths, buffer delays are reduced, and video playback delays are also reduced.

[0031] [Second embodiment] A remote control system according to a second embodiment will be described with reference to Figures 9 and 10. The overall configuration of the system is similar to that of the first embodiment (Figure 1), so a duplicated description will be omitted. This second embodiment differs from the first embodiment in the configurations of the remote control unit 105 mounted on the work machine 100 and the remote control unit 145 mounted on the remote control device 140. In Figures 9 and 10, the same components as those in the first embodiment (Figures 2 and 3) are given the same reference numerals in Figure 9 as in Figure 2, so duplicated descriptions will be omitted below and the differences will be mainly described.

[0032] As shown in Fig. 9, the FEC encoder 231 of the second embodiment not only subjects image data obtained from a plurality of cameras 211A-C to forward error correction encoding collectively, but also subjects IP communication data obtained from the CAN / IP converter 232 to forward error correction. Then, as shown in Fig. 10, the FEC decoder 281 outputs decoded data not only to the video decoders 271A-C, but also to the CAN / IP converter 282. In other words, in the first embodiment, the output data of the CAN / IP converter 232 is output in parallel with the coded data output by the FEC encoder 231 and is not subjected to forward error correction (FEC), whereas in the second embodiment, the output data of the CAN / IP converter 232 is subjected to forward error correction in the FEC encoder 231 together with the video image data of the camera 211 and is output collectively.

[0033] Specifically, the FEC encoder 231 of the second embodiment converts IP communication data based on output signals from the sensor 212 and the drive system 213 into RTP packets in the same way as the signals from the cameras 211A to C, and subjects the data to forward error correction together with the RTP packets derived from the signals from the cameras 211A to C.

[0034] In the second embodiment, all of the signals from the sensor 212 and the drive system 213 may be input to the FEC encoder 231 and subjected to forward error correction, or only some of the signals from the sensor 212 and the drive system 213 may be input to the FEC encoder 231, with the remainder not being input to the FEC encoder 231 but instead being output from the remote control unit 105 mounted on the work machine 100 in parallel with the output signals from the FEC encoder 231. In this case, the proportion of the signals from the sensor 212 and the drive system 213 that are included may be changed according to the amount of video data from the cameras 211A to C. If the amount of data in the buffer is insufficient after the threshold time has elapsed, dummy padding data may be added in addition to the signals from the sensor 212 or the drive system 213, and forward error correction may be performed in the FEC encoder 231.

[0035] As described above, the remote control system of the second embodiment can achieve the same effects as the remote control system of the first embodiment. In addition, according to the system of the second embodiment, the output signals of the sensor 212 and the drive system 213 are also subjected to forward error correction along with the output signals of the cameras 211A to C, so that the delay in buffer accumulation in the FEC decoder 281 can be further reduced compared to the first embodiment.

[0036] [Third embodiment] A remote control system according to the third embodiment will be described with reference to Figures 11 to 13. Figure 11 shows the overall configuration of a remote control system according to the third embodiment. The difference from the previous embodiments is that a gateway device 115 connected to a work site network 110 is provided. This gateway device 115 is a device that relays the transmission and reception of data between the work machine 100 and the remote control device 140. In other words, the gateway device 115 carries out communication, including wireless communication, with the work machine 100 via the work site network 110 and the wireless communication device 111, and transmits data received from the work machine 100 to the remote control device 140 via the work site network 110, the base network 120, and the remote base network 130.

[0037] If there are any missing or error data in the encoded reception data received from the work machine 100 via the video encoders 221A-C, the gateway device 115 executes retransmission control to request the work machine 100 to retransmit the received data. The work machine 100 is configured to receive this retransmission request and execute an operation to retransmit data that has already been transmitted.

[0038] FIG. 12 shows an example of the configuration of a remote control unit 105 in a work machine 100 according to the third embodiment. The remote control unit 105 mounted on this work machine 100 has a retransmission control unit 233 instead of the FEC encoder 231. The retransmission control unit 233 converts coded video data from multiple video encoders 221A-C into packets in a video transmission format such as RTP, and transmits the packets appropriately via an IP network interface 241 to the gateway device 115, and when a retransmission request is received from the gateway device 115, executes retransmission of coded video data that has already been transmitted. The other configuration is the same as in the first embodiment. The configuration of the remote control unit 145 mounted on the remote control device 140 may be the same as in the first embodiment (FIG. 3).

[0039] 13 shows an example of the configuration of the gateway device 115 in the third embodiment. This gateway device 115 comprises an IP network interface 251 which is a communication interface with the remote control unit 105 in the work machine 100, a communication control unit 290, and an IP network interface 254 which is a communication interface with the remote control unit 145 mounted on the remote control device 140. The communication control unit 290 further comprises a retransmission control unit 252 and an FEC encoder 253.

[0040] The retransmission control unit 252 determines whether there are any missing data or errors in the encoded received data received from the remote control unit 105 of the work machine 100, and if there are any missing data or errors, it sends a retransmission request to the retransmission control unit 233 of the remote control unit 105 of the work machine 100 via the IP network interface 251. If there are no missing data or errors, the retransmission control unit 252 transfers the received data to the FEC encoder 253. The function of the FEC encoder 253 is the same as that of the FEC encoder 231 of the first embodiment.

[0041] As described above, the remote control system according to the third embodiment executes retransmission control between the work machine 100 and the gateway device 115 if there is a loss of or an error in the received data, and between the gateway device 115 and the remote control device 140, video data is transmitted with forward error correction (FEC) applied, as in the previously described embodiments.

[0042] Between the work machine 100 and the gateway device 115, which are wirelessly connected via the work site network 110 and the wireless communication device 111, low-volume but low-latency data communication is carried out because the communication distance is short.

[0043] On the other hand, because communication between the gateway device 115 and the remote control device 140 involves the inter-site network 120 (wired connection), even though the data is large, there is a large amount of delay due to the communication distance. In the third embodiment, retransmission control is used for data communication between the gateway device 115 and the work machine 100, which are connected by a short-distance wireless connection with little data delay, while forward error correction (FEC)-based data communication is used between the gateway device 115 and the remote control device 140 (wired connection), which has a large data delay. The FEC encoder 253 of the gateway device 115 performs FEC encoding on the video data from the multiple cameras 211A-C, and the FEC decoder 281 of the remote control device 140 performs FEC decoding, which is the same as the operation of the first embodiment. In other words, in the third embodiment, in addition to the operation of the first embodiment, a retransmission control operation in the gateway device 115 is added.

[0044] As described above, according to the third embodiment, it is possible to obtain the same effects as those of the first embodiment, and further, since retransmission control is performed by the gateway device 115 in a low-latency wireless connection section, it becomes possible to transmit video data with even lower latency.

[0045] The present invention is not limited to the above-described embodiments, but includes various modifications and equivalent configurations within the spirit of the appended claims. For example, the above-described embodiments have been described in detail to clearly explain the present invention, and the present invention is not necessarily limited to having all of the described configurations. Furthermore, part of the configuration of one embodiment may be replaced with the configuration of another embodiment. Furthermore, the configuration of another embodiment may be added to the configuration of one embodiment. Furthermore, part of the configuration of each embodiment may be added, deleted, or replaced with other configurations.

[0046] Furthermore, some or all of them may be realized by hardware such as an integrated circuit. Furthermore, each of the aforementioned configurations, functions, processing units, processing means, etc. may be realized by software, with a processor interpreting and executing a computer program that realizes each function. Furthermore, when there are multiple identical functions in the configuration, the hardware or software that realizes each function may be implemented separately, or multiple processes may be performed using a single implemented hardware or software in a time-multiplexed manner. Furthermore, even if there is a single function in the configuration, distributed processing may be performed using multiple hardware or software having the same function.

[0047] Information such as programs, tables, and files that realize each function can be stored in a storage device such as a memory, a hard disk, or an SSD (Solid State Drive), or in a recording medium such as an IC card, an SD card, or a DVD.

[0048] In addition, the control lines and information lines shown are those that are considered necessary for the explanation, and do not necessarily show all the control lines and information lines that are necessary for implementation. In reality, it can be considered that almost all components are interconnected. [Explanation of symbols]

[0049] 100...work machine, 101...wireless communication device, 105...remote control unit, 110...work site network, 111...wireless communication device, 115...gateway device, 120...inter-base network, 130...remote base network, 140...remote control device, 145...remote control unit, 200...image processing unit, 211A-C...camera, 212...sensor, 213...drive system, 221A-C...video encoder, 222...control network, 230...communication control unit, 231...FEC encoder, 232...CAN / IP conversion unit, 233...retransmission control unit, 241...IP network interface, 250...image processing unit, 251...IP network interface, 252...retransmission control unit, 253...FEC encoder, 254...IP network interface, 261A-C, 262...display device, 263...Control device, 271A-C...Video decoder, 272...Control network, 280...Communication control unit, 281...FEC decoder, 282...CAN / IP conversion unit, 291...IP network interface, 301...CPU / DSP, 302...Memory unit, 303...Logic circuit, 304...LAN interface, 305...Communication bus.

Claims

1. In a remote control system in which a work machine is remotely controlled by a remote control device, a first remote control unit mounted on the work machine; a wireless network that provides wireless communication at a work site where the work machine is installed; a gateway device connected to the first remote control unit via the wireless network; a second remote control unit mounted on the remote control device; a wired network that constitutes at least a part of a communication path between the gateway device and the second remote control unit; Equipped with the first remote control unit and the second remote control unit transmit and receive data via the wireless network, the gateway device, and the wired network; the first remote control unit includes a plurality of first encoders that encode moving image data acquired by a plurality of imaging devices mounted on the work machine, The gateway device a retransmission control unit that requests the first remote control unit to retransmit the data when an error is present in the data received from the work machine; a second encoder that, when there is no error in the data received from the work machine, integrates the coded data encoded by the plurality of first encoders, and then adds an error correction code to the integrated coded data to output first integrated coded data, The second remote control unit a first decoder that decodes the first integrated coded data based on the error correction code and then separates the first integrated coded data into moving image data of each of the plurality of image capture devices; a plurality of second decoders that decode the moving image data of each of the plurality of imaging devices separated by the first decoder; A remote control system comprising:

2. the first remote control unit further includes a CAN / IP conversion unit that converts information acquired by a sensor or a drive system mounted on the work machine from a Controller Area Network (CAN) to an IP communication method, The remote control system according to claim 1 , configured to output the output data of the CAN / IP conversion unit in parallel with the first integrated encoded data.

3. the first remote control unit further includes a CAN / IP conversion unit that converts information acquired by a sensor or a drive system mounted on the work machine from a Controller Area Network (CAN) to an IP communication method, The remote control system according to claim 1 , wherein the second encoder integrates the output data of the CAN / IP conversion unit with the encoded data encoded by the first encoder and outputs the integrated encoded data as the first integrated encoded data.

4. 2. The remote control system of claim 1, wherein the second encoder generates an RTP (Realtime Transport Protocol) packet according to the encoded data encoded by the plurality of first encoders, and a header of the RTP packet includes at least a sequence number indicating a processing procedure in the plurality of image capture devices and image capture device identification information that identifies one of the plurality of image capture devices.

5. The remote control system according to claim 4 , wherein the second encoder arranges the RTP packets in a matrix and generates a forward error correction (FEC) packet for each group in the row and column directions of the matrix.

6. A communication method in a remote control system in which a work machine is remotely controlled by a remote control device, The remote control system includes: a wireless network that provides wireless communication at a work site where the work machine is installed; a gateway device connected to the work machine via the wireless network; a wired network that constitutes at least a part of a communication path between the gateway device and the remote control device; Equipped with the work machine and the remote control device transmit and receive data via the wireless network, the gateway device, and the wired network; In the work machine, the moving image data acquired by the plurality of imaging devices mounted on the work machine are encoded to generate encoded data, In the gateway device, if an error exists in the data received from the work machine, a request is made to the work machine to resend the data; in the gateway device, if there is no error in the data received from the work machine, integrate the coded data related to the plurality of imaging devices, add an error correction code to the integrated coded data, and output first integrated coded data; In the remote control device, the first integrated encoded data is decoded based on the error correction code, and then separated into moving image data of each of the plurality of imaging devices, and further, the separated moving image data of each of the plurality of imaging devices is decoded. A communication method comprising:

7. 7. The communication method according to claim 6, wherein, in the work machine, information acquired by a sensor or a drive system mounted on the work machine is converted from a Controller Area Network (CAN) to an IP communication method, and data conforming to the IP communication method is output in parallel with the first integrated encoded data.

8. 7. The communication method according to claim 6, wherein, in the work machine, information acquired by a sensor or a drive system mounted on the work machine is converted from a Controller Area Network (CAN) to an IP communication format, the data converted to the IP communication format is integrated with coded data obtained by encoding each of the moving image data acquired by the plurality of imaging devices, the error correction code is added to the integrated coded data, and the integrated coded data is output as the first integrated coded data.

9. 7. The communication method according to claim 6, wherein in the work machine, an RTP packet is generated in accordance with encoded data obtained by encoding moving image data acquired by the plurality of imaging devices, and a header of the RTP packet includes at least a sequence number indicating a processing procedure in the plurality of imaging devices and imaging device identification information that identifies one of the plurality of imaging devices.

10. The communication method according to claim 9 , wherein the RTP packets are arranged in a matrix, and an FEC packet is generated for each group in the row and column directions of the matrix.

Images