Work machine and remotely controlled work machine system

Abstract

To provide a work machine which can suppress deterioration in productivity of a work machine while suppressing deterioration in quality of communication between the work machine and a remote control device, and to provide a remove control system of the work machine.SOLUTION: A shovel 1 includes: a work device 5; an imaging device 13 for imaging the work device 5 and an image around it; and a control device 16 for transmitting the image imaged by the imaging device 13 as image data, and controlling an operation of the work device 5 according to a received operation signal. The control device 16 divides the image data into a plurality of image region data corresponding to a plurality of image regions of the image, based on an evaluation result of communication quality and an operation state of the work device 5, and sets a data reduction amount with respect to each of the plurality of image region data. Then, based on the data reduction amount for each of the plurality of set image region data, reduced image data for transmission is generated, and the image data for transmission is transmitted.SELECTED DRAWING: Figure 2

Description

[Technical field]

[0001] The present invention relates to a remotely controlled work machine and a remotely operated work machine system. [Background technology]

[0002] Patent Document 1 discloses a remote-controlled work machine system including a remotely controlled work machine and a remote control device that is located away from the work machine and remotely controls the work machine. The work machine includes a travelling body, a rotating body that is rotatably provided on the upper side of the travel body, a multi-joint work device connected to the front side of the rotating body, a camera device that captures an image in front of the rotating body, and a control device. The remote control device includes a display device that displays the image, an operation device that generates an operation signal in response to an operation by an operator, and a control device.

[0003] The control device of the work machine transmits the image captured by the imaging device to the remote control device as image data. The control device of the remote control device displays an image corresponding to the image data received from the work machine on the display device. The control device of the remote control device transmits an operation signal generated by the operation device to the work machine. The control device of the work machine controls the operation of the traveling body, the rotating body, and the work device in response to the operation signal received from the remote control device.

[0004] When the Internet or the like is used as a communication line between the work machine and the remote control device, it may be affected by other communication lines, resulting in a decrease in communication quality (specifically, determined by communication performance such as communication speed, delay time, and packet loss), and communication delays may occur. Video data has a larger data volume than operation signals, so communication delays are more likely to occur.

[0005] The control device for a work machine in Patent Document 1 calculates a communication delay time for the video data based on the time when the video data is transmitted to the remote control device and the time when the video data returned from the remote control device is received, and then multiplies the amount of operation indicated by the operation signal received from the remote control device by a gain corresponding to the communication delay time for the video data to limit the operation of the work machine. [Prior art documents] [Patent documents]

[0006] [Patent Document 1] JP 2019-068346 A Summary of the Invention [Problem to be solved by the invention]

[0007] The operator remotely operates the work machine using the control device while viewing the image of the surroundings of the work machine displayed on the display device of the remote control device. Therefore, if the quality of communication between the work machine and the remote control device deteriorates and a delay occurs in the display of the image, the operation of the control device will also be delayed, and the productivity of the work machine will decrease. In Patent Document 1, when a delay occurs in the display of the image, the image delay is not improved and the operation of the work machine is restricted, further decreasing the productivity of the work machine.

[0008] When a deterioration in communication quality occurs, it is conceivable to reduce the amount of video data (i.e., communication volume) in order to suppress the deterioration in communication quality. However, if the amount of data is reduced uniformly for all of the video data, that is, if the resolution is reduced uniformly for all of the video data, not only will image areas that are not necessary for the operation of the work machine become difficult to see, but also image areas that are necessary for the operation of the work machine, resulting in a decrease in productivity of the work machine.

[0009] The present invention has been made in consideration of the above-mentioned circumstances, and has an object to provide a work machine and a remotely operated work machine system that can accurately grasp the operating status and work status of the work machine in the remote control device even if the communication quality between the work machine and the remote control device deteriorates, and can suppress any decline in the productivity of the work machine. [Means for solving the problem]

[0010] In order to achieve the above objectives, RepresentativeThe present invention relates to a work machine including a working device, an imaging device for capturing images of the working device and its surroundings, and a control device for transmitting the images captured by the imaging device as image data to a remote control device via a communication device, and for controlling the operation of the working device in response to an operation signal received from the remote control device via the communication device, wherein the control device receives from the remote control device via the communication device an evaluation result of communication quality between the remote control device and the work device, determines an operation status of the working device based on the operation signal, divides the image data into a plurality of image area data corresponding to a plurality of image areas of the image, based on the evaluation result of communication quality and the operation status of the working device, sets a data reduction amount for each of the plurality of image area data, generates video data for transmission that has been reduced based on the set data reduction amount for each of the plurality of image area data, and transmits the video data for transmission to the remote control device via the communication device. Furthermore, when a moving object present around the work machine is detected by processing the image captured by the imaging device or by detection by a moving object detection sensor, and the presence of the moving object around the work machine is detected, the image data is divided into main image area data corresponding to a main image area including the moving object and its surroundings in the image, and sub-image area data corresponding to another image area, and data reduction is performed by either making the data reduction amount or reduction rate of the sub-image area data greater than that of the main image area data, or so that the data amount of the sub-image area data is smaller than the data amount of the main image area data. Effect of the Invention

[0011] According to the present invention, even if the communication quality between a work machine and a remote control device deteriorates, it is possible to cause the remote control device to display real-time, high-quality images without sacrificing the image quality of areas of the image area that can be displayed on the remote control device that are directly related to work, thereby suppressing delays in operation and work, and ultimately a decrease in the productivity of the work machine. [Brief description of the drawings]

[0012] [Figure 1] 1 is a schematic diagram showing a configuration of a remote-controlled shovel system in a first embodiment of the present invention. FIG. [Diagram 2] 1 is a block diagram showing a functional configuration of a control device for a shovel together with related devices in a first embodiment of the present invention. [Diagram 3] 1 is a block diagram showing a functional configuration of a control device of a remote control device together with related devices in a first embodiment of the present invention. [Figure 4]5 is a flowchart showing a procedure of a video data compression process of the control device for the excavator in the first embodiment of the present invention. [Diagram 5] 5A to 5C are diagrams for explaining a specific example of video data compression processing in the control device for the shovel in the first embodiment of the present invention. [Figure 6] 13A and 13B are diagrams for explaining a specific example of a video data compression process of a control device for a shovel in one modified example of the present invention. [Figure 7] FIG. 11 is a block diagram showing a functional configuration of a control device for a shovel together with related devices in a second embodiment of the present invention. [Figure 8] FIG. 11 is a block diagram showing a functional configuration of a control device for a shovel together with related devices in a third embodiment of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0013] A first embodiment of the present invention will be described with reference to the drawings. In this embodiment, a large excavator operating in a mine or the like is taken as an example of a work machine.

[0014] Fig. 1 is a schematic diagram showing the configuration of a remote-operated shovel system in this embodiment. Fig. 2 is a block diagram showing the functional configuration of a control device for a shovel in this embodiment together with related devices. Fig. 3 is a block diagram showing the functional configuration of a control device for a remote operation device in this embodiment together with related devices.

[0015] The shovel system 100 of this embodiment comprises a remotely operable shovel 1 and a remote control device 2 that is located at a location away from the shovel 1, provided outside the shovel 1, and configured to operate the shovel 1 by remote control.

[0016] The excavator 1 includes a travelling body 3, a rotating body 4 rotatably provided on the upper side of the travelling body 3, and an articulated working device 5 provided on the front side (left side in FIG. 1) of the rotating body 4. The travelling body 3 travels by the rotation of a travel motor (not shown), and the rotating body 4 rotates by the rotation of a rotation motor (not shown).

[0017] The working device 5 includes a boom 6 rotatably connected to the rotating body 4, an arm 7 rotatably connected to the tip of the boom 6, and a bucket 8 (work tool) rotatably connected to the tip of the arm 7. The boom 6 rotates by the extension and retraction of a boom cylinder 9, the arm 7 rotates by the extension and retraction of an arm cylinder 10, and the bucket 8 rotates by the extension and retraction of a bucket cylinder 11.

[0018] The excavator 1 includes a drive unit 12 (see FIG. 2) that drives a plurality of hydraulic actuators (more specifically, the above-mentioned traveling motor, swing motor, boom cylinder 9, arm cylinder 10, and bucket cylinder 11). The drive unit 12 includes, for example, an engine, a hydraulic pump and a pilot pump driven by the engine, a plurality of control valves that control the flow of pressure oil from the hydraulic pump to the plurality of hydraulic actuators, and a plurality of electromagnetic proportional valves that generate pilot pressures that operate the plurality of control valves using the discharge pressure of the pilot pump as a source pressure.

[0019] The shovel 1 is equipped with an imaging device 13 that captures images of the working equipment 5 and its surroundings, a plurality of angle sensors 14 (see FIG. 2) that detect the rotation angle of the rotating body 4, the rotation angle of the boom 6, the rotation angle of the arm 7, and the rotation angle of the bucket 8, a communication device 15 for communicating with the remote control device 2, and a control device 16.

[0020] The photographing device 13 is composed of five cameras in this embodiment, but some of them are not shown. The first camera photographs the area in front of the revolving body 4 at a middle height, the second camera photographs the area in front of the revolving body 4 at a higher height, and the third camera photographs the area in front of the revolving body 4 at a lower height. The first to third cameras photograph the working device 5 and its vicinity. The fourth camera photographs the area in front of the revolving body 4 on the left side, and photographs an image of a predetermined location on the left side of the working device 5. The fifth camera photographs the area in front of the revolving body 4 on the right side, and photographs an image of a predetermined location on the right side of the working device 5. The photographing device 13 may be composed of, for example, one wide-angle camera.

[0021] The control device 16 has a processor that executes processing according to a program, a memory that stores the program and data, etc. The control device 16 has, as its functional configuration, an operation signal receiving unit 17, a drive control unit 18, a communication quality receiving unit 19, an action recognition unit 20, a compression setting unit 21, a video data compression unit 22, and a video data transmission unit 23.

[0022] An operation signal receiving unit 17 of the control device 16 receives an operation signal (more specifically, an operation signal instructing the traveling body 3 to travel, the rotating body 4 to rotate, the boom 6 to rotate, the arm 7 to rotate, or the bucket 8 to rotate) from the remote control device 2 via the communication device 15. A drive control unit 18 of the control device 16 controls the drive device 12 (more specifically, a corresponding electromagnetic proportional valve) in response to the operation signal received by the operation signal receiving unit 17. This causes the traveling body 3 to travel, the rotating body 4 to rotate, the boom 6 to rotate, the arm 7 to rotate, or the bucket 8 to rotate.

[0023] The communication quality receiving unit 19 of the control device 16 receives an evaluation value (evaluation result) of the communication quality from the remote control device 2 via the communication device 15. The motion recognition unit 20 of the control device 16 recognizes the motion of the work implement 5 (more specifically, the positions of the boom 6, arm 7, and bucket 8 and the moving direction of the bucket 8) and the motion of the revolving unit 4 (more specifically, the rotation direction of the revolving unit 4) based on the detection results of the multiple angle sensors 14. The compression setting unit 21 of the control device 16 sets the compression rate and the like of data showing the video captured by the imaging device 13 (hereinafter referred to as video data) based on the evaluation value of the communication quality received by the communication quality receiving unit 19 and the motion of the work implement 5 or the revolving unit 4 recognized by the motion recognition unit 20 (details will be described later).

[0024] The video data compression unit 22 of the control device 16 generates video data for transmission by compressing the video data based on the compression ratio set by the compression setting unit 21. The video data transmission unit 23 of the control device 16 transmits the video data for transmission generated by the video data compression unit 22 to the remote control device 2 via the communication device 15.

[0025] The remote control device 2 includes a display device 24 that displays various information including images. The display device 24 has, for example, five display screens 25A, 25B, 25C, 25D, and 25E that correspond to the five cameras constituting the imaging device 13 of the shovel 1, respectively, and display the images captured by the five cameras, respectively.

[0026] The remote control device 2 includes an operating device 26 that generates an operating signal in response to an operation by an operator. The operating device 26 has a plurality of operating members (e.g., an operating lever and an operating pedal) that can be operated by the operator, and a plurality of potentiometers that generate operating signals corresponding to the operation directions and amounts of the plurality of operating members. In this way, an operating signal is generated that instructs the traveling of the traveling body 3 of the shovel 1, the rotation of the rotating body 4, the rotation of the boom 6, the rotation of the arm 7, or the rotation of the bucket 8.

[0027] The remote control device 2 includes a communication device 27 for communicating with the excavator 1, and a control device 28. The control device 28 includes a processor that executes processing according to a program, a memory that stores the program and data, etc. The control device 28 includes, as functional components, a video data receiving unit 29, a display control unit 30, a communication quality evaluation unit 31, a communication quality transmission unit 32, and an operation signal transmission unit 33.

[0028] The video data receiving unit 29 of the control device 28 receives the video data from the shovel 1 via the communication device 27. The display control unit 30 of the control device 28 causes the display device 24 to display an image corresponding to the video data received by the video data receiving unit 29.

[0029] In this embodiment, the communication quality evaluation unit 31 of the control device 28 in the remote operation device 2 evaluates the quality of communication between the shovel 1 and the remote operation device 2 based on the video data received by the video data receiving unit 29. To explain in detail, the communication quality evaluation unit 31 calculates the delay time required for communication of the video data, for example, from the difference between the transmission cycle (fixed value) and the reception cycle (variable value) of the video data. Then, based on the communication delay time and packet loss of the video data, it calculates an evaluation value of the communication quality (more specifically, the lower the communication quality, the smaller the numerical value). The communication quality transmission unit 32 of the control device 28 transmits the evaluation value of the communication quality calculated by the communication quality evaluation unit 31 to the shovel 1 via the communication device 27.

[0030] Next, details of the compression process of the video data in the control device 16 of the shovel 1 will be described with reference to Fig. 4. Fig. 4 is a flowchart showing the procedure of the video data compression process in the control device of the shovel in this embodiment.

[0031] In step S1, the communication quality receiving unit 19 of the control device 16 receives the evaluation value of the communication quality from the remote control device 2 via the communication device 15. In step S2, the compression setting unit 21 of the control device 16 determines whether the evaluation value of the communication quality received by the communication quality receiving unit 19 is less than a preset threshold value.

[0032] If the evaluation value of communication quality is equal to or greater than the threshold value (in other words, communication quality has not deteriorated), the process proceeds to step S3. In step S3, the compression setting unit 21 of the control device 16 sets a standard compression rate so that the total data volume of the video data becomes a standard value. Furthermore, the video data compression unit 22 of the control device 16 uniformly compresses all of the video data representing the video captured by the imaging device 13 (in this embodiment, videos A1 to A5 captured by five cameras) at the standard compression rate to generate video data for transmission. Then, the video data transmission unit 23 transmits the compressed video data for transmission.

[0033] On the other hand, if the evaluation value of communication quality is less than the threshold value (in other words, if the communication quality has deteriorated), the process proceeds to step S4. In step S4, the compression setting unit 21 of the control device 16 sets the overall data reduction amount of the video data (more specifically, the amount to be reduced from the standard value) to be greater as the evaluation value of the communication quality becomes lower. In other words, the compression setting unit 21 sets the compression rate of the video data to be greater as the evaluation value of the communication quality becomes lower. Also, in step S5, the action recognition unit 20 determines whether the action of the working implement 5 or the action of the revolving body 4 has been recognized.

[0034] If the operation of the work implement 5 and the operation of the revolving body 4 are not recognized, the process proceeds to step S3. In step S3, the video data compression unit 22 of the control device 16 uniformly compresses the entire video captured by the imaging device 13 at the compression rate set in step S4, and the video data transmission unit 23 transmits the compressed video.

[0035] On the other hand, if the operation of the working device 5 or the revolving body 4 is recognized in step S5, the process proceeds to step S6. In step S6, the compression setting unit 21 of the control device 16 divides (partitions) the image into a plurality of image areas, and sets a priority order of data reduction for each of the plurality of image data (hereinafter referred to as image area data) corresponding to the plurality of image areas. Thereafter, in step S7, the compression setting unit 21 sets the data reduction amount or reduction rate for each of the plurality of image area data based on the priority order of data reduction set in step S6 so that the data reduction amount becomes the total data reduction amount of the image data set in step S4. That is, the compression setting unit 21 sets the data reduction amount (compression rate) of the image data to be displayed on the display device 24 individually for each image area of ​​the image, based on the operating status of the working device 5 and the communication status with the remote control device 2.

[0036] More specifically, the compression setting unit 21 of the control device 16 selects video data in which the bucket 8 of the work device 5 appears from the video data corresponding to the videos A1 to A5 captured by the five cameras by recognizing the current or previous operation of the work device 5. For example, as shown in Fig. 5, the video data corresponding to the selected video A1 is divided into main video area data corresponding to a main video area R1 showing the bucket 8 of the work device 5 and its surroundings, video area data corresponding to a video area R2 showing the boom 6 of the work device 5, and sub-video area data corresponding to another video area R3.

[0037] The remaining image data corresponding to the images A2 to A5 are treated the same as the image data corresponding to the image A1. That is, for example, if the boom 6 of the working device 5 does not appear, it is treated the same as the image area data corresponding to the image area R3 of the image A1. The first priority of data reduction is set to the image area data corresponding to the image area R2 of the image A1, the second priority is set to the image area data corresponding to the image area R3 of the image A1 and the image area data corresponding to the images A2 to A5, and the third priority is set to the image area data corresponding to the image area R1 of the image A1. Then, the data reduction amounts of the image area data corresponding to the image areas R1, R2, and R3 of the image A1 and the images A2 to A5 are set based on the above-mentioned data reduction priority so as to be the total data reduction amount of the image data set in step S4.

[0038] Then, in step S7, the video data compression unit 22 of the control device 16 generates video data for transmission by non-uniformly compressing the video captured by the imaging device 13 using the data reduction amount of the video area data, which is the video data for each video area set in step S6. The video data transmission unit 23 transmits the compressed video data for transmission.

[0039] As described above, in this embodiment, when the quality of communication between the shovel 1 and the remote control device 2 is degraded, the total data amount of video data is reduced. This makes it possible to suppress the degradation of communication quality. As a result, it is possible to prevent delays in the display of video that is directly linked to understanding the work situation, suppress the associated delays in the operation of the control device 26, and suppress degradation of the productivity of the shovel 1.

[0040] In this embodiment, the image is divided into a plurality of image areas including a main image area R1 showing the bucket 8 of the working device 5 and its surroundings and other image areas according to the operation of the working device 5 or the operation of the revolving body 4, and main image area data, which is image area data corresponding to the main image area R1, and sub-image area data corresponding to the other image areas are generated. Then, a priority order of data reduction is set so that the sub-image area data has priority over the main image area data. For example, data reduction is performed so that the reduction amount or reduction rate of the sub-image area data is larger than that of the main image area data, or the data amount of the sub-image area data is larger than that of the main image area data. This suppresses the decrease in resolution of the main image area R1 in the image displayed on the display device 24. That is, the area necessary for the operation of the excavator 1 is automatically extracted, and the decrease in the resolution can be suppressed. Therefore, the decrease in productivity of the excavator 1 can be suppressed.

[0041] In the first embodiment, the control device 16 of the excavator 1 sets the main image area R1 as an elliptical area with the center of the part of the bucket 8 to be watched (specifically, the opening and claws of the bucket 8) as the center point of the ellipse as shown in FIG. 5, but the present invention is not limited to this. The control device 16 may set the main image area R1 as an elliptical area with the center of the part of the bucket 8 to be watched as the focus of the ellipse as shown in FIG. 6(a) or FIG. 6(b). That is, as shown in FIG. 6(a), when the bucket 8 moves downward, the shape and range of the main image area R1 may be set so that the area on the moving direction side is larger than the area on the opposite direction side with the area of ​​the bucket 8 as a reference. Also, as shown in FIG. 6(b), when the bucket 8 moves leftward, the main image area R1 may be set to a shape such that the area on the left side is larger than the area on the right side with the area of ​​the bucket 8 as a reference.

[0042] In addition, in the first embodiment, when the operation of the rotating body 4 is recognized, the control device 16 of the shovel 1 divides the image into a plurality of image areas including a main image area R1 showing the bucket 8 of the working implement 5 and its surroundings and other image areas, just as when the operation of the working implement 5 is recognized, and performs compression processing for each image area data corresponding to each image area (for example, by changing the data reduction rate between the main image area data and the sub-image area data), but this is not limited to this.

[0043] For example, when the operation of the revolving body 4 is recognized, the control device 16 may divide the image into a plurality of image areas including a main image area showing the movement destination of the working device 5 and other image areas, and perform compression processing on the image area data (main image area data and sub-image area data, respectively) corresponding to the divided image areas. That is, for example, when the working device 5 moves to the left as the revolving body 4 turns left, the image data corresponding to the images A1 to A5 captured by the five cameras may be divided into main image area data corresponding to the image A4 (main image area) and sub-image area data corresponding to the other images A1-A3, A5 (other image areas), and a data reduction priority order may be set so that the sub-image area data corresponding to the images A1-A3, A5 is compressed with priority over the main image area data corresponding to the image A4. Furthermore, for example, when the work implement 5 moves to the right as the rotating body 4 rotates to the right, the image data corresponding to images A1 to A5 captured by the five cameras may be divided into main image area data corresponding to image A5 (main image area) and sub-image area data corresponding to other images A1-A4 (other image areas), and a data reduction priority may be set so that the sub-image area data corresponding to images A1-A4 is given priority over the main image area data corresponding to image A5.

[0044] A second embodiment of the present invention will be described with reference to Fig. 7. Fig. 7 is a block diagram showing the functional configuration of a control device for a shovel in this embodiment together with related devices. In this embodiment, the same parts as those in the first embodiment are given the same reference numerals, and the description thereof will be omitted as appropriate.

[0045] The control device 16A of the shovel 1 in this embodiment has, similar to the control device 16 in the first embodiment, an operation signal receiving unit 17, a drive control unit 18, a communication quality receiving unit 19, an action recognition unit 20, a compression setting unit 21, a video data compression unit 22, and a video data transmission unit 23.

[0046] The control device 16A further includes a moving object detection unit 34. The moving object detection unit 34 of the control device 16A detects a dump truck (moving object) present around the excavator 1 by processing the image captured by the imaging device 13 (or by detection by the moving object detection sensor 35).

[0047] The compression setting unit 21 of the control device 16A sets the overall data reduction amount of the video data when the evaluation value of the communication quality is less than the threshold value. Then, when the moving object detection unit 34 detects a dump truck, the compression setting unit 21 selects the video data in which the dump truck appears from the video data corresponding to the videos A1 to A5 captured by the five cameras. For example, the compression setting unit 21 divides the video data corresponding to the selected video A4 into main video area data corresponding to the main video area showing the dump truck and its surroundings, and sub-video area data corresponding to the other video areas.

[0048] The remaining video data corresponding to the videos A1-A3, A5 are treated the same as the video data corresponding to the video A4. That is, for example, if a dump truck does not appear, it is treated the same as the video area data corresponding to the other video areas of the video A4. The first priority of data reduction is set to the video area data corresponding to the other video areas of the video A4 and the video area data corresponding to the videos A1-A3, A5, and the second priority is set to the video area data corresponding to the main video area of ​​the video A4. The data reduction amounts of the main video area of ​​the video A4, the other video areas, and the video area data corresponding to the videos A1-A3, A5 are set based on the above-mentioned data reduction priority so as to achieve the set total data reduction amount of the video data.

[0049] In the present embodiment configured as described above, similarly to the first embodiment, it is possible to suppress a decrease in the productivity of the shovel 1 while suppressing a decrease in the quality of communication between the shovel 1 and the remote control device 2.

[0050] A third embodiment of the present invention will be described with reference to Fig. 8. Fig. 8 is a block diagram showing the functional configuration of a control device for a shovel in this embodiment together with related devices. In this embodiment, parts equivalent to those in the first and second embodiments are given the same reference numerals, and descriptions thereof will be omitted as appropriate.

[0051] The control device 16B of the shovel 1 in this embodiment, like the control device 16A in the second embodiment, has an operation signal receiving unit 17, a drive control unit 18, a communication quality receiving unit 19, a compression setting unit 21, a video data compression unit 22, a video data transmission unit 23, and a moving object detection unit 34.

[0052] The control device 16B has a motion prediction unit 36 ​​instead of the motion recognition unit 20 of the control device 16A of the second embodiment. Like the motion recognition unit 20, the motion prediction unit 36 ​​of the control device 16B recognizes the motion of the working device 5 and the motion of the revolving body 4 from the detection results of multiple angle sensors. Then, based on the progress of the motion of the working device 5 or the motion of the revolving body 4, and the presence or absence of detection of a dump truck by the mobile body detection unit 34, the motion prediction unit 36 ​​learns and stores the motion patterns of the shovel related to each work of the shovel 1. Then, using the above-mentioned motion patterns, future motion of the shovel is predicted from the current motion of the shovel or the detection of the current vehicle.

[0053] As a specific example, a description will be given using a motion pattern of a shovel related to a loading operation of the shovel 1. In this motion pattern, first, for example, a dump truck appears at a predetermined location away from the left or right side of the working device 5 of the shovel 1. Then, the rotating body 4 of the shovel 1 rotates so as to face the dump truck. Then, the working device 5 of the shovel 1 operates.

[0054] When the moving object detection unit 34 detects a dump truck, the operation prediction unit 36 ​​of the control device 16B predicts that the revolving body 4 of the excavator 1 will revolve toward the dump truck. The compression setting unit 21 of the control device 16B selects video data in which the dump truck appears from among the video data corresponding to the videos A1 to A5 captured by the five cameras. For example, the compression setting unit 21 divides the video data corresponding to the selected video A4 into main video area data corresponding to a main video area showing the dump truck and its surroundings, and sub-video area data corresponding to other video areas. In addition, the compression setting unit 21 selects video data in which the bucket 8 of the working device 5 appears from among the video data corresponding to the videos A1 to A5 captured by the five cameras. For example, the compression setting unit 21 divides the video data corresponding to the selected video A1 into main video area data corresponding to a main video area R1 showing the bucket 8 of the working device 5 and its surroundings, video area data corresponding to a video area R2 showing the boom 6 of the working device 5, and sub-video area data corresponding to other video area R3 (see FIG. 5 or FIG. 6(b) above).

[0055] The remaining video data corresponding to the videos A2, A3, and A5 are treated the same as the video data corresponding to the video A1. That is, for example, if the boom 6 of the working device 5 does not appear, it is treated the same as the video area data corresponding to the video area R3 of the video A1. The first priority of data reduction is set to the video area data corresponding to the video area R2 of the video A1, the second priority is set to the video area data corresponding to the video area R3 of the video A1, the video area data corresponding to the other video areas of the video A4, and the video area data corresponding to the videos A2, A3, and A5, and the third priority is set to the video area data corresponding to the video area R1 of the video A1 and the video area data corresponding to the main video area of ​​the video A4. Then, the data reduction amounts of the video areas R1, R2, and R3 of the video A1, the main video area and other video areas of the video A4, and the video area data corresponding to the videos A2, A3, and A5 are set based on the above-mentioned data reduction priority so that the total data reduction amount of the video data is set according to the deterioration of the communication quality.

[0056] In the present embodiment configured as described above, similarly to the first embodiment, it is possible to suppress a decrease in the productivity of the shovel 1 while suppressing a decrease in the quality of communication between the shovel 1 and the remote control device 2.

[0057] In the first to third embodiments, the shovel 1 has been described as including a plurality of angle sensors 14 that detect the rotation angle of the boom 6, the rotation angle of the arm 7, and the rotation angle of the bucket 8, respectively, as sensors that detect state quantities related to the operation of the working implement 5, but this is not limiting. The shovel 1 may also be provided with a plurality of inertial measurement units that detect, for example, the inclination angle of the boom 6, the inclination angle of the arm 7, and the inclination angle of the bucket 8, respectively. The operation of the working implement 5 may then be recognized based on the detection results of the plurality of inertial measurement units.

[0058] In the first to third embodiments, the control device 16 of the excavator 1 recognizes the operation of the work implement 5 and the operation of the revolving body 4 based on the detection results of the multiple angle sensors 14, but this is not limiting. The control device 16 may recognize the operation of the work implement 5 and the operation of the revolving body 4 based on an operation signal received from the remote control device 2.

[0059] In the above, the working machine is a large shovel, but the present invention is not limited to this. For example, the working machine may be a medium or small shovel, or may be a working machine other than a shovel (more specifically, a crane truck, etc.).

[0060] A first invention relates to a remotely operated work machine system comprising a work machine having a work implement, an imaging device which captures images of the work implement and its surroundings, and a first control device which transmits the images captured by the imaging device as image data via a first communication device and controls the operation of the work implement in response to an operation signal received via the first communication device, and a remote operation device having a display device which displays various information, an operation device which generates the operation signal in response to operation by an operator, and a second control device which causes an image corresponding to the image data received from the work machine via a second communication device to be displayed on the display device and transmits the operation signal generated by the operation device to the work machine via the second communication device, and a second control device of the work machine evaluates the quality of communication with the work machine and transmits the evaluation result to the work machine, and the first control device of the work machine determines the operating status of the work machine based on the operation signal, and when it receives from the remote control device an evaluation result indicating that the communication quality between the work machine and the remote control device has deteriorated and determines that the work machine is operating, divides the video data into a plurality of image area data corresponding to a plurality of image areas of the video, sets a data reduction amount for each of the plurality of image area data, generates video data for transmission that is reduced based on the set data reduction amount for each of the plurality of image area data, and transmits the video data for transmission to the remote control device.

[0061] A second invention relates to a remotely operated work machine system including a work machine having a work implement, an imaging device which captures images of the work implement and its surroundings, and a first control device which transmits the images captured by the imaging device as image data via a first communication device and controls the operation of the work implement in response to an operation signal received via the first communication device; and a remote operation device including a display device which displays various information, an operation device which generates the operation signal in response to operation by an operator, and a second control device which causes an image corresponding to the image data received from the work machine via a second communication device to be displayed on the display device and transmits the operation signal generated by the operation device to the work machine via the second communication device, and a second control device of the work machine evaluates the quality of communication with the work machine and transmits the evaluation result to the work machine, and the first control device of the work machine determines the operating status of the work machine based on the operation signal, and when it receives from the remote control device an evaluation result indicating that the communication quality between the work machine and the remote control device has deteriorated and determines that the work machine is operating, divides the video data into a plurality of image area data corresponding to a plurality of image areas of the video, sets a data reduction amount for each of the plurality of image area data, generates video data for transmission that is reduced based on the set data reduction amount for each of the plurality of image area data, and transmits the video data for transmission to the remote control device.

[0062] According to the first or second invention, even if the communication quality between the work machine and the remote control device deteriorates, for example, it is possible to cause the remote control device to display real-time, high-quality images without sacrificing the image quality of areas directly related to work among the image areas that can be displayed on the remote control device, thereby suppressing delays in operation and work, and ultimately a decrease in the productivity of the work machine. [Explanation of symbols]

[0063] 1. Shovel (working machine) 2 Remote control device 3. Vehicle 4 Rotating body 5. Working Equipment 8 Bucket (work tool) 13 Imaging Equipment 15. Communications Equipment 16, 16A, 16B Control device 24 Display device 26 Operating device 27 Communication Equipment 28 Control device 35 Moving object detection sensor 100 Shovel system (work machine system)

Claims

1. A work machine including a working device, a photographing device that photographs the working device and its surroundings, and a control device that transmits the image photographed by the photographing device as image data to a remote control device via a communication device and controls an operation of the working device in response to an operation signal received from the remote control device via the communication device, The control device includes: receiving from the remote control device via the communication device an evaluation result of communication quality between the remote control device and the remote control device, determining an operation status of the working device based on the operation signal, dividing the video data into a plurality of video area data corresponding to a plurality of video areas of the video based on the evaluation result of communication quality and the operation status of the working device, setting a data reduction amount for each of the plurality of video area data, generating video data for transmission that is reduced based on the set data reduction amount for each of the plurality of video area data, and transmitting the video data for transmission to the remote control device via the communication device; Further, a working machine characterized in that when a moving object present around the work machine is detected by processing the image captured by the imaging device or by detection by a moving object detection sensor, and the presence of the moving object around the work machine is detected, the image data is divided into main image area data corresponding to a main image area of ​​the image including the moving object and its surroundings, and sub-image area data corresponding to another image area, and the data reduction amount or reduction rate of the sub-image area data is made greater than that of the main image area data, or the data reduction is performed so that the data amount of the sub-image area data is smaller than the data amount of the main image area data.

2. A work machine comprising a running body, a rotating body supported so as to be rotatable relative to the running body, a working device, a camera for taking images of the working device and its surroundings, and a control device for transmitting the images taken by the camera to a remote control device via a communication device as video data, and for controlling the operation of the running body, the rotating body, and the working device in response to an operation signal received from the remote control device via the communication device, The control device includes: receiving from the remote control device via the communication device an evaluation result of communication quality between the remote control device and the remote control device, determining an operation status of the working device based on the operation signal, dividing the video data into a plurality of video area data corresponding to a plurality of video areas of the video based on the evaluation result of communication quality and the operation status of the working device, setting a data reduction amount for each of the plurality of video area data, generating video data for transmission that is reduced based on the set data reduction amount for each of the plurality of video area data, and transmitting the video data for transmission to the remote control device via the communication device; Furthermore, the operating status of the running body and the rotating body is judged based on the operation signal, and when an evaluation result indicating that the communication quality is low is received and it is judged that the rotating body is operating, the video data is divided into main video area data corresponding to a main video area including the destination of the working device in the video and sub-video area data corresponding to another video area, and the data reduction amount or reduction rate of the sub-video area data is made larger than that of the main video area data, or data reduction is performed so that the data amount of the sub-video area data is smaller than the data amount of the main video area data, The work machine further comprises: learning and storing an operation pattern of the work machine; using the operation pattern of the work machine to predict at least one of a future operation of the work implement and a future operation of the rotating body from a current operation of the work machine or the presence of a moving object around the work machine; and dividing the video data into a plurality of image area data corresponding to a plurality of image areas of the video according to the predicted at least one of the operation of the work implement and the operation of the rotating body, and setting a data reduction amount for each of the plurality of image area data.

3. 3. The working machine according to claim 1 or 2, The control device includes: A work machine characterized in that, when an evaluation result indicating that the communication quality is low is received and it is determined that the work device is operating, the video data is divided into main image area data corresponding to a main image area of ​​the image including the tip of the work device and its surroundings, and sub-image area data corresponding to another image area, and data is reduced by either making the amount or rate of reduction of the sub-image area data greater than that of the main image area data, or by making the data amount of the sub-image area data smaller than the data amount of the main image area data.

4. 4. The working machine according to claim 3, The control device sets the shape of the main image area so that the image area on the side of the movement direction of the work implement is larger than the image area on the side opposite the movement direction of the work implement, based on the image area including the work implement.

Images