Work machine operation system and work machine

Abstract

To provide a work machine capable of easily grasping a situation around the work machine.SOLUTION: A work machine operation system includes a first work machine, a second work machine, a receiving device which receives an image transmitted from the first work machine or the second work machine, and a display device which is provided in the vicinity of an operation seat for operating the first work machine. The first work machine includes a first undercarriage, a first revolving super structure revolvably mounted on the first undercarriage, a first image capturing device provided on the first revolving super structure so as to capture an image in front of the first work machine, and a first transmitting device for transmitting the image captured by the first image capturing device. The second work machine includes a second undercarriage, a second revolving super structure revolvably mounted on the second undercarriage, a second image capturing device provided on the second revolving super structure so as to capture an image in the vicinity of the second work machine, and a second transmitting device for transmitting the image captured by the second image capturing device. The display device displays the image of the front captured by the first work machine and the image of the vicinity captured by the second work machine.SELECTED DRAWING: Figure 4

Description

[Technical Field]

[0001] The present disclosure relates to a work machine operation system and a work machine. [Background technology]

[0002] Conventionally, there has been a demand for providing operators of work machines with appropriate information about the surrounding environment in which the work machine is working. For example, Patent Document 1 discloses a display system that displays, on a display device external to the work machine, images of the front and side directions taken by an imaging device mounted on the work machine. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent No. 6909641 Summary of the Invention [Problem to be solved by the invention]

[0004] It is difficult to grasp the situation around the work machine using images taken in the forward direction or the lateral direction by an imaging device mounted on the work machine.

[0005] An object of one aspect of the present disclosure is to provide a work machine operation system that makes it easy to grasp the situation around the work machine. [Means for solving the problem]

[0006] A work machine remote control system according to one aspect of the present disclosure is a work machine operation system including a first work machine, a second work machine, a receiving device that receives images transmitted by the first work machine or the second work machine, and a display device provided in the vicinity of an operator's seat that operates the first work machine, wherein the first work machine has a first undercarriage, a first upper rotating body rotatably mounted on the first undercarriage, and a first imaging device that is provided on the first upper rotating body and that images an area ahead of the first work machine. The first work machine comprises a first imaging device and a first transmitting device that transmits images captured by the first imaging device, and the second work machine comprises a second undercarriage, a second upper rotating body rotatably mounted on the second undercarriage, a second imaging device provided on the second upper rotating body that captures images of the surroundings of the second work machine, and a second transmitting device that transmits images captured by the second imaging device, and the display device displays an image of the front captured by the first work machine and an image of the surroundings captured by the second work machine. [Effects of the Invention]

[0007] According to one aspect of the present disclosure, an image of the surroundings captured by an imaging device provided in another work machine is displayed, making it possible to easily grasp the situation around the work machine. [Brief explanation of the drawings]

[0008] [Figure 1] 1 is a schematic diagram illustrating an example of a remote control system according to a first embodiment. [Figure 2] 1 is a side view showing a work machine according to a first embodiment. [Figure 3] 1 is a diagram showing an example of the configuration of a drive control system of a work machine according to a first embodiment. [Figure 4] 1 is a functional block diagram showing an example of the configuration of a remote control system according to a first embodiment. [Figure 5] FIG. 4 is a diagram showing a specific range according to the first embodiment. [Figure 6] FIG. 2 is a diagram showing an example of the layout of a remote control room according to the first embodiment. [Figure 7] FIG. 4 is a sequence diagram showing an example of image display processing according to the first embodiment. [Figure 8] FIG. 4 is a side view showing a work machine according to a second embodiment. [Figure 9] 1 is a diagram for explaining an example of a state in which a work machine equipped with an imaging device whose angle of view cannot be adjusted is working; [Figure 10] 1 is a diagram for explaining an example of a state in which a work machine equipped with an imaging device with an adjustable angle of view is working. [Figure 11] FIG. 2 is a diagram for explaining an example of control for operating two work machines from one remote control room. [Figure 12] FIG. 10 is a sequence diagram showing an example of an image display process according to the second embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0009] Hereinafter, embodiments of the present disclosure will be described with reference to the drawings. The embodiments described below are merely examples and do not limit the invention, and all features and combinations described in the embodiments are not necessarily essential to the invention. In addition, identical or corresponding components in each drawing are denoted by identical or corresponding reference numerals, and descriptions thereof may be omitted.

[0010] The work machine 100 according to the embodiment of the present disclosure is a shovel. The work machine 100 may be a machine other than a shovel, such as a crane, an asphalt finisher, or a forklift. In the illustrated example, the shovel serving as the work machine 100 is an excavator equipped with a bucket 6 as an end attachment, but the work machine 100 may also be an applied machine such as a forestry machine equipped with an end attachment other than the bucket 6.

[0011] (First embodiment) An overview of the remote control system SYS according to the first embodiment will be described with reference to Fig. 1. Fig. 1 is a schematic diagram showing an example of the remote control system SYS according to the first embodiment.

[0012] <Devices that make up the remote control system> As shown in FIG. 1, the remote operation system SYS according to the first embodiment includes two work machines 100 (100-1, 100-2), two remote operation rooms RC (RC-1, RC-2), and a server SVR.

[0013] Hereinafter, when there is more than one work machine 100, they will be distinguished from one another by using sub-numbers such as "work machine 100-1," "work machine 100-2," etc. When simply referring to a "work machine 100," this applies to all work machines 100. Similarly, when distinguishing between the remote control rooms RC, they will be described as "remote control room RC-1," "remote control room RC-2," etc.

[0014] The work machine 100, the remote control room RC, and the server SVR are connected to one another so that they can send and receive data via the communication network NW. Note that the work machine 100, the remote control room RC, and the server SVR may also be connected to one another so that they can send and receive data directly to one another without going through the communication network NW.

[0015] In the illustrated example, the work machine 100-1 transmits information related to the work site to the remote control room RC-1. This allows the operator OP in the remote control room RC-1 to understand the situation at the work site based on the information from the work machine 100-1. The remote control room RC-1 transmits operation signals to the work machine 100-1 to control the operation of the work machine 100-1. This allows the operator OP in the remote control room RC-1 to operate the work machine 100-1 from the remote control room RC-1.

[0016] Similarly, the work machine 100-2 transmits information regarding the work site to the remote control room RC-2. This allows the operator OP in the remote control room RC-2 to understand the situation at the work site based on the information from the work machine 100-2. The remote control room RC-2 transmits operation signals to the work machine 100-2 to control the operation of the work machine 100-2. This allows the operator OP in the remote control room RC-2 to operate the work machine 100-2 from the remote control room RC-2.

[0017] The remote operation system SYS may include three or more work machines 100. This allows the remote operation system SYS to provide information about the work site to the remote operation room RC through the three or more work machines 100.

[0018] The server SVR is, for example, a server computer (a so-called cloud server) or an edge server. The server SVR is typically a fixed terminal device, but may also be a portable terminal device (for example, a laptop computer, a tablet, or a smartphone).

[0019] The server SVR accumulates data received from the work machine 100. In response to a request from the remote control room RC, the server SVR transmits the data received from the work machine 100 to the remote control room RC. For example, the server SVR accumulates images captured by an imaging device mounted on the work machine 100 and transmits them to the remote control room RC. This allows the operator OP in the remote control room RC to check images of the work site captured by the work machine 100.

[0020] The installation location of the server SVR is not limited as long as it is capable of communicating with the work machine 100 and the remote control room RC. The server SVR may be installed in the remote control room RC. The server SVR may be installed in a control room at the work site where the work machine 100 works. The server SVR may be installed at a location (for example, a data center) different from both the work site and the remote control room RC.

[0021] <Example of remote control room configuration> The remote control room RC is equipped with a communication device T2, a remote controller 40, an operation device 42, an operation sensor 43, and a display device D1. The remote control room RC also is equipped with an operation seat DS where an operator OP who remotely operates the work machine 100 sits.

[0022] The communication device T2 is configured to control communication between the communication device T1 (see FIG. 2) attached to the work machine 100 and the server SVR.

[0023] The remote controller 40 is an information processing device that executes various calculations. In this embodiment, the remote controller 40 is configured as a microcomputer including a CPU and a memory. The various functions of the remote controller 40 are realized by the CPU executing programs stored in the memory.

[0024] The display device D1 is a device capable of displaying various types of information. The display device D1 displays a screen based on information transmitted from the work machine 100 so that the operator OP in the remote control room RC can visually confirm the surroundings of the work machine 100. By referring to the display device D1, the operator OP can confirm the status of the work site, including the surroundings of the work machine 100, even though he is in the remote control room RC. In the illustrated example, the display device D1 is a liquid crystal display that displays images captured by an imaging device mounted on the work machine 100. Note that the display device D1 may be a display or projector that realizes stereoscopic vision with the naked eye, or may be VR goggles or the like.

[0025] The operation device 42 is provided with an operation sensor 43 for detecting the operation content of the operation device 42. The operation sensor 43 is, for example, an inclination sensor that detects the inclination angle of the operation lever, or an angle sensor that detects the swing angle of the operation lever around the swing axis. The operation sensor 43 may be composed of other sensors such as a pressure sensor, a current sensor, a voltage sensor, or a distance sensor. The operation sensor 43 outputs information related to the detected operation content of the operation device 42 to the remote controller 40. The remote controller 40 generates an operation signal based on the information received from the operation sensor 43 and transmits the generated operation signal to the work machine 100. The operation sensor 43 may be configured to generate an operation signal. In this case, the operation sensor 43 may output the operation signal to the communication device T2 without going through the remote controller 40. This makes it possible to remotely operate the work machine 100 from the remote control room RC.

[0026] The remote controller 40 or operation sensor 43 may transmit an operation signal to the work machine 100 via the server SVR. For example, the remote controller 40 or operation sensor 43 generates an operation signal with identification information added to identify the work machine 100 to be operated, and transmits the operation signal to the server SVR. The server SVR identifies the work machine 100 based on the identification information added to the operation signal, and transmits the operation signal to the identified work machine 100.

[0027] <Example of work machine configuration> An overview of the work machine 100 according to the first embodiment will be described with reference to Fig. 2. Fig. 2 is a side view of the work machine 100 according to the first embodiment.

[0028] An upper rotating body 3 is rotatably mounted on a lower traveling body 1 of the work machine 100 via a rotating mechanism 2. A boom 4 is attached to the upper rotating body 3. An arm 5 is attached to the tip of the boom 4, and a bucket 6 is attached to the tip of the arm 5 as an end attachment. The end attachment may be a slope bucket, a dredging bucket, or the like.

[0029] In the example shown in FIG. 2, the direction of travel (front-to-rear direction) of the work machine 100 is indicated by the X axis, the width direction of the work machine 100 is indicated by the Y axis, and the height direction of the work machine 100 is indicated by the Z axis.

[0030] Note that in the work machine 100, all or some of the driven parts, such as the lower traveling body 1, upper rotating body 3, boom 4, arm 5, and bucket 6, may be electrically driven. In other words, the work machine 100 may be a hybrid excavator, an electric excavator, or the like, in which all or some of the driven parts are driven by electric actuators.

[0031] The boom 4, arm 5, and bucket 6 constitute an excavation attachment, which is an example of an attachment, and are hydraulically driven by a boom cylinder 7, an arm cylinder 8, and a bucket cylinder 9, respectively. A boom angle sensor S1 is attached to the boom 4, an arm angle sensor S2 is attached to the arm 5, and a bucket angle sensor S3 is attached to the bucket 6. The excavation attachment may be provided with a bucket tilt mechanism.

[0032] The boom angle sensor S1 detects the rotation angle of the boom 4. In this embodiment, the boom angle sensor S1 is an acceleration sensor, and can detect the boom angle, which is the rotation angle of the boom 4 relative to the upper rotating body 3. For example, the boom angle is at its minimum when the boom 4 is lowered to the lowest position, and increases as the boom 4 is raised.

[0033] The arm angle sensor S2 detects the rotation angle of the arm 5. In this embodiment, the arm angle sensor S2 is an acceleration sensor, and can detect the arm angle, which is the rotation angle of the arm 5 relative to the boom 4. For example, the arm angle is at its smallest when the arm 5 is fully closed, and increases as the arm 5 opens.

[0034] The bucket angle sensor S3 detects the rotation angle of the bucket 6. In this embodiment, the bucket angle sensor S3 is an acceleration sensor, and can detect the bucket angle, which is the rotation angle of the bucket 6 with respect to the arm 5. For example, the bucket angle is at its smallest when the bucket 6 is fully closed, and increases as the bucket 6 opens.

[0035] The boom angle sensor S1, arm angle sensor S2, and bucket angle sensor S3 may be a potentiometer using a variable resistor, a stroke sensor that detects the stroke amount of the corresponding hydraulic cylinder, or a rotary encoder that detects the rotation angle around the connecting pin. The boom angle sensor S1, arm angle sensor S2, and bucket angle sensor S3 form a posture sensor that detects the posture of the excavation attachment.

[0036] The upper rotating body 3 is equipped with a cabin 10 as a driver's cab, an engine 11, a body tilt sensor S4, a turning angular velocity sensor S5, a surrounding image capturing device S6, a front image capturing device S7, a positioning device S8, a communication device T1, and the like.

[0037] A controller 30 is installed inside the cabin 10. Also installed inside the cabin 10 are a driver's seat, operating devices, and the like.

[0038] The controller 30 is a calculation device that executes various calculations. The controller 30 is provided, for example, inside the cabin 10, and controls the drive of the work machine 100. The functions of the controller 30 may be realized by any hardware, software, or a combination thereof. For example, the controller 30 is configured mainly with a microcomputer that includes a CPU (Central Processing Unit), a memory device such as RAM (Random Access Memory), a non-volatile auxiliary storage device such as ROM (Read Only Memory), and various interface devices for input and output. The controller 30 realizes various functions, for example, by executing, on the CPU, various programs installed in the non-volatile auxiliary storage device.

[0039] The engine 11 is a drive source for the work machine 100. In this embodiment, the engine 11 is a diesel engine. An output shaft of the engine 11 is connected to input shafts of the main pump 14 and the pilot pump 15, respectively.

[0040] The machine body tilt sensor S4 is configured to detect the tilt of the upper rotating body 3 with respect to a predetermined plane. In this embodiment, the machine body tilt sensor S4 is an acceleration sensor that detects the tilt angle about the longitudinal axis and the lateral axis of the upper rotating body 3 with respect to a horizontal plane. The longitudinal axis and the lateral axis of the upper rotating body 3 are, for example, perpendicular to each other and pass through a center point that is a point on the rotation axis of the work machine 100.

[0041] The rotation angular velocity sensor S5 is configured to detect the rotation angular velocity of the upper rotating body 3. In this embodiment, the rotation angular velocity sensor S5 is a gyro sensor. The rotation angular velocity sensor S5 may be a resolver, a rotary encoder, or the like. The rotation angular velocity sensor S5 may detect a rotation speed. The rotation speed may be calculated from the rotation angular velocity.

[0042] The surroundings imaging device S6 is provided on the upper rotating body 3 or the cabin 10, and captures images of the surroundings of the work machine 100 to obtain surroundings image information showing the surroundings of the work machine 100. In the illustrated example, the surroundings imaging device S6 includes a front camera S6F, a left camera S6L, a right camera S6R, and a rear camera S6B.

[0043] The front camera S6F is a camera that captures images in front of the work machine 100, and is attached outside the cabin 10, such as on the roof of the cabin 10 or the side of the boom 4. The front camera S6F may also be attached inside the cabin 10, for example, on the ceiling of the cabin 10. The left camera S6L is a camera that captures images to the left of the work machine 100, and is attached to the left end of the top surface of the upper rotating body 3. The right camera S6R is a camera that captures images to the right of the work machine 100, and is attached to the right end of the top surface of the upper rotating body 3. The rear camera S6B is a camera that captures images behind the work machine 100, and is attached to the rear end of the top surface of the upper rotating body 3.

[0044] The front camera S6F, left camera S6L, right camera S6R, and rear camera S6B are all monocular wide-angle cameras equipped with imaging elements such as CCD or CMOS, and output captured images to the display device DI. In addition, the images captured by the front camera S6F, left camera S6L, right camera S6R, and rear camera S6B are captured by the controller 30.

[0045] In this embodiment, by arranging the surrounding image capturing device S6 in the above-described arrangement, it is possible to capture images of objects present around the work machine 100. Note that the surrounding image capturing device S6 may be a camera (for example, an RGBD camera or a stereo camera) that can recognize the distance to the object being captured.

[0046] The front imaging device S7 is provided on the upper rotating body 3 or the cabin 10, and captures images of the area in front of the work machine 100 to obtain front image information showing the area in front of the work machine 100. The front imaging device S7 is a camera that captures images of the area in front of the work machine 100, and is attached inside the cabin 10, such as on the ceiling of the cabin 10. The front imaging device S7 may also be attached outside the cabin 10, for example, on the roof of the cabin 10 or the side of the boom 4. The front imaging device S7 may also serve as the front camera S6F. In other words, the image captured by the front camera S6F may be obtained as front image information and as part of surrounding image information.

[0047] The positioning device S8 is configured to acquire information related to the position of the work machine 100. In this embodiment, the positioning device S8 is configured to measure the position and orientation of the work machine 100 in a reference coordinate system. Specifically, the positioning device S8 is a GNSS receiver with an integrated electronic compass, and measures the latitude, longitude, and altitude of the current position of the work machine 100, and measures the orientation of the work machine 100. The reference coordinate system according to this embodiment is, for example, the World Geodetic System. The World Geodetic System is a three-dimensional orthogonal XYZ coordinate system with its origin at the center of gravity of the Earth, its X axis pointing in the direction of the intersection of the Greenwich meridian and the equator, its Y axis pointing in the direction of 90 degrees east longitude, and its Z axis pointing in the direction of the North Pole.

[0048] The communication device T1 is configured to control communications with devices external to the work machine 100. In this embodiment, the communication device T1 is configured to control communications between the communication device T1 and devices external to the work machine 100 via a wireless communication network. The communication device T1 includes, for example, a mobile communication module compatible with mobile communication standards such as LTE (Long Term Evolution), 4G (4th Generation), and 5G (5th Generation), a satellite communication module for connecting to a satellite communication network, and the like.

[0049] Furthermore, the communication device T1 controls wireless communication between the work machine 100 and an external GNSS (Global Navigation Satellite System) surveying system, for example.

[0050] <Work machine drive control system> Figure 3 is a diagram showing an example of the configuration of a drive control system of the work machine 100 of Figure 2. In Figure 3, the mechanical power transmission system is indicated by double lines, the hydraulic oil lines are indicated by thick solid lines, the pilot lines are indicated by dashed lines, and the electric drive and control system is indicated by dotted lines.

[0051] The drive system of the work machine 100 according to this embodiment includes an engine 11, a regulator 13, a main pump 14, a pilot pump 15, and a control valve unit 17. The hydraulic drive system of the work machine 100 according to this embodiment also includes hydraulic actuators such as traveling hydraulic motors 1L, 1R, swing hydraulic motor 2A, boom cylinder 7, arm cylinder 8, and bucket cylinder 9 that hydraulically drive the lower traveling body 1, upper rotating body 3, boom 4, arm 5, and bucket 6, respectively.

[0052] The engine 11 is the main power source in the hydraulic drive system, and is mounted, for example, on the rear of the upper rotating body 3. Specifically, the engine 11 rotates at a constant speed at a preset target speed under direct or indirect control by a controller 30 (described later), and drives the main pump 14 and the pilot pump 15. The engine 11 is, for example, a diesel engine that uses diesel as fuel.

[0053] The regulator 13 controls the discharge amount of the main pump 14. For example, the regulator 13 adjusts the angle (tilt angle) of the swash plate of the main pump 14 in response to a control command from the controller 30. The regulator 13 includes, for example, regulators 13L and 13R, as described below.

[0054] The main pump 14 is mounted, for example, on the rear of the upper rotating body 3, similar to the engine 11, and supplies hydraulic oil to the control valve unit 17 through a high-pressure hydraulic line. The main pump 14 is driven by the engine 11. The main pump 14 is, for example, a variable displacement hydraulic pump, and under the control of the controller 30, the tilt angle of the swash plate is adjusted by the regulator 13, thereby adjusting the stroke length of the piston and controlling the discharge flow rate (discharge pressure).

[0055] The control valve unit 17 is a hydraulic control device that controls the hydraulic system in the work machine 100. In this embodiment, the control valve unit 17 includes control valves 171 to 176. The control valve unit 17 is configured to selectively supply hydraulic oil discharged by the main pump 14 to one or more hydraulic actuators via the control valves 171 to 176. The control valves 171 to 176 control, for example, the flow rate of hydraulic oil flowing from the main pump 14 to the hydraulic actuators and the flow rate of hydraulic oil flowing from the hydraulic actuators to a hydraulic oil tank. The hydraulic actuators include a boom cylinder 7, an arm cylinder 8, a bucket cylinder 9, traveling hydraulic motors 1L and 1R, and a swing hydraulic motor 2A. More specifically, the control valve 171 corresponds to the traveling hydraulic motor 1L, the control valve 172 corresponds to the traveling hydraulic motor 1R, and the control valve 173 corresponds to the swing hydraulic motor 2A. Furthermore, the control valve 174 corresponds to the bucket cylinder 9 , the control valve 175 corresponds to the boom cylinder 7 , and the control valve 176 corresponds to the arm cylinder 8 .

[0056] The pilot pump 15 is an example of a pilot pressure generating device, and is configured to supply hydraulic oil to hydraulic control devices via a pilot line. In this embodiment, the pilot pump 15 is a fixed displacement hydraulic pump. However, the pilot pressure generating device may be realized by the main pump 14. That is, the main pump 14 may have a function of supplying hydraulic oil to various hydraulic control devices via a pilot line, in addition to a function of supplying hydraulic oil to the control valve unit 17 via a hydraulic oil line. In this case, the pilot pump 15 may be omitted.

[0057] The operating device 26 is a device used by an operator to operate the actuators. The actuators include at least one of a hydraulic actuator and an electric actuator.

[0058] The discharge pressure sensor 28 is configured to detect the discharge pressure of the main pump 14. In this embodiment, the discharge pressure sensor 28 outputs the detected value to the controller 30.

[0059] The operation sensor 29 is configured to detect the operation content of the operator using the operation device 26. In this embodiment, the operation sensor 29 detects the operation direction and operation amount of the operation device 26 corresponding to each actuator and outputs the detected value to the controller 30. In this embodiment, the controller 30 controls the opening area of ​​the proportional valve 31 in accordance with the output of the operation sensor 29. The controller 30 then supplies the hydraulic oil discharged from the pilot pump 15 to the pilot ports of the corresponding control valves in the control valve unit 17. The pressure of the hydraulic oil supplied to each pilot port (pilot pressure) is, in principle, a pressure corresponding to the operation direction and operation amount of the operation device 26 corresponding to each hydraulic actuator. In this way, the operation device 26 is configured to supply the hydraulic oil discharged from the pilot pump 15 to the pilot ports of the corresponding control valves in the control valve unit 17.

[0060] The proportional valve 31, which functions as a control valve for machine control, is disposed in a pipe connecting the pilot pump 15 and the pilot port of the control valve in the control valve unit 17, and is configured so that the flow path area of ​​the pipe can be changed. In this embodiment, the proportional valve 31 operates in response to a control command output by the controller 30. Therefore, the controller 30 can supply the hydraulic oil discharged by the pilot pump 15 to the pilot port of the control valve in the control valve unit 17 via the proportional valve 31, regardless of the operation of the operating device 26 by the operator.

[0061] With this configuration, the controller 30 can operate the hydraulic actuator corresponding to a specific operating device 26 even when the specific operating device 26 is not being operated.

[0062] For example, the controller 30 sets a target rotation speed based on a work mode or the like that is set in advance by a predetermined operation by an operator or the like, and performs drive control to rotate the engine 11 at a constant speed.

[0063] Furthermore, for example, the controller 30 outputs a control command to the regulator 13 as necessary to change the discharge rate of the main pump 14.

[0064] Furthermore, for example, the controller 30 performs control relating to a machine guidance function that guides (provides guidance for) the manual operation of the work machine 100 by the operator via the operation device 26. Furthermore, the controller 30 performs control relating to a machine control function that automatically assists the manual operation of the work machine 100 by the operator via the operation device 26.

[0065] Note that some of the functions of the controller 30 may be realized by another controller (control device). That is, the functions of the controller 30 may be realized in a distributed manner by a plurality of controllers. For example, the machine guidance function and the machine control function may be realized by a dedicated controller (control device).

[0066] <Remote control system functional configuration> The functional configuration of the remote operation system SYS according to the first embodiment will be described with reference to Fig. 4. Fig. 4 is a functional block diagram showing an example configuration of the remote operation system SYS according to the first embodiment. Fig. 4 shows the block configuration of each of the work machine 100, remote operation room RC, and server SVR included in the remote operation system SYS. The hardware configuration of the work machine 100 has been described above, so description thereof will be omitted.

[0067] <Functional configuration of the work machine> The following describes each functional block within the controller 30 of the work machine 100. The controller 30 comprises an acquisition unit 301, a state identification unit 302, an imaging control unit 303, a transmission control unit 304, a reception control unit 305, and a drive control unit 306.

[0068] Note that each functional block in the controller 30 is conceptual and does not necessarily have to be physically configured as shown in the figure. All or part of each functional block can be functionally or physically distributed or integrated in any unit. All or any part of the processing functions performed by each functional block are realized by a program executed by a CPU. Alternatively, each functional block may be realized as hardware using wired logic.

[0069] The acquisition unit 301 acquires signals from various detection devices provided on the work machine 100. For example, the acquisition unit 301 acquires the detection results of the boom angle sensor S1, the arm angle sensor S2, and the bucket angle sensor S3. The acquisition unit 301 also acquires measurement results such as the position and orientation of the work machine 100 from the positioning device S8.

[0070] The state identification unit 302 is configured to identify the state of the work machine 100 based on the signal acquired by the acquisition unit 301. In this embodiment, the state of the work machine 100 includes the position and orientation of the work machine 100, and the states of the attachments of the work machine 100 (for example, the positions of the boom 4, arm 5, and bucket 6). The position of the work machine 100 is, for example, the position of the work machine 100 in a reference coordinate system (for example, the latitude, longitude, and altitude of the reference point of the work machine 100). The state identification unit 302 identifies the position and orientation of the work machine 100 based on the output of the positioning device S8.

[0071] The state of the attachment (e.g., the positions of the boom 4, arm 5, and bucket 6) can be determined from the detection results of the angle sensors (boom angle sensor S1, arm angle sensor S2, and bucket angle sensor S3) and the respective sizes of the boom 4, arm 5, and bucket 6.

[0072] The imaging control unit 303 controls imaging by the surrounding imaging device S6 and the front imaging device S7. For example, the imaging control unit 303 instructs the surrounding imaging device S6 to capture an image and acquires surrounding image information output by the surrounding imaging device S6. The imaging control unit 303 also instructs the front imaging device S7 to capture an image and acquires front image information output by the front imaging device S7. Note that the surrounding imaging device S6 and the front imaging device S7 may autonomously capture images at predetermined time intervals or the like, without being controlled by the imaging control unit 303.

[0073] The imaging control unit 303 may acquire surrounding image information that represents a specific range around the work machine 100. The imaging control unit 303 may cut out the specific range from the surrounding image information acquired by the surrounding image capture device S6. For example, the imaging control unit 303 may cut out the specific range by trimming processing or cropping processing.

[0074] The specific range may be a predetermined range. The specific range may be a range specified by an operator OP who operates the work machine 100. The specific range may be a range that includes a specific object. The specific object may be another work machine 100 that exists in the same work site.

[0075] For example, the imaging control unit 303 of the controller 30 in the work machine 100-1 may acquire surrounding image information of a range including the work machine 100-2. Similarly, the imaging control unit 303 of the controller 30 in the work machine 100-1 may acquire surrounding image information of a range including the work machine 100-2.

[0076] The specific range may be a range that includes a specific position at the work site. The specific position may be, for example, an entrance / exit to the work site. For example, at a work site where the work machine 100 is remotely operated, entry to the work site where the work machine 100 is working may be prohibited. By displaying surrounding image information of a range that includes the entrance / exit to the work site on the display device D1 of the remote control room RC, the operator OP in the remote control room RC can know that someone is entering the work site.

[0077] When the specific range is an area including a specific object or a specific position, the imaging control unit 303 may move the position of the specific range in the surrounding image information so that the object is included in the specific range. In this case, the imaging control unit 303 may track the position of the object by recognizing the object from the surrounding image information acquired by the surrounding imaging device S6.

[0078] FIG. 5 is a diagram showing a specific range according to the first embodiment. In the illustrated example, the work machine 100-2 cuts out a specific range including the work machine 100-1. The surroundings image capturing device S6 mounted on the work machine 100-2 captures an image of a 360-degree range V0 around the work machine 100-2. In the illustrated example, the 360 ​​degrees around the work machine 100-2 is shown as the imaging range V0 of the surroundings image capturing device S6, but the imaging range V0 of the surroundings image capturing device S6 does not have to be 360 ​​degrees. For example, the imaging range V0 of the surroundings image capturing device S6 may be approximately 270 degrees, including the left, right, and rear of the work machine 100-2.

[0079] In the illustrated example, the imaging control unit 303 of the work machine 100-2 cuts out an image included in a specific range V1 in the direction in which the work machine 100-1 is located from the imaging range V0 of the surrounding image capture device S6. The cutting angle θ may be determined in advance or may be specified by the operator OP.

[0080] The transmission control unit 304 performs control to transmit various information to the remote control room RC or the server SVR via the communication device T1. For example, the transmission control unit 304 performs control to transmit to the remote control room RC, front image information captured by the front image capturing device S7, position information indicating the position and orientation of the work machine 100, and status information indicating the status of the attachment. The transmission control unit 304 also performs control to transmit surrounding image information captured by the surrounding image capturing device S6 to the server SVR. The surrounding image information captured by the surrounding image capturing device S6 may be an image representing the imaging range V0 of the surrounding image capturing device S6, or may be an image representing a specific range V1.

[0081] Transmission by the transmission control unit 304 is performed at predetermined time intervals. The predetermined time may be any time, but should be a time interval that allows recognition of changes in the situation due to work by the work machine 100. For example, the transmission control unit 304 may transmit the above information at one-second intervals.

[0082] The reception control unit 305 performs control to receive various information from the remote control room RC via the communication device T1. For example, the reception control unit 305 receives an operation signal for controlling the operation of the work machine 100 from the remote control room RC.

[0083] The drive control section 306 is configured to drive the actuators mounted on the work machine 100. In this embodiment, the drive control section 306 generates and outputs actuation signals for each of the multiple solenoid valves included in the proportional valve 31 based on an operation signal transmitted from the remote control room RC.

[0084] Each solenoid valve that receives the actuation signal increases or decreases the pilot pressure acting on the pilot port of the corresponding control valve in the control valve unit 17. As a result, the hydraulic actuator corresponding to each control valve operates at a speed that corresponds to the stroke amount of the control valve.

[0085] <Server functional configuration> The following describes each functional block in the server SVR: The server SVR comprises a communication control unit 501, a storage control unit 502, and an image processing unit 503.

[0086] Note that each functional block within the server SVR is conceptual and does not necessarily have to be physically configured as shown in the figure. All or part of each functional block can be functionally or physically distributed or integrated in any unit. All or any part of the processing functions performed by each functional block are realized by a program executed by a CPU. Alternatively, each functional block may be realized as hardware using wired logic.

[0087] The communication control unit 501 performs control to transmit and receive various information between the communication device T1 attached to the work machine 100 and the communication device T2 installed in the remote control room RC. For example, the communication control unit 501 performs control to receive surrounding image information transmitted from the work machine 100 between the communication device T1. Furthermore, the communication control unit 501 performs control to transmit an operation signal received from the remote control room RC to the work machine 100 between the communication device T1. For example, the communication control unit 501 performs control to receive an operation signal from the remote control room RC between the communication device T2. Furthermore, the communication control unit 501 performs control to transmit surrounding image information received from the work machine 100 to the remote control room RC between the communication device T2.

[0088] The communication control unit 501 transmits surrounding image information captured by a work machine 100 different from the work machine 100 being operated from the remote control room RC. For example, the communication control unit 501 transmits surrounding image information captured by the work machine 100-2 to the remote control room RC-1 that operates the work machine 100-1. The communication control unit 501 also transmits surrounding image information captured by the work machine 100-1 to the remote control room RC-2 that operates the work machine 100-2.

[0089] The storage control unit 502 stores the surrounding image information received by the communication control unit 501. The storage control unit 502 stores the surrounding image information so that the work machine 100 that captured the surrounding image information can be identified. For example, the storage control unit 502 associates the surrounding image information with the identification information of the work machine 100 that received the surrounding image information and stores the surrounding image information.

[0090] The memory control unit 502 may read out surrounding image information in response to a request from the remote control room RC. The memory control unit 502 reads out surrounding image information captured by a work machine 100 different from the work machine 100 being operated from the remote control room RC. For example, the memory control unit 502 reads out surrounding image information captured by the work machine 100-2 in response to a request from the remote control room RC-1 that operates the work machine 100-1. The memory control unit 502 also reads out surrounding image information captured by the work machine 100-1 in response to a request from the remote control room RC-2 that operates the work machine 100-2.

[0091] The image processing unit 503 generates surrounding image information that represents a specific range around the work machine 100, based on the surrounding image information received by the communication control unit 501 or the surrounding image information read by the storage control unit 502. The image processing unit 503 may cut out the specific range from the surrounding image information. For example, the image processing unit 503 may cut out the specific range by trimming processing or cropping processing.

[0092] When the communication control unit 501 receives surrounding image information representing a specific range (for example, when surrounding image information representing a specific range is acquired by the imaging control unit 303 of the controller 30), the image processing unit 503 simply uses the surrounding image information as it is as surrounding image information representing the specific range.

[0093] <Remote controller functional configuration> The following describes each functional block in the remote controller 40 in the remote control room RC. The remote controller 40 includes a range designation unit 401, a reception control unit 402, a display control unit 403, a signal generation unit 404, and a transmission control unit 405.

[0094] Note that each functional block in the remote controller 40 is conceptual and does not necessarily have to be physically configured as shown in the figure. All or part of each functional block can be functionally or physically distributed or integrated in any unit. All or any part of the processing functions performed by each functional block are realized by a program executed by a CPU. Alternatively, each functional block may be realized as hardware using wired logic.

[0095] In this embodiment, a case will be described in which the remote controller 40 in the remote control room RC is a control unit for the display device D1. In other words, the configuration in the remote control room RC functions as a display device for supporting the remote operation of the work machine 100.

[0096] The range designation unit 401 accepts designation of a specific range to be displayed on the display device D1 from within the image of the surroundings of the work machine 100. The range designation unit 401 may accept designation of the specific range in response to an operation by the operator OP. For example, the operator OP may designate the specific range by operating the display device D1 or the operation device 42. The designation of the specific range may be designation of a specific object or a specific position to be included in the specific range.

[0097] When the range designation unit 401 receives the designation of a specific range, it transmits a signal designating the specific range to either the server SVR or the work machine 100. When the signal designating the specific range is transmitted to the work machine 100, the specific range is set in the imaging control unit 303. This enables the imaging control unit 303 to acquire surrounding image information representing the specific range from the surrounding image information captured by the surrounding image capture device S6. Furthermore, when the signal designating the specific range is transmitted to the server SVR, the specific range is set in the image processing unit 503. This enables the image processing unit 503 to generate surrounding image information representing the specific range from the surrounding image information received by the communication control unit 501.

[0098] The reception control unit 402 performs control to receive various information from the work machine 100 or server SVR via the communication device T2. For example, the reception control unit 402 performs control to receive, from the work machine 100, forward image information captured by the forward image capture device S7, position information indicating the position and orientation of the work machine 100, and status information indicating the status of the attachment. The reception control unit 402 also performs control to receive, from the server SVR, surrounding image information captured by the surrounding image capture device S6.

[0099] The display control unit 403 generates a display image including the forward image information and surrounding image information received by the reception control unit 402. The display control unit 403 generates the display image using the forward image information captured by the work machine 100 operated from the remote control room RC and the surrounding image information captured by a work machine 100 not operated from the remote control room RC. The display control unit 403 outputs the generated display image to the display device D1.

[0100] For example, the display control unit 403 of the remote controller 40 in the remote operation room RC-1 generates a display image including forward image information captured by the work machine 100-1 and surrounding image information captured by the work machine 100-2. Furthermore, the display control unit 403 of the remote controller 40 in the remote operation room RC-2 generates a display image including forward image information captured by the work machine 100-2 and surrounding image information captured by the work machine 100-1.

[0101] The signal generation unit 404 generates an operation signal for controlling the operation of the work machine 100 in accordance with the operation received by the operation sensor 43. For example, the signal generation unit 404 of the remote controller 40 in the remote operation room RC-1 generates an operation signal for controlling the operation of the work machine 100-1. Also, the signal generation unit 404 of the remote controller 40 in the remote operation room RC-2 generates an operation signal for controlling the operation of the work machine 100-2.

[0102] The transmission control unit 405 transmits the operation signal generated by the signal generation unit 404 to the work machine 100. For example, the transmission control unit 405 of the remote controller 40 in the remote operation room RC-1 transmits the operation signal to the work machine 100-1. Also, the transmission control unit 405 of the remote controller 40 in the remote operation room RC-2 transmits the operation signal to the work machine 100-2.

[0103] <Location of the remote control room> The layout of the remote control room RC will be described with reference to Fig. 6. Fig. 6 is a diagram showing an example of the layout of the remote control room according to the first embodiment. In the remote control room RC, a plurality of operating devices 42 are provided with respect to the operator's seat DS. Here, the remote control room RC will be described as operating a work machine 100.

[0104] In this embodiment, the display device D1 is a multi-display consisting of nine monitors arranged in three rows and three columns, as shown in Fig. 5. Specifically, the display device D1 includes a center monitor D1a, an upper monitor D1b, a lower monitor D1c, a left monitor D1d, a right monitor D1e, an upper left monitor D1f, an upper right monitor D1g, a lower left monitor D1h, and a lower right monitor D1i.

[0105] The display device D1 displays forward image information captured by the forward image capture device S7 of the work machine 100-1 and surrounding image information captured by the surrounding image capture device S6 of the work machine 100-2. The display device D1 divides a display area made up of nine monitors into a plurality of partial areas, and displays forward image information in a first partial area of ​​the plurality of partial areas, and displays surrounding image information in a second partial area. The partial areas may correspond to one or more of the nine monitors, or may be areas obtained by arbitrarily dividing the entire display area without corresponding to a specific monitor. The display device D1 may further display any image other than the forward image information and partial image information.

[0106] <Image display process flow> The image display process executed by the remote control system SYS will be described. Fig. 7 is a sequence diagram showing an example of the image display process according to the first embodiment. The illustrated example is an example of the image display process in which a display image is displayed on a display device D1 installed in the remote control room RC-1.

[0107] In step S101, the front image capturing device S7 of the work machine 100-1 captures an image of the area in front of the work machine 100-1 in accordance with the control of the imaging control unit 303. The front image capturing device S7 outputs the captured front image information to the controller 30.

[0108] In step S102, the imaging control unit 303 of the work machine 100-1 acquires forward image information from the forward imaging device S7. The imaging control unit 303 sends the forward image information to the transmission control unit 304. The transmission control unit 304 transmits the forward image information to the remote control room RC-1 via the communication device T1.

[0109] In step S103, the reception control unit 402 of the remote control room RC-1 receives the forward image information from the work machine 100-1 via the communication device T2. The reception control unit 402 sends the received forward image information to the display control unit 403. The display control unit 403 generates a display image including the forward image information and displays it on the display device D1.

[0110] In step S104, the surroundings imaging device S6 of the work machine 100-2 images the surroundings of the work machine 100-2 in accordance with the control of the imaging control section 303. The surroundings imaging device S6 outputs surrounding image information obtained by imaging to the controller 30.

[0111] In step S105, the imaging control unit 303 of the work machine 100-2 acquires surrounding image information from the surrounding image capture device S6. The imaging control unit 303 sends the surrounding image information to the transmission control unit 304. The transmission control unit 304 transmits the surrounding image information to the server SVR via the communication device T1.

[0112] In step S106, the communication control unit 501 of the server SVR receives the surrounding image information from the work machine 100-2. The communication control unit 501 sends the received surrounding image information to the storage control unit 502. The storage control unit 502 stores the surrounding image information.

[0113] In step S107, the range designation unit 401 in the remote control room RC-1 accepts designation of a specific range in response to an operation by the operator OP. The range designation unit 401 transmits a signal instructing the specific range to the server SVR.

[0114] In step S108, the communication control unit 501 of the server SVR receives a signal instructing a specific range from the remote control room RC- 1. The communication control unit 501 sets the specific range in the image processing unit 503.

[0115] In step S109, the storage control unit 502 of the server SVR reads out surrounding image information captured by the work machine 100-2. The storage control unit 502 sends the read out surrounding image information to the image processing unit 503. The image processing unit 503 cuts out a specific range from the surrounding image information. The image processing unit 503 sends the surrounding image information representing the specific range to the communication control unit 501.

[0116] In step S110, the communication control unit 501 of the server SVR receives surrounding image information representing a specific range from the image processing unit 503. The communication control unit 501 transmits the surrounding image information representing the specific range to the remote control room RC-1.

[0117] In step S111, the reception control unit 402 in the remote control room RC-1 receives surrounding image information from the server SVR via the communication device T2. The reception control unit 402 sends the received surrounding image information to the display control unit 403. The display control unit 403 generates a display image including the forward image information received in step S103 and the surrounding image information received in step S111. The display control unit 403 displays the display image on the display device D1.

[0118] (Effects of the first embodiment) When remotely operating a work machine 100, it is important to appropriately obtain information about the environment surrounding the work machine 100. Conventionally, in remote operation, images captured by an imaging device mounted on the work machine 100 are displayed, and the operator OP operates the work machine 100 while viewing the images. However, the imaging device mounted on the work machine 100 is often a monocular camera, and images captured by a monocular camera lack a sense of depth, making it difficult to improve work efficiency.

[0119] On the other hand, by displaying a bird's-eye view image of the work machine 100, it is possible to compensate for the sense of depth that is difficult to grasp with a monocular camera, or for blind spots in the imaging device mounted on the work machine 100. Bird's-eye views can be captured using, for example, fixed cameras, drones, camera vehicles, or mobile robots, but there are high hurdles to implementation, such as the effort required for installation, the need to become proficient in operating them, and the need for administrative permission.

[0120] The remote operation system SYS displays an image of the area in front of the work machine 100-1 received from the work machine 100-1, and an image of the area around the work machine 100-2 received from the work machine 100-2, on a display device D1 provided near the operator's seat DS that operates the work machine 100-1. The operator OP operating the work machine 100-1 can easily grasp the situation in front of and around the work machine 100-1 by referring to the image displayed on the display device D1. As a result, safety and work efficiency are improved when remotely operating the work machine 100-1.

[0121] Conventionally, the work machine 100 is equipped with an imaging device that captures images of the surroundings of the work machine 100 in order to grasp the conditions around the work machine 100. Therefore, there is no need to introduce a new imaging device for capturing images of the surroundings of the work machine 100. Furthermore, since the operator of the work machine 100 is accustomed to using the imaging device that has conventionally been installed, there is no need to learn how to operate a new imaging device.

[0122] The remote operation system SYS displays an image representing a specific range of the surroundings of the work machine 100-2 on the display device D1. The image of the surroundings of the work machine 100-2 may include an image of a range that is not important for the operation of the work machine 100-1. By displaying an image of a specific range of the surrounding image, it is possible to provide the operator OP of the work machine 100-1 with appropriate information about the environment surrounding the work machine 100-1.

[0123] The image representing the specific range may be cut out from the image of the surroundings captured by the surrounding image capture device S6. By cutting out the specific range from the image of the surroundings, the image captured by the surrounding image capture device S6 can be used for two purposes: front image information and surrounding image information, which is efficient.

[0124] The specific range may be determined in advance. By continuously displaying an image of the predetermined range, an image of the surroundings of the work machine 100 can be displayed stably.

[0125] The specific range may be specified by the operator OP in the remote control room RC-1. By specifying the specific range, the operator OP can display an image of the required range.

[0126] The specific range may include a specific object. By continuously displaying an image of the range including the specific object, the operator OP can operate the work machine 100 while understanding the status of the specific object that exists around the work machine 100.

[0127] The specific object may be the work machine 100-1. By displaying an image of the work machine 100-1 captured by another work machine 100-2, the operator OP operating the work machine 100-1 can easily understand the situation around the work machine 100-1.

[0128] (Second embodiment) In the first embodiment, a work machine 100 was described that captures images of the surroundings of the work machine 100 using a surrounding image capture device S6 that includes multiple cameras with different image capture directions. In the second embodiment, a configuration will be described in which the work machine 100 is further equipped with a gimbal camera having a gimbal mechanism in addition to the surrounding image capture device S6, and images of the surroundings of the work machine 100 are captured by the gimbal camera.

[0129] A gimbal mechanism is a mechanism that combines two gimbals that rotate an object around a single axis, with the axes perpendicular to each other. By controlling the rotation of the two gimbals, a gimbal camera can adjust the camera's angle of view in any direction.

[0130] Figure 8 is a side view of a work machine 100 according to the second embodiment. The work machine 100 shown in Figure 8 differs from the work machine 100 shown in Figure 2 in that it is equipped with a surroundings image capture device S9. In the work machine 100 shown in Figure 8, the surroundings image capture device S9 is configured by a gimbal camera.

[0131] In the illustrated example, the surroundings imaging device S9 is provided on the roof of the cabin 10. The surroundings imaging device S9 may be installed in a high position, for example, using a support or the like, so that the body of the work machine 100 does not enter the angle of view. The surroundings imaging device S9 may also be provided, for example, on the top surface of the upper rotating body 3 or on the side of the boom 4.

[0132] The angle of view of the surrounding imaging device S9 is adjusted by the imaging control unit 303. The image captured by the surrounding imaging device S9 is captured by the controller 30. The image captured by the controller 30 is acquired by the imaging control unit 303 as surrounding image information representing a specific area. In other words, the imaging control unit 303 according to this embodiment acquires a surrounding image area representing a specific range by adjusting the angle of view of the surrounding imaging device S9.

[0133] (Application example 1) By using an imaging device with an adjustable angle of view, the work machine 100 can perform work while acquiring surrounding image information that includes other work machines 100. Here, an example will be described in which a work machine 100-2 equipped with a gimbal camera performs work while acquiring surrounding image information that includes work machine 100-1.

[0134] Figure 9 is a diagram for explaining an example of a state in which a work machine equipped with an imaging device whose angle of view cannot be adjusted is working. Figure 9(A) is a diagram showing the state before the work machine 100-2 starts working. Figure 9(B) is a diagram showing the state in which the work machine 100-2 is working.

[0135] In the illustrated example, the work machine 100-2 is equipped with an imaging device with a fixed angle of view V1 behind the work machine 100-2. As shown in Fig. 9(A), when the work machine 100-1 is present behind the work machine 100-2, the imaging device mounted on the work machine 100-2 can acquire surrounding image information including the work machine 100-1. On the other hand, as shown in Fig. 9(B), when the work machine 100-2 turns while working, the work machine 100-1 moves out of the angle of view V1 of the imaging device mounted on the work machine 100-2. As a result, the imaging device mounted on the work machine 100-2 can no longer acquire surrounding image information including the work machine 100-1.

[0136] Figure 10 is a diagram for explaining an example of a state in which a work machine equipped with an imaging device with an adjustable angle of view is working. Figure 10(A) is a diagram showing the state before the work machine 100-2 starts working. Figure 10(B) is a diagram showing the state in which the work machine 100-2 is working.

[0137] In the illustrated example, the work machine 100-2 is equipped with a gimbal camera that captures images of the surroundings of the work machine 100-2. As shown in Fig. 10(A), when the work machine 100-1 is present behind the work machine 100-2, surrounding image information including the work machine 100-1 can be acquired by adjusting the angle of view V1 of the gimbal camera to be behind the work machine 100-2. On the other hand, as shown in Fig. 10(B), when the work machine 100-2 turns in conjunction with work, the work machine 100-1 moves out of the angle of view V1 of the gimbal camera mounted on the work machine 100-2. At this time, surrounding image information including the work machine 100-1 can be acquired by adjusting the angle of view V1 of the gimbal camera in the direction opposite to the turning direction.

[0138] (Application example 2) The remote control room RC may be capable of operating two work machines 100. Here, an example will be described in which work machine 100-2 can be operated from remote control room RC-1 which operates work machine 100-1.

[0139] 11 is a diagram illustrating an example of control for operating two work machines from one remote control room. In the illustrated example, work machine 100-1 is performing work at position P1-1, and work machine 100-2 is stopped at position P2-1. At this time, the angle of view V1 of the gimbal camera mounted on work machine 100-2 is adjusted to a range that includes work machine 100-1. Therefore, work machine 100-2 can acquire surrounding image information that represents the range that includes work machine 100-1.

[0140] Thereafter, suppose that work machine 100-1 is operated from remote control room RC-1 and caused to travel to position P1-2 along route R1. Work machine 100-1 moves out of the angle of view V1 of the gimbal camera mounted on work machine 100-2. At this time, work machine 100-2 is operated from remote control room RC-1 and caused to travel to position P2-2 along route R2, and the angle of view V1 of the gimbal camera is adjusted to a range that includes work machine 100-1. In this way, by operating two work machines from one remote control room, it is possible to obtain surrounding image information for a range that includes work machine 100-1, even when work machine 100-1 is traveling.

[0141] (Effects of the second embodiment) In this embodiment, the image representing the specific range is captured by adjusting the angle of view of the surrounding image capture device S6. By capturing the image by adjusting the angle of view, it is possible to suppress deterioration of the image captured by the surrounding image capture device S6, and it is possible to obtain the image representing the specific range with high image quality.

[0142] The operator OP operating the work machine 100-1 may also operate the work machine 100-2. When the operator OP operates the work machine 100-1 to travel, it is possible that the work machine 100-1 may move out of the imaging range of the imaging device mounted on the work machine 100-2. In this case, if the operator OP operating the work machine 100-1 can also operate the work machine 100-2, he or she can move the work machine 100-2 to a position where the work machine 100-1 can always be imaged from the work machine 100-2. Furthermore, because the operator OP of the work machine 100-1 is proficient in operating the work machine 100, there is no need for the operator OP to learn how to operate the work machine 100-2. Therefore, the operator OP can continue to grasp the situation around the work machine 100-1 even when operating the work machine 100-1 to travel.

[0143] (Third embodiment) In the first embodiment, a work machine 100 was described that captures images of the surroundings of the work machine 100 using a surrounding image capture device S6 that includes multiple cameras with different image capture directions. In the third embodiment, a configuration will be described in which the work machine 100 is further equipped with a 360-degree camera that is capable of capturing images in 360 degrees, separate from the surrounding image capture device S6, and images of the surroundings of the work machine 100 are captured by the 360-degree camera.

[0144] A 360-degree camera is a camera equipped with multiple wide-angle lenses that allows it to capture images in all directions simultaneously. 360-degree cameras are also called spherical cameras. Images captured by a 360-degree camera capture images in all directions, so any range can be cut out.

[0145] The work machine 100 according to this embodiment is configured in the same manner as the work machine 100 shown in Figure 8. However, in this embodiment, the surroundings image capture device S9 is configured by a 360-degree camera.

[0146] The image captured by the 360-degree camera is captured by the controller 30. The image captured by the controller 30 is acquired by the imaging control unit 303. The imaging control unit 303 may cut out a specific range from the image captured by the 360-degree camera. Alternatively, the image captured by the controller 30 may be transmitted to the server SVR by the transmission control unit 304, and the image processing unit 503 of the server SVR may cut out a specific range from the image captured by the 360-degree camera.

[0147] (Fourth embodiment) In the above-described embodiments, a configuration has been described in which forward image information captured by the work machine 100-1 and surrounding image information captured by the work machine 100-2 are displayed on a display device D1 installed in the remote control room RC-1. In the present embodiment, a configuration will be described in which forward image information captured by the work machine 100-1 and surrounding image information captured by the work machine 100-2 are displayed on a display device DI installed in the work machine 100-1.

[0148] In this embodiment, at least the work machine 100-1 is operated by an operator OP seated in a driver's seat inside the cabin 10. The display device DI is installed in a position visible to the operator OP inside the cabin 10. The display device D1 may be provided with an input device such as a touch panel.

[0149] In this embodiment, the remote operation system SYS does not necessarily include at least the remote operation room RC-1. The controller 30 in the work machine 100-1 further includes a range designation unit 401 and a display control unit 403 in addition to the acquisition unit 301, state identification unit 302, imaging control unit 303, transmission control unit 304, reception control unit 305, and drive control unit 306 shown in FIG.

[0150] <Image display process flow> The image display processing executed by the remote operation system SYS will be described. Fig. 12 is a sequence diagram showing an example of the image display processing according to the fourth embodiment. The illustrated example is an example of the image display processing in which a display image is displayed on the display device DI installed in the cabin 10 of the work machine 100-1.

[0151] In step S201, the front image capturing device S7 of the work machine 100-1 captures an image of the area in front of the work machine 100-1 in accordance with the control of the imaging control section 303. The front image capturing device S7 outputs the captured front image information to the controller 30.

[0152] In step S202, the imaging control unit 303 of the work machine 100-2 acquires forward image information from the forward imaging device S7. The imaging control unit 303 sends the acquired forward image information to the display control unit 403. The display control unit 403 generates a display image including the forward image information and displays it on the display device DI.

[0153] In step S203, the surroundings imaging device S6 of the work machine 100-2 images the surroundings of the work machine 100-2 in accordance with the control of the imaging control section 303. The surroundings imaging device S6 outputs surrounding image information obtained by imaging to the controller 30.

[0154] In step S204, the imaging control unit 303 of the work machine 100-2 acquires surrounding image information from the surrounding image capture device S6. The imaging control unit 303 sends the surrounding image information to the transmission control unit 304. The transmission control unit 304 transmits the surrounding image information to the server SVR via the communication device T1.

[0155] In step S205, the communication control unit 501 of the server SVR receives the surrounding image information from the work machine 100-2. The communication control unit 501 sends the received surrounding image information to the storage control unit 502. The storage control unit 502 stores the surrounding image information.

[0156] In step S206, the range designation unit 401 of the work machine 100-1 accepts the designation of a specific range in response to an operation by the operator OP. The range designation unit 401 transmits a signal instructing the specific range to the server SVR.

[0157] In step S207, the communication control unit 501 of the server SVR receives a signal indicating a specific range from the work machine 100-1. The communication control unit 501 sets the specific range in the image processing unit 503.

[0158] In step S208, the storage control unit 502 of the server SVR reads out the surrounding image information captured by the work machine 100-2. The storage control unit 502 sends the read out surrounding image information to the image processing unit 503. The image processing unit 503 cuts out a specific range from the surrounding image information. The image processing unit 503 sends the surrounding image information representing the specific range to the communication control unit 501.

[0159] In step S209, the communication control unit 501 of the server SVR receives surrounding image information representing a specific range from the image processing unit 503. The communication control unit 501 transmits the surrounding image information representing the specific range to the work machine 100-1.

[0160] In step S210, the reception control unit 305 of the work machine 100-1 receives surrounding image information from the server SVR via the communication device T1. The reception control unit 305 sends the received surrounding image information to the display control unit 403. The display control unit 403 generates a display image including the forward image information acquired in step S201 and the surrounding image information received in step S210. The controller 30 displays the display image on the display device DI.

[0161] (Effects of the fourth embodiment) The work machine 100-1 displays an image of the front of the work machine 100-1 acquired by the front imaging device S7 and an image of the surroundings of the work machine 100-2 received from the work machine 100-2 on a display device DI provided near the driver's seat of the work machine 100-1. By referring to the image displayed on the display device DI, the operator OP operating the work machine 100-1 can easily grasp the situation in front of and around the work machine 100-1. As a result, safety and work efficiency are improved when operating the work machine 100-1 on site.

[0162] In the above-described embodiment and modified examples, a shovel is used as an example of a work machine. However, the configurations shown in the embodiment and modified examples are not limited to being applied to a shovel as a work machine, and may be applied to a crane, a forklift, etc.

[0163] While an embodiment showing an example of a remote control system for a work machine has been described above, the present disclosure is not limited to the above embodiment. Various changes, modifications, substitutions, additions, deletions, and combinations are possible within the scope of the claims. Naturally, these also fall within the technical scope of the present disclosure. [Explanation of symbols]

[0164] 100 Work Machinery 1 Undercarriage 2. Swivel mechanism 3 Upper rotating body 4. Boom 5 Arm 6 buckets S6 Surroundings imaging device S7 Front camera S8 Positioning Device S9 Surrounding Imager T1 communications equipment DI display device 30 Controllers 301 Acquisition Department 302 Status Identification Unit 303 Imaging control unit 304 Transmission control section 305 Reception control section 306 Drive control unit 31 Proportional valve RC remote control room T2 communications equipment D1 display device 40 Remote Controller 401 Range Specification 402 Reception control section 403 Display control unit 404 Signal Generation Unit 405 Transmission control section

Claims

1. a first work machine; a second work machine; a receiving device that receives an image transmitted by the first work machine or the second work machine; a display device provided in the vicinity of an operator's seat for operating the first work machine; An operating system for a work machine comprising: The first work machine comprises: a first undercarriage; and a first upper rotating body rotatably mounted on the first lower traveling body; a first imaging device provided on the first upper rotating body for capturing an image of a front of the first work machine; a first transmitting device that transmits an image captured by the first imaging device; Equipped with The second work machine comprises: a second undercarriage; and a second upper rotating body rotatably mounted on the second lower traveling body; a second imaging device provided on the second upper rotating body and configured to capture an image of the surroundings of the second work machine; a second transmitting device that transmits an image captured by the second imaging device; Equipped with the display device displays the image of the front area captured by the first work machine and the image of the surrounding area captured by the second work machine. Work machine operating systems.

2. the display device displays an image representing a specific range captured by the second work machine.

2. The operating system for a work machine according to claim 1.

3. the image representing the specific range is an image cut out from an image captured by the second imaging device, or an image captured by adjusting the angle of view of the second imaging device.

3. The operating system for a work machine according to claim 2.

4. The specific range includes a predetermined range or a range specified by an operator of the operation system.

3. The operating system for a work machine according to claim 2.

5. The specific range includes a specific object or a specific position.

3. The operating system for a work machine according to claim 2.

6. the specific object is the first work machine; 6. The operating system for a work machine according to claim 5.

7. the operator seat further operates the second work machine; 2. The operating system for a work machine according to claim 1.

8. A work machine, a lower running body; an upper rotating body rotatably mounted on the lower traveling body; an operator's cab provided on the upper rotating body; an imaging device provided on the upper rotating body for capturing an image of a front of the work machine; a receiving device that receives an image of the surroundings of the other work machine captured by an imaging device provided in the other work machine; a display device provided in the cab that displays the forward image captured by the work machine and the surrounding image captured by the other work machine; A work machine comprising:

Images