Remote control system for industrial machinery

Abstract

This makes it easier to judge distances and reduces the computational load. [Solution] A remote control system for a work machine according to one embodiment includes a work machine that includes a projection device that projects visible light onto the ground showing information that can identify the direction away from the device, an imaging device that images an area including the ground onto which the visible light is projected, and a first communication device that transmits the captured image, a second communication device that receives the image, and a display device that displays the image received by the second communication device.

Description

Technical Field

[0001] The present invention relates to a remote control system for a working machine.

Background Art

[0002] In recent years, technologies for remotely operating a working machine have been proposed. When an operator remotely operates a working machine while referring to an image captured by an imaging device, it becomes difficult to grasp the distance between the work target and the working machine. For this reason, a technique has been proposed in which the information on the position of the work target obtained by a distance detection device that obtains the information on the distance to the work target, and the information on the posture of the working machine are used to synthesize and display an image of a part corresponding to the working tool on the image of the work target (see Patent Document 1).

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In the technique described in Patent Document 1, it is only possible to recognize the position of the work target corresponding to the working tool, and since the displayed image is two-dimensional, it is difficult to grasp the sense of distance in the direction away from the working machine, and there is a problem that the computational load for specifying the position of the work target corresponding to the working tool increases.

[0005] In view of the above, it is possible to easily grasp the sense of distance in the direction away from the working machine by the operator and to reduce the computational load.

Means for Solving the Problems

[0006] A remote control system for a work machine according to one aspect of the present invention includes a work machine comprising: a projection device that projects visible light onto the ground showing information that can identify the direction away from the device; an imaging device that images an area including the ground onto which the visible light is projected; and a first communication device that transmits the captured image; a second communication device that receives the image; and a display device that displays the image received by the second communication device. [Effects of the Invention]

[0007] According to one aspect of the present invention, it is possible to facilitate the operator's perception of distance in the direction away from the work machine and to reduce the computational load. [Brief explanation of the drawing]

[0008] [Figure 1] This is a schematic diagram showing an example of a remote control system according to the first embodiment. [Figure 2] This diagram schematically shows an example of the configuration of a work machine according to the first embodiment. [Figure 3] This is a functional block diagram showing an example configuration of a remote control system according to the first embodiment. [Figure 4] This figure shows an example of the layout of a remote control room according to the first embodiment. [Figure 5] This is an explanatory diagram showing the situation in which the projection device of the work machine according to the first embodiment is projecting visible light onto the ground. [Figure 6] This figure shows an example of a screen displayed on the central monitor by the display control unit according to the first embodiment. [Figure 7] This figure shows an example of a screen displayed on the central monitor by the display control unit according to the first embodiment. [Figure 8] This is an explanatory diagram showing how to change the projection direction of the projection device when the work machine according to the second embodiment is in contact with an inclined surface. [Figure 9] This is an explanatory diagram illustrating an example of an arc projected by a projection device provided on a work machine according to the third embodiment. [Figure 10]This is an explanatory diagram illustrating an example of an arc projected by a projection device installed on a work machine when a person located in front of the work machine according to the fourth embodiment is detected. [Figure 11] This is a flowchart showing the procedure for switching the information to be projected by the controller according to the fourth embodiment. [Modes for carrying out the invention]

[0009] Embodiments of this disclosure will be described below with reference to the drawings. The embodiments described below are illustrative and do not limit the invention. Not all features and combinations thereof in the embodiments of this disclosure are necessarily essential to the invention. In each drawing, the same or corresponding components are denoted by the same or corresponding reference numerals, and redundant descriptions may be omitted.

[0010] The working machine 100 according to the embodiment of this disclosure is a shovel. The working machine 100 may be a machine other than a shovel, such as a crane, an asphalt finisher, or a forklift. In the illustrated example, the shovel as the working machine 100 is an excavator equipped with a bucket 6 as an end attachment, but it may be an applied machine such as a forestry machine equipped with an end attachment other than the bucket 6. Furthermore, it may be a crawler crane equipped with a lower traveling body, an upper rotating body, and an attachment provided on the upper rotating body.

[0011] (First embodiment) First, with reference to Figure 1, an overview of the remote control system SYS (an example of a remote control system for work machinery) according to the first embodiment will be described. Figure 1 is a schematic diagram showing an example of the remote control system SYS according to the first embodiment.

[0012] <Equipment that constitutes a remote control system> As shown in FIG. 1, the remote operation system SYS according to the first embodiment includes a working machine 100 and a remote operation room RC. The configuration of the remote operation system SYS shown in FIG. 1 is an example and is not limited to this configuration.

[0013] The working machine 100 and the remote operation room RC are connected so as to be able to transmit and receive data via a communication line NW.

[0014] The working machine 100 enables wireless communication. Then, the working machine 100 can transmit and receive data to and from a device (for example, the remote operation room RC) connected to the communication line NW.

[0015] The working machine 100 is present at the work site where the working machine 100 performs work. And the working machine 100 can transmit information about the work site to the remote operation room RC. Thereby, the remote operation room RC can confirm the work site according to the information from the working machine 100. Note that in this embodiment, the device for measuring the work site is not limited to the working machine 100, and other devices such as a fixed-point measuring device present at the work site, a drone flying over the work site, or an imaging device that can be held by the user may be used.

[0016] The number of working machines 100 included in the remote operation system SYS may be one or a plurality. Thereby, the remote operation system SYS can provide information about the work site to the remote operation room RC through one or a plurality of working machines 100.

[0017] The remote operation room RC includes a communication device T2, a remote controller R40, an operation device R42, an operation sensor R43, and a display device D1E. Also, an operation seat DS on which a remote operator OP who remotely operates the working machine 100 sits is installed in the remote operation room RC.

[0018] The communication device (an example of the second communication device) T2 is configured to control communication with a communication device T1 (see FIG. 2) attached to the working machine 100.

[0019] The remote controller (an example of a remote control device) R40 is an information processing device that performs various calculations in order to remotely control the work machine 100. In this embodiment, the remote controller R40 is composed of a microcomputer including a CPU and memory. The various functions of the remote controller R40 are realized by the CPU executing a program stored in memory.

[0020] The display device D1E displays a screen based on information transmitted from the work machine 100, allowing the remote operator OP in the remote control room RC to visually check the area around the work machine 100. The display device D1E allows the operator to check the conditions of the work site, including the area around the work machine 100, even though the operator is in the remote control room RC.

[0021] An operating device R42 (an example of an operating unit) is equipped with an operating sensor R43 for detecting the operation of the operating device R42. The operating sensor R43 is, for example, a tilt sensor that detects the tilt angle of the operating lever, or an angle sensor that detects the oscillation angle of the operating lever around its pivot axis. The operating sensor R43 may also consist of other sensors such as a pressure sensor, a current sensor, a voltage sensor, or a distance sensor. The operating sensor R43 outputs information regarding the operation of the operating device R42 that it has detected to the remote controller R40. The remote controller R40 generates an operation signal based on the received information and transmits the generated operation signal to the work machine 100. The operating sensor R43 may also be configured to generate an operation signal. In this case, the operating sensor R43 may output the operation signal to the communication device T2 without going through the remote controller R40. This enables remote control of the work machine 100 from the remote control room RC.

[0022] The work machine 100 comprises a lower traveling body 1, an upper rotating body 3 mounted on the lower traveling body 1 so as to be rotatable via a slewing mechanism 2, an attachment AT for performing various tasks, and a driver's cab 10. The driver's cab 10 is also called a cabin or cab. The front side of the work machine 100 (upper rotating body 3) corresponds to the side on which the attachment AT is attached to the upper rotating body 3 when the work machine 100 is viewed from directly above along the slewing axis of the upper rotating body 3. The left, right, and rear sides of the work machine 100 (upper rotating body 3) correspond to the left, right, and rear sides as seen from the perspective of an operator seated in the driver's seat inside the driver's cab 10, respectively.

[0023] The lower travel body 1 includes, for example, a pair of left and right crawlers (not shown). Specifically, the crawlers include a left crawler and a right crawler. The left crawler is driven by a left travel hydraulic motor 2ML (see Figure 2), and the right crawler is driven by a right travel hydraulic motor 2MR (see Figure 2). The left travel hydraulic motor 2ML is a travel drive unit that drives the left crawler, which is the driven part, and can rotate the left crawler. The right travel hydraulic motor 2MR is a travel drive unit that drives the right crawler, which is the driven part, and can rotate the right crawler. Note that the travel drive units may also be electric motors.

[0024] A boom 4 is rotatably mounted to the front center of the upper slewing body 3, an arm 5 is rotatably mounted to the tip of the boom 4, and a bucket 6 is rotatably mounted to the tip of the arm 5. In the illustrated example, the boom 4, arm 5, and bucket 6 constitute an excavation attachment, which is an example of attachment AT. The boom 4, arm 5, and bucket 6 are driven by a boom cylinder 7, arm cylinder 8, and bucket cylinder 9, respectively.

[0025] Bucket 6 is an example of a work tool (end attachment). Bucket 6 is used, for example, for excavation work. Depending on the work content, other work tools may be attached to the tip of arm 5 instead of bucket 6. Other work tools may be other types of buckets, such as large buckets, slope buckets, or dredging buckets. Other work tools may also be types of work tools other than buckets, such as agitators, breakers, grapples, or lifting magnets. The excavation attachment may be provided with a bucket tilt mechanism.

[0026] Furthermore, the work machine 100 may have all or part of its driven parts, such as the lower traveling body 1, upper slewing body 3, boom 4, arm 5, and bucket 6, electrically driven. In other words, the work machine 100 may be a hybrid excavator or electric excavator, in which all or part of its driven parts are driven by electric actuators.

[0027] Furthermore, the work machine 100 is equipped with an imaging device S6, an indoor imaging device S7, and a projection device S8.

[0028] The imaging device S6 is mounted on the upper rotating body 3 and captures images of the area around the work machine 100, acquiring image information representing the area around the work machine 100. In the illustrated example, the imaging device S6 includes a front camera S6F, a left camera S6L, a right camera S6R, and a rear camera S6B.

[0029] The front camera S6F is a camera that captures images in front of the work machine 100 and is mounted on the roof of the cab 10, the side of the boom 4, or other external locations on the cab 10. The left camera S6L is a camera that captures images to the left of the work machine 100, the right camera S6R is a camera that captures images to the right of the work machine 100, and the rear camera S6B is a camera that captures images behind the work machine 100. Specifically, the front camera S6F, left camera S6L, right camera S6R, and rear camera S6B are all monocular wide-angle cameras equipped with image sensors such as CCD or CMOS, and they output the captured image information to the display device D1. The image information captured by the imaging device S6 is also taken into the controller 30.

[0030] In the illustrated example, the front camera S6F is mounted on the roof of the driver's cab 10, the left camera S6L is mounted on the upper left end of the upper surface of the upper rotating body 3, the right camera S6R is mounted on the upper right end of the upper surface of the upper rotating body 3, and the rear camera S6B is mounted on the upper rear end of the upper surface of the upper rotating body 3.

[0031] The controller 30 is an example of a control device and is composed of a computer including, for example, a CPU, a volatile memory device, a non-volatile memory device, and various input / output interfaces. The controller 30 implements various functions, for example, by reading a program from the non-volatile memory device, loading it into the volatile memory device, and having the CPU execute it. In the illustrated example, the controller 30 is configured to implement various functions and control the work machine 100. These functions include, for example, a machine guidance function that guides the operator in manually operating the work machine 100. The functions may also include a contact avoidance function that automatically or autonomously operates or stops the work machine 100 to avoid contact between the work machine 100 and objects within the monitoring range around the work machine 100.

[0032] The operator's cab 10 is a compartment where the operator sits and is located on the front left side of the upper rotating body 3. However, the operator's cab 10 may be omitted if the work machine 100 is remotely controlled or if the work machine 100 operates by fully automatic operation.

[0033] The indoor imaging device S7 is an imaging device installed inside the operator's cab 10, and is a camera that captures images in front of the work machine 100. For example, the indoor imaging device S7 is installed in the operator's cab 10 so as to be near the operator's line of sight from where the operator is seated.

[0034] The projection device S8 may be mounted on the roof of the operator's cab 10 of the upper rotating body 3. The projection device S8 is a device that projects visible light onto the ground. The projection device S8 is, for example, a liquid crystal projector for 3D projection mapping, a DLP® projector, etc. In this embodiment, the projection device S8 is a DLP® projector having a resolution of 1024 × 768 pixels. The projection device S8 may have a higher resolution or a lower resolution. Therefore, the projection device S8 displays information to the remote operator OP and people in the vicinity by irradiating the ground with visible light, using at least one of characters and lines.

[0035] In this embodiment, an example is described in which the projection device S8 is provided to project visible light in the forward direction of the work object to be worked on by the attachment AT of the work machine 100. However, this does not limit the position in which the projection device S8 is provided, and for example, the projection device S8 may be provided to project visible light in the right, left, or backward direction.

[0036] Furthermore, a drive mechanism S8a (see Figure 3) is provided between the projection device S8 and the roof of the driver's cab 10. The drive mechanism S8a has, for example, a two-axis rotation mechanism driven by a motor. Therefore, the drive mechanism S8a makes it possible to rotate the projection device S8 in the vertical and horizontal directions.

[0037] The work machine 100 according to this embodiment is configured to be remotely operated from outside the work machine 100. When the work machine 100 is remotely operated, the inside of the operator's cab 10 may be unoccupied.

[0038] Furthermore, the work machine 100 may automatically operate the actuators regardless of the operator's actions. This enables the work machine 100 to automatically operate at least a portion of the driven parts, such as the lower traveling body 1, the upper slewing body 3, the boom 4, the arm 5, and the bucket 6, that is, to achieve a so-called "machine control function".

[0039] Figure 2 is a schematic diagram showing an example of the configuration of the work machine 100 according to the first embodiment. In Figure 2, the mechanical power transmission system, hydraulic fluid line, pilot line, and electrical control system are indicated by double lines, thick solid lines, thick dashed lines, and dotted lines, respectively.

[0040] The drive system of the work machine 100 includes an engine 11, a regulator 13, a main pump 14, and a control valve unit 17. The hydraulic drive system of the work machine 100 also includes hydraulic actuators such as a slewing hydraulic motor 2A, a left travel hydraulic motor 2ML, a right travel hydraulic motor 2MR, a boom cylinder 7, an arm cylinder 8, and a bucket cylinder 9.

[0041] The engine 11 is an example of a power source for the work machine 100, and is mounted, for example, at the rear of the upper rotating body 3. The power source for the work machine 100 may also be a combination of a battery or fuel cell and an electric motor. Specifically, the engine 11 rotates at a constant speed at a preset target rotational speed under direct or indirect control by the controller 30, driving the main pump 14 and the pilot pump 15. The engine 11 is, for example, a diesel engine that uses light oil as fuel. The engine 11 may also be a gasoline engine or a hydrogen engine, etc.

[0042] The regulator 13 controls the discharge rate of the main pump 14. For example, the regulator 13 controls the discharge rate of the main pump 14 by adjusting the angle (tilt angle) of the swash plate of the main pump 14 in response to a control command from the controller 30.

[0043] The main pump 14, for example, is mounted at the rear of the upper rotating body 3, similar to the engine 11, and supplies hydraulic fluid to the control valve unit 17 through the hydraulic fluid line. In the illustrated example, the main pump 14 is a variable displacement hydraulic pump.

[0044] The control valve unit 17 is one of the hydraulic control devices that control the hydraulic system in the work machine 100. In the illustrated example, the control valve unit 17 includes control valves 171 to 176. The control valve unit 17 is configured to selectively supply hydraulic fluid discharged by the main pump 14 to one or more hydraulic actuators through the control valves 171 to 176. The control valves 171 to 176 control the flow rate of hydraulic fluid flowing from the main pump 14 to the hydraulic actuators, and the flow rate of hydraulic fluid flowing from the hydraulic actuators to the hydraulic fluid tank. The hydraulic actuators include a boom cylinder 7, an arm cylinder 8, a bucket cylinder 9, a left-travel hydraulic motor 2ML, a right-travel hydraulic motor 2MR, and a slewing hydraulic motor 2A. Specifically, control valve 171 corresponds to the left-travel hydraulic motor 2ML, control valve 172 corresponds to the right-travel hydraulic motor 2MR, and control valve 173 corresponds to the slewing hydraulic motor 2A. Furthermore, control valve 174 corresponds to bucket cylinder 9, control valve 175 corresponds to boom cylinder 7, and control valve 176 corresponds to arm cylinder 8.

[0045] The pilot pump 15 is an example of a pilot pressure generating device and is configured to supply hydraulic fluid to a hydraulic control device via a pilot line. In the illustrated example, the pilot pump 15 is a fixed-displacement hydraulic pump. However, the pilot pressure generating device may be implemented by the main pump 14. That is, the main pump 14 may have the function of supplying hydraulic fluid to the control valve unit 17 via a hydraulic fluid line, as well as the function of supplying hydraulic fluid to various hydraulic control devices via a pilot line. In this case, the pilot pump 15 may be omitted.

[0046] The discharge pressure sensor 28 is configured to detect the discharge pressure of the main pump 14. In the example shown in the figure, the discharge pressure sensor 28 outputs the detected value to the controller 30.

[0047] The operating device 26 is a device used by the operator to operate the actuator. The operating device 26 includes, for example, an operating lever and an operating pedal. The actuator may be a hydraulic actuator or an electric actuator.

[0048] The operation sensor 29 is configured to detect the operator's actions using the operation device 26. In this embodiment, the operation sensor 29 detects the operating direction and amount of the operation device 26 corresponding to each actuator and outputs the detected values ​​to the controller 30. In the illustrated example, the controller 30 can control the opening area of ​​the proportional valve 31 according to the output of the operation sensor 29. The controller 30 then supplies the hydraulic fluid discharged by the pilot pump 15 to the pilot port of the corresponding control valve in the control valve unit 17. The pressure of the hydraulic fluid supplied to each pilot port (pilot pressure) is, in principle, the pressure corresponding to the operating direction and amount of the operation device 26 corresponding to each hydraulic actuator. Thus, the operation device 26 is configured to supply the hydraulic fluid discharged by the pilot pump 15 to the pilot port of the corresponding control valve in the control valve unit 17.

[0049] The proportional valve 31, which functions as a control valve for machine control, is located in the pipeline connecting the pilot pump 15 and the pilot port of the control valve in the control valve unit 17, and is configured to change the flow area of ​​the pipeline. In the illustrated example, the proportional valve 31 operates in response to control commands output by the controller 30. Therefore, the controller 30 can adjust the pilot pressure acting on the pilot port of the control valve by the proportional valve 31, independently of the operation of the operating device 26 by the operator.

[0050] This configuration allows the controller 30 to operate the hydraulic actuator corresponding to a specific operating device 26 even when no operation is being performed on that particular operating device 26.

[0051] Furthermore, as shown in Figure 2, the control system of the work machine 100 includes a controller 30, angle sensors S1, S2, S3, machine tilt sensor S4, rotation sensor S5, imaging device S6, indoor imaging device S7, projection device S8, positioning device PS, display device D1, and communication device T1, etc. Note that the imaging device S6, indoor imaging device S7, and projection device S8 are as described above and will not be explained further.

[0052] A communication device (an example of a first communication device) T1 communicates with external devices through a communication network including a mobile communication network, a satellite communication network, or the Internet. The communication device T1 is, for example, a mobile communication module compatible with mobile communication standards such as LTE (Long Term Evolution), 4G (4th Generation), or 5G (5th Generation), a communication module compatible with short-range wireless communication standards such as Wi-Fi (registered trademark) or Bluetooth (registered trademark), or a satellite communication module for connecting to a satellite communication network.

[0053] The boom angle sensor S1 detects the boom angle, which is the rotation angle of the boom 4 relative to the upper slewing body 3. The arm angle sensor S2 detects the arm angle, which is the rotation angle of the arm 5 relative to the boom 4. The bucket angle sensor S3 detects the bucket angle, which is the rotation angle of the bucket 6 relative to the arm 5.

[0054] The boom angle sensor S1, arm angle sensor S2, and bucket angle sensor S3 may each be, for example, a rotary encoder, acceleration sensor, 6-axis sensor, IMU (Inertial Measurement Unit), or a potentiometer using a variable resistor, or a cylinder stroke sensor for detecting the stroke amount of a hydraulic cylinder.

[0055] The detection signals corresponding to the boom angle from the boom angle sensor S1, the detection signals corresponding to the arm angle from the arm angle sensor S2, and the detection signals corresponding to the bucket angle from the bucket angle sensor S3 are input to the controller 30. The detection signals may include angular velocity in addition to angle.

[0056] In this embodiment, the boom angle sensor S1, the arm angle sensor S2, and the bucket angle sensor S3 are collectively referred to as angle sensors S1, S2, and S3.

[0057] The machine tilt sensor S4 detects the tilt state of the machine (lower traveling body 1 or upper rotating body 3) relative to the horizontal plane. The machine tilt sensor S4 is, for example, attached to the upper rotating body 3 and detects the tilt angle of the work machine 100 (i.e., the upper rotating body 3) around two axes: the longitudinal direction and the lateral direction. The machine tilt sensor S4 may be, for example, an acceleration sensor, a 6-axis sensor, or an IMU. The detection signal corresponding to the tilt angle from the machine tilt sensor S4 is input to the controller 30.

[0058] The rotation sensor S5 outputs information regarding the rotation of the upper rotating body 3. The rotation sensor S5 detects, for example, the rotational angular velocity of the upper rotating body 3 relative to the lower traveling body 1. The rotation sensor S5 may also detect the rotation angle. The rotation sensor S5 may be, for example, a gyro sensor, a resolver, or a rotary encoder. The detection signal corresponding to the rotation angle or rotational angular velocity of the upper rotating body 3 detected by the rotation sensor S5 is input to the controller 30.

[0059] The positioning device PS measures the position of the upper rotating body 3. The positioning device PS is, for example, a GNSS (Global Navigation Satellite System) compass and detects the position and orientation of the upper rotating body 3. The detection signals corresponding to the position and orientation of the upper rotating body 3 are received by the controller 30. The function of detecting the orientation of the upper rotating body 3 may be realized by an orientation sensor attached to the upper rotating body 3. In this embodiment, the positioning device PS measures the current position of the work machine 100 in a globally identifiable reference coordinate system.

[0060] A reference coordinate system is, for example, the World Geodetic System, which can determine a location on Earth. The World Geodetic System is a three-dimensional orthogonal XYZ coordinate system with its origin at the Earth's center of mass, the X-axis pointing in the direction of the intersection of the Greenwich Meridian and the equator, the Y-axis pointing in the direction of 90 degrees east longitude, and the Z-axis pointing in the direction of the North Pole.

[0061] The controller 30 is an example of a control device and is composed of a computer including, for example, a CPU, a volatile memory device, a non-volatile memory device, and various input / output interfaces. The controller 30 implements various functions, for example, by reading a program from the non-volatile memory device, loading it into the volatile memory device, and having the CPU execute it. In the illustrated example, the controller 30 is configured to implement various functions and control the work machine 100. These functions include, for example, a machine guidance function that guides the operator in manually operating the work machine 100. The functions may also include a contact avoidance function that automatically or autonomously operates or stops the work machine 100 to avoid contact between the work machine 100 and objects within the monitoring range around the work machine 100.

[0062] The controller 30 is configured to output control commands to the regulator 13 as needed, thereby changing the discharge rate of the main pump 14.

[0063] Furthermore, the controller 30 may be configured to perform control related to a machine guidance function that guides the manual operation of the work machine 100 by the operator through the operating device 26. Alternatively, the controller 30 may be configured to perform control related to a machine control function that automatically assists the manual operation of the work machine 100 by the operator through the operating device 26.

[0064] Furthermore, some of the functions of controller 30 may be implemented by other controllers (control devices). In other words, the functions of controller 30 may be implemented in a manner distributed among multiple controllers. For example, machine guidance functions and machine control functions may be implemented by dedicated controllers (control devices).

[0065] <Block configuration of the remote control system> Figure 3 is a functional block diagram showing an example configuration of the remote control system SYS according to this embodiment. In the example shown in Figure 3, the block configurations of the remote control room RC and the work machine 100, which are included in the remote control system SYS, are shown. The hardware configuration of the work machine 100 will not be explained.

[0066] <Configuration of the remote control room> The remote control room RC includes a remote controller R40, a communication device T2, an operation sensor R43, an operation device R42, a display device D1E, and an input device D2E. The communication device T2, operation sensor R43, and operation device R42 have been described above, so their explanation is omitted.

[0067] The input device D2E is located within reach of the operator seated in the remote control room RC and receives various operation inputs from the operator, outputting signals corresponding to the operation inputs to the remote controller R40. The input device D2E includes a touch panel mounted on the display of a display device that shows various information images, a knob switch located at the tip of the lever part of the operation device R42, and button switches, levers, toggles, rotary dials, etc., installed around the display device D1E. Signals corresponding to the operations performed on the input device D2E are received by the remote controller R40.

[0068] Next, we will explain the remote control room RC. Figure 4 shows an example of the layout of the remote control room RC. The remote control room RC is equipped with multiple control devices R42, with the operator's seat DS as the reference point.

[0069] In this embodiment, the display device D1E is a multi-display consisting of six monitors arranged in two vertical rows and three horizontal columns, as shown in Figure 4. Specifically, the display device D1E includes the central monitor D1Ea, the upper monitor D1Eb, the left monitor D1Ec, the right monitor D1Ed, the upper left monitor D1Ee, and the upper right monitor D1Ef.

[0070] <<Explanation of the situation regarding the ground at the work site from the remote control room>> The display device D1E in the remote control room RC displays image information captured by the imaging device S6 installed on the work machine 100. The remote operator OP in the remote control room RC then refers to the image information displayed on the display device D1E to understand the situation around the work machine 100. When the imaging device S6 is a normal monocular camera, there is a problem in that it is difficult for the remote operator OP to understand the sense of distance in the direction away from the work machine 100 and in the height direction when referring to the image information captured by the imaging device S6.

[0071] Conventional techniques involve overlaying detection results, which indicate ground topography measured by LiDAR, onto image information captured by an imaging device, thereby allowing the operator to recognize the topography. However, processing point cloud data detected by LiDAR is computationally intensive, which can cause delays in the process of overlaying the detection results indicating ground topography onto the captured image information.

[0072] Therefore, the projection device S8 according to this embodiment projects visible light onto the ground that represents information that can identify the direction away from the work machine 100 (depth direction).

[0073] Figure 5 is an explanatory diagram showing the situation in which the projection device S8 of the work machine 100 according to this embodiment is projecting visible light onto the ground.

[0074] In the example shown in Figure 5, the projection device S8 projects identifiable information onto the ground 1501, which includes grid lines that allow identification of the distance from the work machine 100 in the forward direction (away from it).

[0075] In the example shown in Figure 5, lines are shown at predetermined intervals in both the front-to-back and left-to-right directions within the front-facing region 1502 of the work machine 100.

[0076] Then, the indoor imaging device S7 in the driver's cab 10 captures an image of the area including the ground in front of the driver's seat, and the communication device T1 transmits the image information captured by the indoor imaging device S7 to the remote control room RC.

[0077] Then, the communication device T2 in the remote control room RC receives image information from the work machine 100, and the display device D1E displays the received image information.

[0078] The image information displayed on the display device D1E shows lines on the ground at predetermined intervals in the front-to-back direction (away from the work machine 100) and left-to-right direction. Therefore, the remote operator OP can perceive the distance from the work machine 100 in the front-to-back direction (away from the work machine 100) and left-to-right direction.

[0079] <<Functional Blocks of Work Machines>> Returning to Figure 3, we will now describe each functional block within the controller 30 of the work machine 100. Each functional block within the controller 30 is conceptual and does not necessarily need to be physically configured as shown in the figure. All or part of each functional block can be configured by distributing and integrating them functionally or physically in any unit. Each processing function performed in each functional block is realized, all or any part thereof, by a program executed on the CPU. Alternatively, each functional block may be realized as hardware using wired logic. The controller 30, by realizing the program, includes an acquisition unit 301, a generation unit 302, a projection control unit 303, a drive control unit 305, a transmission control unit 306, a reception control unit 304, and an actuator drive unit 307.

[0080] The acquisition unit 301 acquires signals from various detection devices installed on the work machine 100. For example, the acquisition unit 301 acquires position information from the positioning device PS, which shows the measurement results such as the position and orientation of the work machine 100.

[0081] For example, the acquisition unit 301 acquires image information from the imaging device S6 showing the area around the work machine 100, and also acquires image information from the indoor imaging device S7 showing the forward direction from the driver's cab 10.

[0082] The generation unit 302 generates image information showing grid lines, which is projected by the projection device S8. The generation unit 302 generates image information such that square or rectangular grid lines are shown on the horizontal plane, taking into consideration, for example, the inclination of the work machine 100 and the rotation angle of the drive mechanism S8a, as indicated by the measurement results of the positioning device PS. The horizontal plane is a horizontal plane that includes a predetermined point on which the lower traveling body 1 is in contact with the ground. For example, the generation unit 302 generates image information such that square or rectangular grid lines are shown on the horizontal plane by performing calculations on the image information showing grid lines using matrices that indicate the inclination of the work machine 100 and the rotation angle of the drive mechanism S8a, respectively.

[0083] The projection control unit 303 projects the image information generated by the generation unit 302 onto the ground as visible light using the projection device S8.

[0084] The receiving control unit 304 receives operation signals from the remote control room RC to control the operation of the work machine 100. In another example, the receiving control unit 304 receives a drive signal from the remote control room RC to rotate the drive mechanism of the projection device S8.

[0085] The drive control unit 305 is configured to drive the motor mounted on the drive mechanism of the projection device S8. In this embodiment, the drive control unit 305 generates and outputs an operation signal for the drive mechanism S8a of the projection device S8 based on the drive signal transmitted from the remote control room RC.

[0086] The transmission control unit 306 controls the transmission of various information based on the acquisition results of the acquisition unit 301 to the remote control room RC via the communication device T1. For example, the transmission control unit 306 controls the transmission of image information captured by the imaging device S6, image information captured by the indoor imaging device S7, position information indicating the position and orientation of the work machine 100, and detection results from various detection devices to the remote control room RC.

[0087] The actuator drive unit 307 is configured to drive the actuator mounted on the work machine 100. In this embodiment, the actuator drive unit 307 generates and outputs an operating signal for each of the multiple solenoid valves included in the proportional valve 31 based on the operation signal transmitted from the remote control room RC.

[0088] Upon receiving an activation signal, each solenoid valve increases or decreases the pilot pressure acting on the pilot port of the corresponding control valve in the control valve unit 17. As a result, the hydraulic actuator corresponding to each control valve operates at a speed corresponding to the stroke amount of the control valve.

[0089] <<Functional Blocks of the Remote Control Room>> This section describes the functional blocks within the remote controller R40 of the remote control room RC. Each functional block within the remote controller R40 is conceptual and does not necessarily need to be physically configured as shown in the diagram. All or part of each functional block can be configured by distributing and integrating them functionally or physically in any unit. Each processing function performed by each functional block is realized, all or any part thereof, by a program executed on the CPU. Alternatively, each functional block may be realized as hardware using wired logic. The remote controller R40, by implementing the program, includes a receiving control unit 401, a display control unit 402, a signal generation unit 403, a transmission control unit 404, and an input control unit 405.

[0090] The receiving control unit 401 controls the reception of various information from each of the work machines 100 via the communication device T2.

[0091] For example, the receiving control unit 401 controls the receiving of image information captured by the imaging device S6, image information captured by the indoor imaging device S7, position information indicating the position and orientation of the working machine 100, and detection results from various detection devices from the working machine 100.

[0092] The display control unit 402 controls the display of various information on the display device D1E. For example, the display control unit 402 controls the display of the image information captured by the indoor imaging device S7 as an image showing the work target of the work machine 100 on the display device D1E. In addition, the display control unit 402 controls the display of information showing the current status of the work machine 100 on the display device D1E based on the detection results of the various detection devices of the work machine 100 received by the receiving control unit 401.

[0093] Figure 6 shows an example of a screen displayed on the central monitor D1Ea by the display control unit 402 according to this embodiment. In the example shown in Figure 6, the central monitor D1Ea displays image information 1600 captured by the indoor imaging device S7.

[0094] The image information 1600 shown in Figure 6 represents the grid lines 1601 projected by the projection device S8.

[0095] The grid lines 1601 shown in Figure 6 are drawn at predetermined intervals in the front-to-back (away from the work machine 100) and left-to-right directions relative to the work machine 100. The interval between these lines can be determined according to the embodiment, and may be, for example, 1 m. Therefore, the remote operator OP can grasp the sense of distance in the front-to-back and left-to-right directions relative to the work machine 100 by referring to the grid lines 1601.

[0096] Furthermore, if there are irregularities in the ground, the grid line 1601 will bend to conform to those irregularities. For example, if there is a hole 1611 in the ground, a portion 1601a of the grid line 1601 will bend into a concave shape. Similarly, if there is an object placed on the ground or if soil or sand is piled up on the ground, a portion of the grid line 1601 will bend to follow the object or soil. Therefore, the remote operator OP can recognize irregularities in the ground by referring to the grid line 1601.

[0097] The length of the grid line 1601 in the left-right direction according to this embodiment is set to be at least longer than the width of the work machine 100. Therefore, the remote operator OP can recognize the unevenness of the ground along the travel path when the work machine 100 travels forward by referring to the grid line 1601.

[0098] The projection control unit 303 of the work machine 100 according to this embodiment does not limit the information projected from the projection device S8 to grid lines. For example, the projection control unit 303 may project text information onto the ground using the projection device S8. For example, when the work machine 100 is performing compaction work, the projection control unit 303 may display the required compaction pressure "compaction ○t" on the ground. In this way, the projection control unit 303 may display text information necessary for the work of the work machine 100. For example, the projection control unit 303 may project the amount of soil to be excavated, or the depth to be excavated, as text information.

[0099] The image information 1600 displayed on the central monitor D1Ea includes text information projected onto the ground (for example, "Compaction ○t"), allowing the remote operator OP to recognize the work to be performed using the work machine 100.

[0100] The projection control unit 303 may project the area to be compacted. The image information 1600 displayed on the central monitor D1Ea shows the area to be compacted 1612, so the remote operator OP can recognize the area to be worked on using the work machine 100.

[0101] Furthermore, the information projected from the projection device S8 by the projection control unit 303 of the work machine 100 according to this embodiment may be other than character information and grid lines. For example, the projection control unit 303 recognizes the position of the tip of the bucket 6 based on angle sensors S1, S2, and S3, and projects information (e.g., a line) 1613 indicating the tip of the bucket 6 onto the ground vertically downward from the tip of the bucket 6.

[0102] Furthermore, the information projected by the projection device S8 may include information indicating the current status of the work machine 100. For example, the projection control unit 303 may project information indicating at least one of the following: the bucket load weight or the remaining load. The bucket load weight is the weight of the load taken into the bucket 6. The bucket load weight may be calculated, for example, based on the loads detected by the boom cylinder pressure sensor, the arm cylinder pressure sensor, and the bucket cylinder pressure sensor. The remaining load is the maximum load capacity of the dump truck minus the weight of the objects already loaded on the dump truck bed. For example, the projection control unit 303 may project information indicating at least one of the following, such as the bucket load weight or the remaining load, onto, for example, the underside of the arm 5.

[0103] Thus, the projection control unit 303 may project different useful information onto the ground for each task. The different useful information for each task may include, for example, at least one of instruction information or numerical information that changes according to the progress of the task.

[0104] Furthermore, the projection device S8 is not limited to projecting useful information related to the work, but may also project information related to the movement of the work machine 100. For example, the projection device S8 may project information indicating the direction in which the lower traveling body 1 can currently travel (e.g., forward or backward). The information indicating the direction in which the lower traveling body 1 can currently travel may be, for example, an arrow or a trajectory. By projecting information indicating the direction in which the lower traveling body 1 can currently travel with the projection device S8 according to this embodiment, the operator OP can recognize the current orientation of the lower traveling body 1 when referring to the information displayed on the central monitor D1Ea. Therefore, this embodiment can prevent the work machine 100 from moving in a direction unintended by the operator OP when the operator OP has given a travel command for the work machine 100. Thus, safety can be improved.

[0105] Returning to Figure 3, the signal generation unit 403 generates signals to control the operation of the work machine 100 according to the information received from the operation sensor R43.

[0106] The transmission control unit 404 controls the transmission of various types of information to the work machine 100. For example, the transmission control unit 404 controls the transmission of operation signals generated by the signal generation unit 403 to the work machine 100.

[0107] The input control unit 405 processes the information received by the input device D2E. For example, it processes the settings related to the information to be projected by the projection device S8, which have been received as input from the input device D2E.

[0108] For example, input device D2E receives input from input device D2E for at least one setting of the following: the color of the visible light projected by projection device S8, the intensity of the visible light, and the direction in which projection device S8 projects the visible light.

[0109] The input control unit 405 then processes the received settings. The transmission control unit 404 then transmits information indicating the processed settings to the work machine 100. The controller 30 of the work machine 100 controls the projection device S8 to project visible light onto the ground according to the received settings. Specifically, the generation unit 302 generates image information in which the color of the projected visible light is changed according to the received color settings. The projection control unit 303 also changes the intensity of the projected visible light according to the received intensity settings. The drive control unit 305 also controls the drive mechanism S8a to change the direction in which the projection device S8 projects the visible light according to the received direction settings.

[0110] Therefore, the remote operator OP can change the visible light to a color that is easy to see, taking into account the color of the ground. In addition, the remote operator OP can change the intensity of the visible light projected by the projection device S8 when the grid lines are difficult to see due to sunlight or other factors. Furthermore, the remote operator OP can adjust the projection direction of the projection device S8 so that the visible light is projected onto the ground whose condition they want to check. Therefore, by changing the settings, the remote operator OP can recognize the ground conditions with greater accuracy, thereby improving the work efficiency using the work machine 100.

[0111] Thus, the remote control system SYS according to this embodiment can change the information projected from the projection device S8 according to the information received from the remote control room RC. In this embodiment, the information that the remote control room RC can change is not limited to the color of the visible light, the intensity of the visible light, or the direction in which the projection device S8 projects the visible light. For example, it is possible to adjust the size of the mesh, the range in which the visible light is projected, and the type of information to be projected (e.g., a grid or a circle) when projecting a mesh grid onto the ground.

[0112] As a detailed example, input device D2E can accept settings for the area (size) over which the projection device S8 projects visible light. For example, input device D2E can accept settings that limit the area over which the projection device S8 projects visible light to the work target of the bucket 6 of the work machine 100. Since the area over which visible light is projected is narrowed, power consumption can be reduced.

[0113] Another example is that the input device D2E can accept settings to change the size of the grid lines. For example, the input device D2E can accept a setting to change the grid size from 1m intervals to 50cm intervals. The projection control unit 303 then projects grid lines according to the accepted setting. In this embodiment, the remote operator OP can easily understand the ground conditions by adjusting the spacing of the grid lines according to the unevenness of the ground.

[0114] In this embodiment, the projection device S8 projects grid lines onto the ground using visible light. The grid lines are drawn at predetermined intervals in the forward direction (away from) the work machine 100. Therefore, when the remote operator OP refers to the image information captured by the indoor imaging device S7, they can grasp the sense of distance in the forward direction (away from) by looking at the grid lines in the image information. Furthermore, if there are irregularities in the ground, the grid lines will be displayed curved according to those irregularities. Consequently, when the remote operator OP refers to the image information captured by the indoor imaging device S7, they can recognize changes in the height direction of the ground by looking at the grid lines in the image information.

[0115] (Modified version of the first embodiment) In the first embodiment, an example was described in which the display device D1E displays image information captured by the indoor imaging device S7. However, the first embodiment does not limit the image information displayed by the display device D1E to image information captured by the indoor imaging device S7. Therefore, in a modification of the first embodiment, an example will be described in which image information captured by the imaging device S6 is displayed.

[0116] Figure 7 shows an example of a screen displayed on the central monitor D1Ea by the display control unit 402 according to this embodiment. In the example shown in Figure 7, the central monitor D1Ea displays image information 1700 captured by the front camera S6F.

[0117] The image information 1700 shown in Figure 7 represents the grid lines 1701 projected by the projection device S8.

[0118] The grid lines 1701 shown in Figure 7, similar to the embodiments described above, are drawn at predetermined intervals in the front-to-back direction (away from the work machine 100) and the left-to-right direction of the work machine 100.

[0119] Furthermore, if there are irregularities in the ground, a portion of the grid line 1701 will bend to conform to those irregularities. For example, if there are holes or other irregularities in the ground, a portion 1701a of the grid line 1701 will bend into a concave shape. Similarly, if there are objects placed on the ground or if soil or other material is piled up on the ground, a portion 1701b of the grid line 1701 will bend into a convex shape to follow the object or soil.

[0120] Thus, the image information captured by the imaging device S6 is taken from a viewpoint higher than that of the remote operator OP, making it difficult to grasp the unevenness of the ground. In contrast, in the modified version, the projection device S8 projects grid lines 1701 onto the ground, allowing the remote operator OP to recognize the unevenness of the ground.

[0121] (Second embodiment) The embodiments and modifications described above describe the case where the work machine 100 is in contact with level ground. However, the embodiments and modifications described above are not limited to the case where the work machine 100 is in contact with level ground. Therefore, the second embodiment will describe the case where the work machine 100 is in contact with an inclined surface. Note that the remote control system SYS in the second embodiment has the same configuration as in the first embodiment, so its description will be omitted.

[0122] Figure 8 is an explanatory diagram showing the change in the projection direction of the projection device S8 when the work machine 100 according to the second embodiment is in contact with an inclined surface. In the example shown in Figure 8(a), the work machine 100 is in contact with an inclined surface 1821 and is moving forward 1811.

[0123] As shown in Figure 8(a), when the projection device S8 projects visible light onto the range 1801, it is difficult for the remote operator OP to grasp the ground conditions in front of the work machine 100 1811.

[0124] Therefore, the remote operator OP sets the projection device S8 to project visible light downwards using the input device D2E. Then, the transmission control unit 404 transmits information to the work machine 100 indicating that the drive mechanism S8a is set to be driven downwards.

[0125] Then, the drive control unit 305 drives the drive mechanism S8a according to the received settings.

[0126] In the example shown in Figure 8(b), the operation of the drive mechanism S8a changes the projection direction of the projection device S8 downwards.

[0127] Furthermore, when the projection device S8 projects visible light onto the range 1802, the remote operator OP can grasp the ground conditions in the forward direction 1812 of the work machine 100.

[0128] In this embodiment, the change in projection direction by the projection device S8 is not limited to input by the remote operator OP. For example, the projection control unit 303 may determine the angle of the slope based on the inclination of the work machine 100 detected by the positioning device PS, and control the drive mechanism S8a to drive in the vertical direction so that the illumination direction corresponds to the angle of the slope.

[0129] Furthermore, the projection control unit 303 may control the drive mechanism s8a to drive in the left-right direction so that when the travel path of the work machine 100 is changed to the right or left side, the ground on the right or left side is projected with visible light.

[0130] In this embodiment, the remote control system SYS allows the controller 30 to switch the projection direction of the projection device S8 according to the ground conditions of the work machine 100, so that the ground conditions can be appropriately recognized, thereby improving work efficiency when working on the ground and enabling operation of the work machine 100 according to the ground conditions, thus improving safety.

[0131] (Third embodiment) The embodiments and modifications described above illustrate examples of projecting grid lines. However, the embodiments and modifications described above do not limit the visible light information projected by the projection device S8 to grid lines. Therefore, the third embodiment describes an example in which the projection device S8 projects information other than grid lines. Note that the remote control system SYS in the third embodiment has the same configuration as in the first embodiment, so its description is omitted.

[0132] Figure 9 is an explanatory diagram illustrating an example of an arc projected by the projection device S8 provided on the work machine 100 according to this embodiment.

[0133] In the example shown in Figure 9, the projection device S8 is defined as the region 1901 from which visible light can be projected.

[0134] The projection control unit 303 then uses the projection device S8 to project visible light onto the ground, showing the arc (an example of a region) 1912 of the maximum working radius that the tip of the bucket 6 (an example of a predetermined part) located at the tip of the attachment AT can reach, and the arc (an example of a region) 1911 of the operating radius that the tip of the bucket 6 can reach in the current position of the attachment AT.

[0135] Specifically, the projection control unit 303 projects an arc 1912 that indicates the furthest area reachable by the tip of the bucket 6 when the attachment AT is extended away from the work machine 100 (when a swivel operation is performed), based on design information including the dimensions of the attachment AT. In this embodiment, the remote control system SYS will describe an example in which the area reachable by the tip of the attachment AT is projected in visible light. However, this embodiment does not limit the part that serves as the reference for projecting visible light to the tip of the attachment AT. In other words, it is sufficient for the remote operator OP or workers in the vicinity to recognize the working range or current operating status of the attachment AT, and the projection control unit 303 may project in visible light the area reachable by any part of the attachment AT (for example, the bottom surface of the bucket 6).

[0136] Furthermore, the projection control unit 303 determines the current orientation of the attachment AT based on the detection results of the angle sensors S1, S2, and S3, and design information including the dimensions of the boom 4, arm 5, and bucket 6, and projects an arc 1911 that indicates the area that the tip of the bucket 6 can reach (when a swivel motion is performed) in the determined current orientation.

[0137] Furthermore, the projection control unit 303 projects character information indicating the distance from the work machine 100 for each arc. For example, the projection control unit 303 projects the character string "4m" corresponding to arc 1911 and the character string "10m" corresponding to arc 1912.

[0138] In this embodiment, the projection control unit 303 projects an arc with a radius equal to the line moving away from the work machine 100, allowing the remote operator OP to perceive a sense of distance in the direction away from the work machine 100.

[0139] In this embodiment, an example is shown of displaying the arc 1912 representing the maximum working radius that the tip of the bucket 6 can reach, and the arc 1911 representing the operating radius that the tip of the bucket 6 can reach in the current position of the attachment AT. However, the method is not limited to projecting both arc 1912 and arc 1911, and projection of either arc 1912 or arc 1911 is also acceptable.

[0140] Furthermore, the projection control unit 303 is not limited to projecting only arcs 1912 and 1911; for example, it may combine the projection of arcs 1912 and 1911 with the projection of grid lines.

[0141] In the remote control system SYS according to this embodiment, an arc is projected with the radius being the direction away from the work machine 100. Therefore, when the remote operator OP refers to the image information captured by the indoor imaging device S7, they can perceive the sense of distance in the direction away, and thus grasp the sense of distance in the forward direction (away from the machine). Furthermore, if there are irregularities in the ground, the lines of the arcs 1911 and 1912 will be displayed curved according to the irregularities. Therefore, when the remote operator OP refers to the image information captured by the indoor imaging device S7, they can perceive changes in the height direction of the ground by the lines of the arcs 1911 and 1912 shown in the image information.

[0142] Furthermore, the remote control system SYS projects the arc 1912 of the maximum working radius and the arc 1911 of the operating radius that the tip of the bucket 6 can reach in the current orientation of the attachment AT onto the ground, so that workers present around the work machine 100 can recognize the working range of the work machine 100. Therefore, workers can perform their work while taking into account the working range of the work machine 100 and avoiding contact with the work machine 100, thereby improving safety.

[0143] (Fourth embodiment) For example, the visible light information projected by the projection control unit 303 may be switched according to the detection results of various detection devices provided on the work machine 100.

[0144] Therefore, in the fourth embodiment, an example will be described in which the projection control unit 303 of the work machine 100 switches the information of the visible light to be projected according to the result of detecting a person.

[0145] In this embodiment, the projection control unit 303 detects a person in front of the work machine 100 based on image information captured by the front camera (an example of a detection device) S6F. In this embodiment, an example is described in which the front camera S6F is used to detect a person in the vicinity of the work machine 100, but the detection unit for detecting a person is not limited to the imaging device S6, and other detection units may be used.

[0146] The projection control unit 303 then changes the visible light information to be projected depending on whether or not it has detected a person in front of the work machine 100.

[0147] If the projection control unit 303 does not detect a person in front of the work machine 100, it projects arcs 1912 and 1911 onto the ground as shown in Figure 9, similar to the third embodiment.

[0148] The projection control unit 303 detects a person in front of the work machine 100 and changes the information projected onto the ground.

[0149] Figure 10 is an explanatory diagram illustrating an example of an arc projected by a projection device S8 provided on the work machine 100 when a person is detected in front of the work machine 100 according to the fourth embodiment.

[0150] In the example shown in Figure 10, the projection device S8 is defined as the area 2001 from which visible light can be projected. Figure 10 shows an example in which a person 2002 is detected by the front camera S6F.

[0151] Then, when the front camera S6F detects a person 2002, the projection control unit 303 projects a mark 2013 indicating the presence of a person in the direction where the person 2002 is located. Furthermore, the projection control unit 303 projects an arc 2012 in a different color from the arc 1912. The projection control unit 303 also projects an arc 2011 indicating the area that the tip of the bucket 6 can reach (if a rotational movement is performed) in the current orientation, similar to the arc 1911 in the third embodiment.

[0152] Furthermore, the projection control unit 303 according to this embodiment changes the information to be projected according to the distance from the work machine 100 to the person detected. For example, the projection control unit 303 switches the intensity of the projected visible light depending on whether the distance from the work machine 100 to the person detected is within a reference distance obtained by adding a predetermined distance (e.g., 3m) to the maximum working radius.

[0153] In this embodiment, the projection control unit 303 changes the projected information depending on whether a person is detected and whether the distance to the detected person is within a reference distance. Note that this embodiment is not limited to changing the color of the visible light, changing the intensity of the visible light, and projecting a mark indicating the presence of a person.

[0154] Next, the procedure for switching the information to be projected in the controller 30 according to this embodiment will be described. Figure 11 is a flowchart showing the procedure for switching the information to be projected by the controller 30 according to this embodiment. In this embodiment, the projection device S8 can switch whether or not to project visible light in response to an operation from the remote control room RC.

[0155] First, the receiving control unit 304 determines whether or not it has received a command from the remote control room RC to start projecting visible light (S2101). If the receiving control unit 304 determines that it has not received the command (S2101: NO), it repeats the process until it determines that it has received the command.

[0156] On the other hand, if the receiving control unit 304 determines that it has received a command from the remote control room RC to start projecting visible light (S2101: YES), the projection control unit 303 starts projecting visible light (S2102).

[0157] The projection control unit 303 determines whether or not a person has been detected from the image information captured by the front camera S6F (S2103). If the projection control unit 303 determines that no person has been detected (S2103: NO), it projects normal visible light information from the projection device S8 (S2104). As normal visible light information, for example, the projection control unit 303 projects arcs 1911 and 1912 shown in Figure 9 onto the ground. After that, the process proceeds to S2109.

[0158] If the projection control unit 303 determines that a person has been detected (S2103: YES), it determines the distance from the work machine 100 to the detected person (S2105). Any method can be used to determine the distance from the work machine 100 to the detected person. For example, a pre-trained model may be used. For example, a pre-trained model can be prepared by machine learning using training data that shows image information indicating a person and the distance to that person. The projection control unit 303 then inputs the image information captured by the front camera S6F into the pre-trained model and receives information indicating the distance from the work machine 100 to the person.

[0159] The projection control unit 303 then determines whether the distance from the work machine 100 to the person is within a reference distance obtained by adding a predetermined distance (for example, 3m) to the maximum working radius (S2106).

[0160] If the projection control unit 303 determines that the distance from the work machine 100 to the person is greater than the reference distance (S2106: NO), the projection control unit 303 changes the color of the arc of the maximum working radius and projects a mark indicating the direction in which the person is located, outside the arc of the maximum working radius but in the direction in which the detected person is located (S2107). The color of the arc is preferably a color that will draw the attention of the remote operator OP, such as yellow. The process then proceeds to S2108.

[0161] On the other hand, if the projection control unit 303 determines that the distance from the work machine 100 to the person is within the reference distance (S2105: YES), the projection control unit 303 changes the color of the arc of the maximum working radius to weaken the intensity of the visible light of the arc of the maximum working radius, and projects a mark indicating the direction in which the person is located outside the arc (S2108). The color of the arc is preferably a color that will alert the remote operator OP, such as red. After that, the process proceeds to S2109.

[0162] The receiving control unit 304 then determines whether or not it has received a command from the remote control room RC to stop the projection of visible light (S2109). If the receiving control unit 304 determines that it has not received the command (S2109: NO), it proceeds to process again from S2103.

[0163] On the other hand, if the receiving control unit 304 determines that it has received a command from the remote control room RC to stop the projection of visible light (S2109: YES), it terminates processing.

[0164] The controller 30 according to this embodiment changes the information projected as visible light depending on whether the detected person is within a reference distance or not by performing the control described above. For example, if the projection control unit 303 approaches the work machine 100 within a reference distance, it weakens the intensity of the visible light, thereby suppressing any impact on the person even if the visible light directly hits them, and improving safety. The area in which the intensity of the visible light is weakened may be a predetermined range from the position where the detected person is located within the area that the projection device S8 can project. In other words, by partially weakening the intensity of the visible light in the area where the person is located, it is possible to achieve both improved safety and maintenance of visibility.

[0165] Furthermore, the projection control unit 303 may perform control to display a different mark indicating the direction of the person's presence when it determines that the distance from the work machine 100 to the person is within a reference distance, compared to when it determines that the distance is longer than the reference range. For example, when the projection control unit 303 approaches the work machine 100 to within a reference distance, it may project a mark that more strongly emphasizes the presence of the person. Furthermore, when the work machine 100 approaches to within a reference distance, it may also project text information indicating that the person is approaching.

[0166] In this embodiment, the projection control unit 303 projects a mark indicating the presence of a person when a person is detected. Therefore, not only the remote operator OP but also workers present around the work machine 100 can be aware that a person has been detected around the work machine 100. As a result, the remote operator OP and workers can perform their work in a manner that prevents workers from coming into contact with the work machine 100, thereby improving safety.

[0167] <effect> In the remote control system SYS for the work machine in the above-described embodiment and modified example, the work machine 100 includes a projection device S8 that projects visible light onto the ground, which contains information indicating the direction away from the machine; an indoor imaging device S7 that images the area including the ground onto which the visible light is projected; and a communication device T1 that transmits the captured image information. The display device D1E in the remote control room RC displays the image information received by the communication device T2. Therefore, the remote operator OP can grasp the situation around the work machine 100 using the image information which includes information indicating the direction away from the work machine 100. Thus, the remote control system SYS enables work that recognizes the sense of distance away from the work machine 100. Consequently, it can improve work efficiency.

[0168] Furthermore, in the SYS remote control system, there is no need to superimpose the detection results, which show the unevenness of the ground measured by LiDAR, onto the image information captured by the imaging device, as in conventional systems, thus reducing the computational load. Moreover, in the SYS remote control system, since no calculations are performed to superimpose the detection results onto the image information, computational delays can be suppressed.

[0169] Preferred embodiments and modifications of the present disclosure have been described above. However, the inventions of the present disclosure are not limited to the embodiments described above. Various modifications, substitutions, etc., can be applied to the embodiments described above without departing from the scope of the inventions of the present disclosure. Furthermore, each of the features described with reference to the embodiments described above may be combined as appropriate, as long as they do not conflict technically. [Explanation of Symbols]

[0170] 100 working machines 1. Lower running body 2. Swivel mechanism 3. Upper rotating body 4 Boom 5 Arms 6 buckets T1 Communication Device S6 imaging device S7 Indoor Imaging System S8 projection device 30 controllers 301 Acquisition Department 302 Generation part 303 Projection Control Unit 304 Receiving Control Unit 305 Drive Control Unit 306 Transmission Control Unit 307 Actuator drive unit 31 Proportional valve RC Remote Control Room T2 Communication Device D1E display device D2E Input Device R40 Remote Controller 401 Receiving Control Unit 402 Display Control Unit 403 Signal Generation Unit 404 Transmission Control Unit 405 Input Control Unit

Claims

1. A work machine including a projection device that projects visible light onto the ground, which contains information that can identify the direction away from the device; an imaging device that images the area including the ground onto which the visible light is projected; and a first communication device that transmits the captured image. A second communication device that receives the aforementioned image, A display device that displays the image received by the second communication device, A remote control system for work machinery equipped with the following features.

2. The projection device projects visible light onto the ground, in which grid lines capable of identifying the distance in the direction of separation are shown as the information. A remote control system for a work machine according to claim 1.

3. The aforementioned work machine further comprises an attachment, The projection device projects visible light onto the ground, indicating at least one of the following: the maximum area that a predetermined part of the attachment can reach, and the area that a predetermined part of the attachment can reach in its current orientation. A remote control system for a work machine according to claim 1.

4. The machine further comprises a detection unit capable of detecting people present around the machine, The projection device changes the projected visible light based on whether or not a person is detected by the detection unit. A remote control system for a work machine according to claim 1.

5. An input device that accepts input for setting at least one of the following: the color of the visible light projected by the projection device, the intensity of the visible light, and the direction in which the projection device projects the visible light. The projection device projects the visible light onto the ground according to the settings received as input. A remote control system for a work machine according to claim 1.

Images