Operation system of work machine, and remote operation device of work machine

Abstract

To improve work efficiency.SOLUTION: An operation system of a work machine includes: a work machine having a lower traveling body, an upper turning body which is turnably mounted on the lower traveling body, an attachment mounted on the upper turning body, and a work tool provided on the tip of the attachment; an operation device for operating the attachment; a detection unit for detecting a rotation angle of the attachment; and a display device for displaying information for recognizing correspondence between the current position of the attachment based on the rotation angle, and a range where the attachment is rotatable, on a region in a front direction from an operation seat.SELECTED DRAWING: Figure 4

Description

[Technical field]

[0001] The present invention relates to a work machine operation system and a work machine remote control device. [Background technology]

[0002] When working with a shovel, the work is usually performed from the viewpoint of an operator who is in the cabin. There is a technology that allows the operator to grasp the current shape of the ground and the shape of the work target on the target surface from the viewpoint of the operator (for example, Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] International Publication No. 2018 / 164172 Summary of the Invention [Problem to be solved by the invention]

[0004] However, the conventional technology of Patent Document 1 and the like is limited to grasping the shape of the work target from the operator's viewpoint, and is not a technology for grasping the status of the attachment. In other words, there is a problem that it is difficult to grasp the status of the attachment, such as the range of movement of the attachment, from the operator's viewpoint. For example, it is difficult for the operator to grasp the situation, such as whether the working tool attached to the tip of the attachment can be moved to the position intended by the operator, and therefore unnecessary attachment operation may occur. In this way, when it is difficult for the operator to grasp the status of the attachment, there is a possibility that the work efficiency will decrease.

[0005] One aspect of the present invention realizes improved work efficiency by enabling an operator to recognize the state of an attachment. [Means for solving the problem]

[0006] A work machine operating system according to one embodiment of the present invention comprises a work machine having a lower running body, an upper rotating body mounted on the lower running body so as to be freely rotatable, an attachment attached to the upper rotating body, and a work tool provided at the tip of the attachment, an operating device for operating the attachment, a detection unit for detecting the rotation angle of the attachment, and a display device in an area forward from the operator's seat for displaying information allowing a user to recognize the correspondence between the current position of the attachment based on the rotation angle and the range within which the attachment can rotate. Effect of the Invention

[0007] According to one aspect of the present invention, improvement of work efficiency is achieved. [Brief description of the drawings]

[0008] [Figure 1] 1 is a schematic diagram illustrating an example of a remote control system according to a first embodiment. [Diagram 2] FIG. 1 is a side view showing a shovel (excavator) according to a first embodiment. [Diagram 3] FIG. 2 is a block diagram showing a configuration example of a drive system of the shovel according to the first embodiment. [Figure 4] 1 is a functional block diagram illustrating a configuration example of a remote control system according to a first embodiment. [Diagram 5] FIG. 2 is a diagram showing an example of the layout of a remote control room according to the first embodiment. [Figure 6] 3A to 3C are diagrams illustrating an example of a screen displayed on the display device under the control of a display control unit according to the first embodiment. [Figure 7] 3A to 3C are diagrams illustrating an example of a screen displayed on the display device under the control of a display control unit according to the first embodiment. [Figure 8] 13A and 13B are diagrams illustrating an example of a screen displayed on a display device under the control of a display control unit according to a second embodiment. [Figure 9] 13A and 13B are diagrams illustrating AR glasses according to a modified example of the second embodiment. [Figure 10]FIG. 11 is a functional block diagram showing a configuration example of a shovel according to a third embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] Hereinafter, an embodiment of the present invention will be described with reference to the drawings. The embodiment described below is an example, not a limitation of the invention, and all features and combinations described in the embodiment are not necessarily essential to the invention. In addition, the same or corresponding components in each drawing are denoted by the same or corresponding reference numerals, and the description may be omitted.

[0010] In the following embodiment of the present invention, an example will be described in which a shovel shown below is used as an example of a work machine, however, the present invention is not limited to the shovel shown below and may be applied to, for example, a crane or the like.

[0011] (First embodiment) First, an overview of a remote control system (an example of a work machine operation system) SYS according to a first embodiment will be described with reference to Fig. 1. Fig. 1 is a schematic diagram showing an example of the remote control system SYS according to the first embodiment.

[0012] <Devices that make up the remote control system> As shown in FIG. 1, the remote operation system SYS according to the first embodiment includes a shovel 100 and a remote operation room RC.

[0013] The shovel 100 is capable of wireless communication. The shovel 100 is capable of transmitting and receiving data to and from a device (for example, a remote control room RC) connected to the communication line NW.

[0014] The excavator 100 exists, for example, at the work site. In this embodiment, the equipment existing at the work site is not limited to the excavator, and for example, one or more of a drone and a fixed-point camera may be provided. And the equipment existing at the work site (for example, the excavator 100) transmits information regarding the work site to the remote operation room RC. Thereby, the remote operation room RC can confirm the work site.

[0015] For example, the excavator 100 is provided with a space recognition device S6 (see FIG. 2). Therefore, the excavator 100 generates image information showing the measurement results of the periphery of the work site by the space recognition device S6, and transmits the generated image information to the remote operation room RC.

[0016] The excavator 100 included in the remote operation system SYS may be one or a plurality. Thereby, the remote operation system SYS can provide information regarding the work site to the remote operation room RC through a plurality of excavators 100.

[0017] <Configuration example of remote operation room> The remote operation room RC is provided with a communication device T2, a remote controller R30, an operation device R26, an operation sensor R29, an indoor imaging device RS1, and a display device RD1. Also, an operation seat DS on which an operator OP who remotely operates the excavator 100 sits is installed in the remote operation room RC.

[0018] The communication device T2 is connected to the communication line NW and communicates with the communication device T1 (see FIG. 2) attached to the excavator 100 and the like. The communication device T2 may include, for example, a mobile communication module compliant with a standard such as 4G (4th Generation) or 5G (5th Generation). Also, the communication device T2 may include, for example, a satellite communication module. Also, the communication device T2 may include, for example, a Wi-Fi (registered trademark) communication module or a Bluetooth (registered trademark) communication module.

[0019] The remote controller R30 is a calculation device that executes various calculations. In this embodiment, the remote controller R30 is configured with a microcomputer including a CPU and a memory. The various functions of the remote controller R30 are realized by the CPU executing a program stored in the memory.

[0020] The display device RD1 displays a screen based on information transmitted from the shovel 100 so that the operator OP in the remote control room RC can visually confirm the surroundings of the shovel 100. The display device RD1 enables the operator to confirm the status of the work site including the surroundings of the shovel 100 even though he is in the remote control room RC.

[0021] The operation device R26 (one example of an operation unit) is provided with an operation sensor R29 for detecting the operation content of the operation device R26. The operation sensor R29 is, for example, an inclination sensor that detects the inclination angle of the operation lever, or an angle sensor that detects the swing angle of the operation lever around the swing axis. The operation sensor R29 may be composed of other sensors such as a pressure sensor, a current sensor, a voltage sensor, or a distance sensor. The operation sensor R29 outputs information on the detected operation content of the operation device R26 to the remote controller R30. The remote controller R30 generates an operation signal based on the received information and transmits the generated operation signal to the excavator 100. The operation sensor R29 may be configured to generate an operation signal. In this case, the operation sensor R29 may output the operation signal to the communication device T2 without passing through the remote controller R30. This makes it possible to realize remote operation of the excavator 100 from the remote operation room RC.

[0022] The indoor imaging device RS1 is configured to capture an image of the inside of the remote control room RC. In this embodiment, the indoor imaging device RS1 is a camera installed inside the remote control room RC, and is configured to capture an image of an operator OP seated at an operation seat DS. The indoor imaging device RS1 transmits image information capturing an image of the face of the operator OP to the remote controller R30. This allows the remote controller R30 to recognize the line of sight of the operator OP.

[0023] [Outline of the excavator] First, an overview of a shovel 100 according to this embodiment will be described with reference to FIG.

[0024] FIG. 2 is a side view of the shovel 100 according to this embodiment.

[0025] The excavator 100 in this embodiment comprises a lower running body 1, an upper rotating body 3 mounted on the lower running body 1 so as to be freely rotatable via a rotating mechanism 2, a boom 4, an arm 5, and a bucket 6 as an attachment AT, and a cabin 10.

[0026] The lower traveling body 1 (an example of a traveling body) includes, for example, a pair of left and right crawlers, and the excavator 100 travels when the respective crawlers are hydraulically driven by traveling hydraulic motors 2ML, 2MR (see FIG. 3).

[0027] The upper rotating body 3 (an example of a rotating body) rotates relative to the lower traveling body 1 by being driven by a swing hydraulic motor 2A (see FIG. 3).

[0028] The attachment AT (an example of an attachment) is attached to the upper rotating body 3 and includes a boom 4, an arm 5, and a bucket 6.

[0029] A boom 4 is attached to the front center of the upper rotating body 3 so as to be able to tilt up and down, an arm 5 is attached to the tip of the boom 4 so as to be able to rotate up and down, and a bucket 6 is attached to the tip of the arm 5 so as to be able to rotate up and down.

[0030] The bucket 6 is an example of a working tool. The bucket 6 is used, for example, for excavation work. The bucket 6 according to this embodiment includes a toe 6a and a back surface 6b as portions for forming a horizontal surface.

[0031] Also, instead of the bucket 6, other working tools may be attached to the tip of the arm 5 depending on the work content, etc. The other working tools may be other types of buckets, such as a large bucket, a bucket for slopes, a bucket for dredging, etc. Also, the other working tools may be types of working tools other than a bucket, such as an agitator, a breaker, a grapple, etc.

[0032] The boom 4, the arm 5, and the bucket 6 are hydraulically driven by a boom cylinder 7, an arm cylinder 8, and a bucket cylinder 9, which serve as hydraulic actuators, respectively, with hydraulic oil discharged from a main pump 14 (see FIG. 3).

[0033] The cabin 10 is a cockpit where an operator sits, and is mounted on the front left side of the upper rotating body 3.

[0034] The shovel 100 may be configured such that some of the driven elements, such as the lower traveling body 1, the upper rotating body 3, the boom 4, the arm 5, and the bucket 6, are electrically driven. In other words, the shovel 100 may be a hybrid shovel, an electric shovel, or the like, in which some of the driven elements are driven by an electric actuator.

[0035] [Excavator configuration] Next, a specific configuration of the shovel 100 will be described with reference to FIG. 3 in addition to FIG.

[0036] FIG. 3 is a block diagram showing an example of the configuration of the shovel 100 according to this embodiment.

[0037] In the figure, mechanical power lines are indicated by double lines, high pressure hydraulic lines are indicated by solid lines, pilot lines are indicated by dashed lines, and electrical drive and control lines are indicated by dotted lines.

[0038] The hydraulic drive system that hydraulically drives the hydraulic actuators of the excavator 100 according to this embodiment includes the engine 11, the regulator 13, the main pump 14, and the control valve unit 17. As described above, the hydraulic drive system of the excavator 100 according to this embodiment also includes hydraulic actuators such as the traveling hydraulic motors 2ML, 2MR, the swing hydraulic motor 2A, the boom cylinder 7, the arm cylinder 8, and the bucket cylinder 9 that hydraulically drive the lower traveling structure 1, the upper rotating structure 3, the boom 4, the arm 5, and the bucket 6, respectively.

[0039] The engine 11 is a main power source in the hydraulic drive system, and is mounted, for example, on the rear of the upper rotating body 3. Specifically, the engine 11 rotates at a constant speed at a preset target speed under direct or indirect control by a controller 30 (described later), and drives the main pump 14 and the pilot pump 15. The engine 11 is, for example, a diesel engine that uses diesel as fuel.

[0040] The regulator 13 controls the discharge amount of the main pump 14. For example, the regulator 13 adjusts the angle (tilt angle) of the swash plate of the main pump 14 in response to a control command from the controller 30.

[0041] The main pump 14 (an example of a hydraulic pump) is mounted on the rear of the upper rotating body 3, for example, like the engine 11, and supplies hydraulic oil to the control valve unit 17 through a high-pressure hydraulic line 16. The main pump 14 is driven by the engine 11 as described above. The main pump 14 is, for example, a variable displacement hydraulic pump, and as described above, under the control of the controller 30, the tilt angle of the swash plate is adjusted by the regulator 13 to adjust the stroke length of the piston, thereby controlling the discharge flow rate (discharge pressure). The high-pressure hydraulic line 16 is a piping for converting the power from the engine 11 into hydraulic force by the main pump 14 and transmitting the hydraulic force to the control valve unit 17.

[0042] The control valve unit 17 is a hydraulic control device that controls the hydraulic system in the excavator 100. In this embodiment, the control valve unit 17 includes control valves 171 to 176. The control valve unit 17 is configured to selectively supply hydraulic oil discharged by the main pump 14 to one or more hydraulic actuators through the control valves 171 to 176. The control valves 171 to 176 control, for example, the flow rate of hydraulic oil flowing from the main pump 14 to the hydraulic actuators and the flow rate of hydraulic oil flowing from the hydraulic actuators to a hydraulic oil tank. The hydraulic actuators include a boom cylinder 7, an arm cylinder 8, a bucket cylinder 9, traveling hydraulic motors 2ML and 2MR, and a swing hydraulic motor 2A. More specifically, the control valve 171 corresponds to the left traveling hydraulic motor 2ML, the control valve 172 corresponds to the right traveling hydraulic motor 2MR, and the control valve 173 corresponds to the swing hydraulic motor 2A. Moreover, the control valve 174 corresponds to the bucket cylinder 9 , the control valve 175 corresponds to the boom cylinder 7 , and the control valve 176 corresponds to the arm cylinder 8 .

[0043] The pilot pump 15 is an example of a pilot pressure generating device, and is configured to be able to supply hydraulic oil to hydraulic control devices via a pilot line. In this embodiment, the pilot pump 15 is a fixed displacement hydraulic pump. However, the pilot pressure generating device may be realized by the main pump 14. That is, the main pump 14 may have a function of supplying hydraulic oil to various hydraulic control devices via a pilot line, in addition to a function of supplying hydraulic oil to the control valve unit 17 via a hydraulic oil line. In this case, the pilot pump 15 may be omitted.

[0044] The discharge pressure sensor 28 is configured to detect the discharge pressure of the main pump 14. In this embodiment, the discharge pressure sensor 28 outputs the detected value to the controller 30.

[0045] The operating device 26 is a device used by an operator to operate the actuator. The operating device 26 includes, for example, an operating lever and an operating pedal. The actuator includes at least one of a hydraulic actuator and an electric actuator.

[0046] The proportional valve 31, which functions as a control valve for machine control, is disposed in a pipe connecting the pilot pump 15 and the pilot port of the control valve in the control valve unit 17, and is configured so as to be able to change the flow passage area of ​​the pipe. In this embodiment, the proportional valve 31 operates in response to a control command output by the controller 30. Therefore, the controller 30 can supply the hydraulic oil discharged by the pilot pump 15 to the pilot port of the control valve in the control valve unit 17 via the proportional valve 31, regardless of the operation of the operating device 26 by the operator.

[0047] With this configuration, the controller 30 can operate the hydraulic actuator corresponding to a specific operating device 26 even when no operation is being performed on that specific operating device 26.

[0048] The control system of the shovel 100 according to this embodiment includes a controller 30, a display device D1, an input device D2, and a communication device T1. The control system of the shovel 100 also includes, as components related to the semi-automatic operation function, a proportional valve 31, a boom angle sensor S1, an arm angle sensor S2, a bucket angle sensor S3, a machine body inclination sensor S4, a swing angle sensor S5, a spatial recognition device S6, a positioning device PS, a boom rod pressure sensor S7R, a boom bottom pressure sensor S7B, an arm rod pressure sensor S8R, an arm bottom pressure sensor S8B, a bucket rod pressure sensor S9R, and a bucket bottom pressure sensor S9B. Hereinafter, the boom angle sensor S1, the arm angle sensor S2, and the bucket angle sensor S3 are also referred to as angle sensors S1 to S3. Hereinafter, the boom rod pressure sensor S7R, the boom bottom pressure sensor S7B, the arm rod pressure sensor S8R, the arm bottom pressure sensor S8B, the bucket rod pressure sensor S9R, and the bucket bottom pressure sensor S9B will also be referred to as cylinder pressure sensors S7 to S9.

[0049] The operation sensor 29 is configured to detect the operation content of the operator using the operating device 26. In this embodiment, the operation sensor 29 detects the operation direction and operation amount of the operating device 26 corresponding to each actuator, and outputs the detected value to the controller 30. In this embodiment, the controller 30 controls the opening area of ​​the proportional valve 31 according to the output of the operation sensor 29. Then, the controller 30 supplies the hydraulic oil discharged by the pilot pump 15 to the pilot port of the corresponding control valve in the control valve unit 17. The pressure of the hydraulic oil (pilot pressure) supplied to each pilot port is, in principle, a pressure according to the operation direction and operation amount of the operating device 26 corresponding to each hydraulic actuator. In this way, the operating device 26 is configured to be able to supply the hydraulic oil discharged by the pilot pump 15 to the pilot port of the corresponding control valve in the control valve unit 17.

[0050] The display device D1 is provided at a location where it can be easily viewed by an operator seated in the cabin 10, and displays various information images under the control of the controller 30. The display device D1 may be connected to the controller 30 via an in-vehicle communication network such as a Controller Area Network (CAN), or may be connected to the controller 30 via a one-to-one dedicated line.

[0051] The display device D1 is not limited to a device provided in advance in the cabin 10, and may be a monitor that can be installed separately. Furthermore, the display device D1 may be any device capable of displaying information, and may be, for example, a tablet terminal or the like that can communicate with the communication device T1.

[0052] The controller 30 (an example of a control device) is provided, for example, in the cabin 10 and controls the driving of the excavator 100. The functions of the controller 30 may be realized by any hardware, software, or a combination thereof. For example, the controller 30 is mainly configured with a microcomputer including a CPU (Central Processing Unit), a ROM (Read Only Memory), a RAM (Random Access Memory), a non-volatile auxiliary storage medium, various input / output interfaces, etc. The controller 30 realizes various functions by, for example, executing various programs stored in the ROM or the non-volatile auxiliary storage medium on the CPU.

[0053] For example, the controller 30 sets a target rotation speed based on an operation by an operator or the like, and performs drive control to rotate the engine 11 at a constant speed.

[0054] Furthermore, for example, the controller 30 outputs a control command to the regulator 13 as necessary to change the discharge amount of the main pump 14.

[0055] In addition, for example, the controller 30 controls the regulator 13 and adjusts the discharge volume of the main pump 14 based on the detection value of the pilot pressure corresponding to the operation state of various operating elements (i.e., various hydraulic actuators) in the operating device 26, which is input from the operation sensor 29.

[0056] Furthermore, for example, the controller 30 controls the regulator 13 and adjusts the discharge volume of the main pump 14 based on the detection value of the pilot pressure corresponding to the operating state of various operating elements (i.e., various hydraulic actuators) contained in the operation signal received from the remote control room RC via the communication device T1.

[0057] Also, for example, the controller 30 performs control related to a machine guidance function that guides (guides) the manual operation of the shovel 100 by the operator through the operation device 26 or the remote operation from the remote operation room RC. Also, the controller 30 performs control related to a machine control function that automatically supports, for example, the manual operation of the shovel 100 by the operator through the operation device 26 or the remote operation from the remote operation room RC.

[0058] Note that some of the functions of the controller 30 may be realized by another controller (control device). That is, the functions of the controller 30 may be realized in a distributed manner by a plurality of controllers. For example, the machine guidance function and the machine control function may be realized by a dedicated controller (control device).

[0059] More specifically, the controller 30 acquires information from the boom angle sensor S1, the arm angle sensor S2, the bucket angle sensor S3, the machine body inclination sensor S4, the turning angle sensor S5, the spatial recognition device S6, the cylinder pressure sensors S7, S8, S9, the communication device T1, the positioning device PS, etc. Furthermore, the controller 30 appropriately controls the proportional valve 31 in accordance with the acquired information, and can automatically adjust the pilot pressure acting on the control valve corresponding to the hydraulic actuator, thereby automatically operating each actuator.

[0060] The proportional valve 31 is provided in a pilot line connecting the pilot pump 15 and the pilot ports of the control valves 171 to 176, and is configured to be able to change its flow path area (cross-sectional area through which the hydraulic oil can flow). The proportional valve 31 operates in response to a control command input from the controller 30. As a result, even when the operating device 26 is not operated by an operator, the controller 30 can supply the hydraulic oil discharged by the pilot pump 15 to the pilot port of the corresponding control valve in the control valve unit 17 via the proportional valve 31. Then, the controller 30 can apply the pilot pressure generated by the proportional valve 31 to the pilot port of the corresponding control valve.

[0061] With this configuration, the controller 30 can operate the hydraulic actuator corresponding to a specific operating device 26 even when no operation is being performed on that specific operating device 26. Furthermore, the controller 30 can forcibly stop the operation of the hydraulic actuator corresponding to that specific operating device 26 even when an operation is being performed on that specific operating device 26.

[0062] The boom angle sensor S1 is attached to the boom 4 and detects the elevation angle of the boom 4 relative to the upper rotating body 3 (hereinafter referred to as the "boom angle"), for example, the angle formed by a straight line connecting the fulcrums at both ends of the boom 4 relative to the rotation plane of the upper rotating body 3 in a side view. The boom angle sensor S1 may include, for example, a rotary encoder, an acceleration sensor, a six-axis sensor, an IMU (Inertial Measurement Unit), etc. The boom angle sensor S1 may also include a potentiometer using a variable resistor, a cylinder stroke sensor that detects the stroke amount of a hydraulic cylinder (boom cylinder 7) corresponding to the boom angle, etc. The same applies to the arm angle sensor S2, bucket angle sensor S3, and machine body inclination sensor S4 below. A detection signal corresponding to the boom angle by the boom angle sensor S1 is taken into the controller 30.

[0063] The arm angle sensor S2 is attached to the arm 5 and detects the rotation angle of the arm 5 relative to the boom 4 (hereinafter referred to as the “arm angle”), for example, the angle formed by a line connecting the fulcrums at both ends of the arm 5 with respect to a line connecting the fulcrums at both ends of the boom 4 in a side view. A detection signal corresponding to the arm angle by the arm angle sensor S2 is input to the controller 30.

[0064] The bucket angle sensor S3 is attached to the bucket 6 and detects the rotation angle of the bucket 6 with respect to the arm 5 (hereinafter referred to as the “bucket angle”), for example, the angle formed by a line connecting the fulcrum of the bucket 6 and the tip (blade tip) with respect to a line connecting the fulcrums of both ends of the arm 5 in a side view. A detection signal corresponding to the bucket angle by the bucket angle sensor S3 is input to the controller 30.

[0065] The machine body inclination sensor S4 detects the inclination state of the machine body (upper rotating body 3 or lower running body 1) with respect to the horizontal plane. The machine body inclination sensor S4 is attached to, for example, the upper rotating body 3, and detects the inclination angles (hereinafter, "fore-aft inclination angle" and "left-right inclination angle") about two axes in the front-rear and left-right directions of the shovel 100 (i.e., upper rotating body 3). A detection signal corresponding to the inclination angle (fore-aft inclination angle and left-right inclination angle) by the machine body inclination sensor S4 is input to the controller 30.

[0066] The rotation angle sensor S5 outputs detection information related to the rotation state of the upper rotating body 3. The rotation angle sensor S5 detects, for example, the rotation angular velocity and the rotation angle of the upper rotating body 3. The rotation angle sensor S5 includes, for example, a gyro sensor, a resolver, a rotary encoder, etc.

[0067] The spatial recognition device S6 captures images of the surroundings of the shovel 100. The spatial recognition device S6 includes a camera S6F that captures an image in front of the shovel 100, a camera S6L that captures an image to the left of the shovel 100, a camera S6R that captures an image to the right of the shovel 100, and a camera S6B that captures an image behind the shovel 100.

[0068] Camera S6F is attached, for example, to the ceiling of the cabin 10, i.e., inside the cabin 10. Camera S6F may also be attached to the outside of the cabin 10, such as to the roof of the cabin 10 or the side of the boom 4. Camera S6L is attached to the left end of the upper surface of the upper rotating body 3, camera S6R is attached to the right end of the upper surface of the upper rotating body 3, and camera S6B is attached to the rear end of the upper surface of the upper rotating body 3.

[0069] The spatial recognition device S6 (cameras S6F, S6B, S6L, and S6R) are, for example, monocular wide-angle cameras having a very wide angle of view. The spatial recognition device S6 may also be a stereo camera or a distance image camera. Images captured by the spatial recognition device S6 are captured by the controller 30.

[0070] The spatial recognition device S6 may include a LIDAR (Light Detection and Ranging) to capture an image of a work site. The LIDAR measures the distance between the LIDAR and, for example, one million or more points within a monitoring range. Note that the spatial recognition device S6 according to the present embodiment is not limited to a method including LIDAR, and may include a spatial recognition device other than the LIDAR that can measure the distance between an object. As another example, a stereo camera may be used, or a distance measuring device such as a millimeter wave radar may be combined.

[0071] The boom rod pressure sensor S7R and the boom bottom pressure sensor S7B are respectively attached to the boom cylinder 7 and detect the pressure in the rod side oil chamber (hereinafter, "boom rod pressure") and the pressure in the bottom side oil chamber (hereinafter, "boom bottom pressure") of the boom cylinder 7. Detection signals corresponding to the boom rod pressure and the boom bottom pressure by the boom rod pressure sensor S7R and the boom bottom pressure sensor S7B are respectively input to the controller 30.

[0072] The arm rod pressure sensor S8R and the arm bottom pressure sensor S8B respectively detect the pressure in the rod side oil chamber (hereinafter "arm rod pressure") and the pressure in the bottom side oil chamber (hereinafter "arm bottom pressure") of the arm cylinder 8. Detection signals corresponding to the arm rod pressure and the arm bottom pressure by the arm rod pressure sensor S8R and the arm bottom pressure sensor S8B are respectively input to the controller 30.

[0073] The bucket rod pressure sensor S9R and the bucket bottom pressure sensor S9B respectively detect the pressure in a rod side oil chamber (hereinafter, “bucket rod pressure”) and the pressure in a bottom side oil chamber (hereinafter, “bucket bottom pressure”) of the bucket cylinder 9. Detection signals corresponding to the bucket rod pressure and the bucket bottom pressure by the bucket rod pressure sensor S9R and the bucket bottom pressure sensor S9B are respectively input to the controller 30.

[0074] The positioning device PS is configured to acquire information related to the position of the shovel 100. In this embodiment, the positioning device PS is configured to measure the position and orientation of the shovel 100. Specifically, the positioning device PS is a GNSS receiver incorporating an electronic compass, and measures the latitude, longitude, and altitude of the current position of the shovel 100, and measures the orientation of the shovel 100.

[0075] The communication device T1 communicates with external devices through a predetermined network including a mobile communication network with a base station as a terminal, a satellite communication network, the Internet, etc. The communication device T1 is, for example, a mobile communication module compatible with mobile communication standards such as LTE (Long Term Evolution), 4G (4th Generation), and 5G (5th Generation), or a satellite communication module for connecting to a satellite communication network.

[0076] In response to operation by an operator in the cabin 10, the shovel 100 operates actuators (e.g., hydraulic actuators) to drive operating elements (hereinafter referred to as "driven elements") such as the lower running body 1, upper rotating body 3, boom 4, arm 5, and bucket 6.

[0077] Further, instead of or in addition to being configured to be operable by an operator in the cabin 10, the shovel 100 is configured to be remotely operable from a remote control room RC. When the shovel 100 is remotely operated, the inside of the cabin 10 may be unmanned.

[0078] The shovel 100 may automatically operate the actuators regardless of the operation by the operator. This allows the shovel 100 to realize a function of automatically operating at least some of the driven elements such as the lower traveling body 1, the upper rotating body 3, the boom 4, the arm 5, and the bucket 6, i.e., a so-called "automatic driving function" or "machine control function."

[0079] The automatic operation function may include a function for automatically operating a driven element (actuator) other than the driven element (actuator) to be operated in response to an operator's operation of the operating device 26 or remote operation, i.e., a so-called "semi-automatic operation function" or "operation assistance type machine control function." The semi-automatic operation function, etc. may include a mode in which the operation content of the driven element (actuator) to be operated automatically is determined according to a rule that is specified in advance. The semi-automatic operation function, etc. may also include a mode in which the excavator 100 autonomously makes various judgments and autonomously determines the operation content of the driven element (hydraulic actuator) to be operated automatically in accordance with the judgment results (so-called "automatic operation function").

[0080] <Block configuration of remote control system> Fig. 4 is a functional block diagram showing a configuration example of the remote control system SYS according to this embodiment. In the example shown in Fig. 4, block configurations of the remote control room RC and the excavator 100 included in the remote control system SYS are shown.

[0081] The excavator 100 includes angle sensors S1 to S3, a spatial recognition device S6, cylinder pressure sensors S7 to S9, a positioning device PS, a controller 30, a proportional valve 31, and a communication device T1. The remote control room RC includes an operation sensor R29, a remote controller R30, a display device RD1, and a communication device T2.

[0082] The following describes functions of the controller 30 mounted on the shovel 100. As shown in FIG. 4, the controller 30 has an acquisition unit 301, a communication control unit 302, and an actuator driving unit 303 as functional blocks.

[0083] The acquisition unit 301 acquires the detection results of various sensors provided in the shovel 100.

[0084] For example, the acquisition unit 301 acquires image information captured by the spatial recognition device S6. The acquisition unit 301 also acquires a boom angle, an arm angle, and a bucket angle from the angle sensors S1 to S3. The acquisition unit 301 also acquires cylinder pressures from the cylinder pressure sensors S7 to S9. The acquisition unit 301 also acquires the position and orientation of the excavator 100 from the positioning device PS.

[0085] The communication control unit 302 transmits the detection results of the various sensors acquired by the acquisition unit 301 to the communication device T2 in the remote control room RC.

[0086] For example, the communication control unit 302 transmits image information representing the situation around the shovel 100, captured by the spatial recognition device S6, in order to notify the remote control room RC of the situation around the shovel 100. Furthermore, the communication control unit 302 transmits the boom angle, the arm angle, the bucket angle, the cylinder pressure, and the position and orientation of the shovel 100.

[0087] The communication control unit 302 receives an operation signal of the shovel 100 from the communication device T2 in the remote control room RC.

[0088] The actuator driving unit 303 is configured to drive an actuator mounted on the excavator 100. The actuator driving unit 303 generates and outputs an actuation signal for each of the plurality of solenoid valves included in the proportional valve 31 based on an operation signal input from the operation sensor 29. Furthermore, the actuator driving unit 303 generates and outputs an actuation signal for each of the plurality of solenoid valves included in the proportional valve 31 based on an operation signal received from the remote controller R30.

[0089] Each solenoid valve that receives the actuation signal increases or decreases the pilot pressure acting on the pilot port of the corresponding control valve in the control valve unit 17. As a result, the hydraulic actuator corresponding to each control valve operates at a speed according to the stroke amount of the control valve.

[0090] Next, a description will be given of functions of the remote controller R30 installed in the remote operation room RC. The remote controller R30 is configured to remotely operate the shovel 100. The remote controller R30 has, as functional blocks, a communication control unit 351, an acquisition unit 352, a generation unit 353, a determination unit 354, a superimposition unit 355, a display control unit 356, and an operation signal generation unit 357.

[0091] The communication control unit 351 receives detection results of various sensors attached to the shovel 100 from the communication device T1 of the shovel 100 via the communication device T2.

[0092] For example, the communication control unit 351 receives image information captured by the spatial recognition device S6 and showing the situation around the shovel 100. Furthermore, the communication control unit 351 receives the boom angle (an example of a rotation angle), the arm angle (an example of a rotation angle), the bucket angle (an example of a rotation angle), the cylinder pressure, and the position and orientation of the shovel 100.

[0093] The acquisition unit 352 acquires information about the situation inside the remote control room RC. For example, the acquisition unit 352 acquires image information captured by the indoor image capturing device RS1.

[0094] The generation unit 353 generates image information to be displayed on the display device RD1 based on the current state of the shovel 100 received by the communication control unit 351. The generation unit 353 according to this embodiment generates image information related to the position of the attachment AT being operated by the operating device R26 based on, for example, the boom angle, the arm angle, and the bucket angle. The generation unit 353 according to this embodiment generates shovel image information showing a side of the shovel 100 with the attachment AT shown in its current position as the image information related to the position of the attachment AT.

[0095] Furthermore, the generation unit 353 generates overhead image information that shows an overhead view of the periphery of the shovel 100 from above the shovel 100, based on image information captured by the spatial recognition device S6.

[0096] Furthermore, based on the detection results of the various sensors, the generation unit 353 generates at least one of character string information and image information indicating the current status of the shovel 100. For example, the generation unit 353 generates status information indicating the current status of the shovel 100. The status information will be described later.

[0097] The determination unit 354 determines whether or not to display various information based on the detection results of various sensors provided on the shovel 100. For example, the determination unit 354 determines whether or not the current rotation angle of each attachment AT is approaching within a predetermined angle (for example, 10 degrees) of the end of the rotational range. Note that the predetermined angle is not limited to 10 degrees and may be determined according to the embodiment. The predetermined angle may be greater than 10 degrees, or may be determined between 1 degree and 9 degrees.

[0098] The superimposing unit 355 superimposes at least one of the overhead image information and the status information on the image information of the surroundings of the shovel 100 captured by the spatial recognition device S6.

[0099] Furthermore, when the determination unit 354 determines that the current rotation angle of each attachment AT is approaching within a predetermined angle (e.g., 10 degrees) of the end of the rotation range, the superimposition unit 355 superimposes shovel image information showing the side of the shovel 100 (an example of information related to the position of the attachment AT) on image information captured by the spatial recognition device S6 of the situation around the shovel 100.

[0100] The display control unit 356 displays on the display device RD1 a screen in which one or more of the shovel image information, the overhead image information, the status information, and the character string information are superimposed on the image information showing the situation around the shovel 100. Furthermore, the display control unit 356 may vary the display mode based on the line of sight direction of the operator OP detected based on the image information captured by the indoor imaging device RS1. Specific display screens will be described later. The shovel image information, the overhead image information, the status information, and the character string information may have a predetermined transmittance. This allows the operator OP to check the situation around the shovel 100 via the shovel image information, the overhead image information, the status information, and the character string information.

[0101] The operation signal generating unit 357 is configured to generate an operation signal. The operation signal generating unit 357 according to the present embodiment is configured to generate an operation signal based on the output of the operation sensor R29. The operation signal is transmitted to the shovel 100 via the communication device T2.

[0102] Next, the remote control room RC will be described. Fig. 5 is a diagram showing an example of the layout of the remote control room RC. In the remote control room RC, a plurality of operating devices R26 are provided with respect to the operating seat DS.

[0103] In this embodiment, the display device RD1 is a large-screen display as shown in Fig. 5. The large-screen display may be realized as a multi-display configured of multiple monitors.

[0104] Then, the display device RD1 displays image information showing the surroundings of the shovel 100 under control of the display control unit 356. The operator OP can operate the shovel 100 via the operating device R26.

[0105] Fig. 6 is a diagram showing an example of a screen displayed on the display device RD1 under the control of the display control unit 356 according to this embodiment. In the example of the screen shown in Fig. 6, shovel image information 1601, overhead image information 1602, status information 1603, and distance information 1604 are superimposed on image information 1600 showing the situation around the shovel 100 captured by the camera S6F provided inside the cabin 10. The overhead image information 1602, status information 1603, and distance information 1604 shown in Fig. 6 may be made switchable between display and non-display in response to an operation by the operator.

[0106] 6 shows an example of a screen displayed when the determination unit 354 determines that the current rotation angle of the boom 4 is approaching within a predetermined angle (for example, 10 degrees) of the end of the rotatable range. In this embodiment, this is an example of a screen displayed on the display device RD1 in the forward direction from the operator seat DS, and shovel image information 1601 is displayed in an area visible when the operator OP operates the shovel 100.

[0107] The shovel image information 1601 is image information showing a side of the shovel 100. The shovel image information 1601 is displayed when the determination unit 354 determines that the current rotation angle of each of the attachments AT is approaching within a predetermined angle (e.g., 10 degrees) of the end of the rotatable range. Moreover, the display of the shovel image information 1601 is suppressed when the determination unit 354 determines that the current rotation angle of each of the attachments AT is farther than a predetermined angle (e.g., 10 degrees) from the end of the rotatable range.

[0108] The shovel image information 1601 shown in FIG. 6 indicates the current position of the attachment AT.

[0109] However, from the viewpoint inside the cabin 10, there is a problem that it is difficult for the operator OP to grasp the positions of the boom 4 and the arm 5 due to the ceiling of the cabin 10, etc. In this embodiment, therefore, when the determination unit 354 determines that the current rotation angle of each of the attachments AT is approaching within a predetermined angle (e.g., 10 degrees) of the end of the rotatable range, the shovel image information 1601 is displayed. When the operator OP refers to the shovel image information 1601, he or she can recognize whether the attachment AT is approaching the end of the rotatable range (also referred to as the stroke end, for example).

[0110] Furthermore, in the shovel image information 1601, for example, a display area 1611 representing the boom 4 is highlighted. This allows the operator OP to recognize the configuration of the attachment AT approaching the end of the rotatable range. Any method may be used to highlight the display area 1611 representing the boom 4, for example, a method of blinking the display may be used. In this embodiment, the display area 1611 representing the boom 4 is highlighted as information for allowing the operator to recognize that the current position of the boom 4 (an example of the attachment AT) based on the rotation angle is approaching the end of the rotatable range of the boom 4 (an example of the attachment). In other words, when the boom 4 is highlighted, the operator OP can recognize that the boom 4 is approaching the end of the rotatable range.

[0111] Furthermore, there is a method for indicating an end 1612 of a rotatable range in the shovel image information 1601. This allows the operator OP to recognize how much the attachment AT needs to be operated to reach the end of the rotatable range.

[0112] Furthermore, the remote controller R30 may output an alert sound from a speaker (not shown) provided in the remote control room RC.

[0113] In this embodiment, the operator OP can recognize the range within which the attachment AT can be rotated and perform work using the shovel 100, thereby improving work efficiency.

[0114] 6 may show a dotted line 1613 indicating a range in which the tip 6a of the bucket 6 can move. This allows the operator OP to recognize whether the work target is within the range in which the tip 6a of the bucket 6 can move. This allows the improvement of work efficiency.

[0115] The overhead image information 1602 is image information that is generated by the generating unit 353 and shows an overhead view of the periphery of the shovel 100 .

[0116] The status information 1603 is image information showing the current status of the shovel 100 based on the detection results of various sensors of the shovel 100 received by the communication control unit 351. The status information 1603 shows an example in which the current status of the shovel 100 is shown by a bar, but the current status may be shown by a character string or the like. The current status of the shovel 100 shown in the status information 1603 may be any information and may include, for example, the number of revolutions of the engine 11, the remaining amount of fuel stored in the fuel tank, the remaining amount of urea water stored in the urea water tank, the temperature state of the hydraulic oil in the hydraulic oil tank, the current temperature state of the engine coolant, and the like.

[0117] The distance information 1604 is character string information indicating the distance from the tip 6a of the bucket 6 to the ground. The position of the tip 6a of the bucket 6 is calculated by the generating unit 353 based on the boom angle, the arm angle, and the bucket angle. The method of identifying the position of the ground may be any method, regardless of whether it is a well-known method. The distance information 1604 indicates a distance calculated based on the position of the tip 6a of the bucket 6 and the position of the ground. The distance information 1604 includes a triangular icon 1604a indicating the position of the ground vertically downward from the tip 6a, and a dotted line 1604b connecting the icon 1604a to the tip 6a. The operator OP can recognize the distance from the tip 6a to the ground by referring to the distance information 1604. In this embodiment, the operator OP can grasp the distance between the ground and the tip 6a of the bucket 6, thereby improving convenience.

[0118] Fig. 7 is a diagram showing an example of a screen displayed on the display device RD1 under the control of the display control unit 356 according to this embodiment. In the example of the screen shown in Fig. 7, overhead image information 1602, status information 1603, load factor information 1701, and an arrow icon 1702 are superimposed on image information 1700 showing the situation around the excavator 100 captured by the camera S6F provided inside the cabin 10. The overhead image information 1602 and the status information 1603 are the same as those in Fig. 6. The load factor information 1701 and the arrow icon 1702 shown in Fig. 7 may be made switchable between display and non-display in response to an operation by the operator.

[0119] In the example shown in Fig. 7, it is assumed that the determination unit 354 has determined that the current rotation angle of each attachment AT is not approaching within a predetermined angle (for example, 10 degrees) of the end of the rotatable range. Therefore, in the example screen shown in Fig. 7, the shovel image information 1601 is not displayed.

[0120] In the example shown in Fig. 7, the excavator 100 is shaping a slope with the bucket 6. In this case, the remote controller R30 determines the direction of the reaction force acting on the tip 6a of the bucket 6 due to the slope shaping, based on the moving direction of the bucket 6. For example, the direction of the reaction force is the opposite direction to the moving direction. Furthermore, the remote controller R30 determines the magnitude of the reaction force acting on the tip 6a of the bucket 6, based on the detection results of the cylinder pressure sensors (an example of a load detection unit) S7 to S9.

[0121] Then, the generating unit 353 generates an arrow icon 1702 indicating a vector (direction and magnitude) of a reaction force acting on the tip (one example of a predetermined part) 6a of the bucket 6. Then, the display control unit 356 displays the arrow icon 1702 indicating the vector of the reaction force superimposed on the image information 1700. The arrow icon 1702 is displayed so as to indicate the vector of the reaction force with the tip 6a of the bucket 6 as a reference. The superimposing unit 355 may specify the position of the tip 6a of the bucket 6 shown in the image information 1700 by, for example, image processing, and superimpose the arrow icon 1702 on the specified position. In this embodiment, the operator OP can recognize the reaction force acting on the tip 6a of the bucket 6 by referring to the arrow icon 1702. Then, the operator OP can adjust the operation of the attachment AT according to the reaction force. In this way, in this embodiment, the occurrence of an overload on the bucket 6 can be suppressed by the operation of the operator OP, and therefore the occurrence of an abnormality in the shovel 100 can be suppressed, thereby improving reliability.

[0122] The determination unit 354 determines whether or not the load factor of the engine 11 is equal to or greater than a predetermined threshold value (e.g., 80%) based on the detection results of various sensors provided in the shovel 100. When the determination unit 354 determines that the load factor of the engine 11 is equal to or greater than the predetermined threshold value (e.g., 80%), load factor information 1701 is displayed. The load factor information 1701 is information illustrating an example of the load state of the engine 11 (an example of a drive source) of the shovel 100. In this manner, in this embodiment, the operator OP can grasp the load on the engine 11 by referring to the load factor information 1701. Therefore, since the occurrence of an overload on the engine 11 can be suppressed by the operation of the operator OP, the occurrence of an abnormality in the shovel 100 can be suppressed, and improvement of reliability can be achieved.

[0123] Note that this embodiment is not limited to displaying the load factor of the engine 11, and may display any information related to the load applied to the excavator 100. For example, information related to the relief state of the main pump 14 by the regulator 13 (e.g., information indicating the relief state with a numerical value of 0 to 100%) may be displayed. Furthermore, information related to the thrust of each of the boom cylinder 7, arm cylinder 8, and bucket cylinder 9 provided on the attachment AT may be displayed. Furthermore, the specific force (strain) applied to the attachment AT may be displayed. The display mode of this information may be any mode, and may be displayed as a numerical value or a percentage, or may be displayed with a gauge or the like.

[0124] Furthermore, the display control unit 356 may change the display mode of the screen depending on the situation of the operator OP. For example, when the determination unit 354 determines that the operator OP is looking at the overhead image information 1602 or the status information 1603 based on the line of sight of the operator OP, the display control unit 356 may perform a display in which the overhead image information 1602 or the status information 1603 looked by the operator OP is enlarged. This makes it easier for the operator OP to grasp the contents displayed in the overhead image information 1602 or the status information 1603, thereby improving convenience.

[0125] In this embodiment, the display control unit 356 displays a screen in which shovel image information is superimposed on image information showing the situation around the shovel 100. This allows the operator to simultaneously recognize the situation around the shovel 100 and the position of the attachment AT of the shovel 100, enabling operation according to the current situation, thereby improving work efficiency.

[0126] Second Embodiment In the above-described embodiment, an example has been described in which shovel image information showing the side of the shovel 100 is displayed as information relating to the position of the attachment AT operated by the operating device R26. However, the above-described embodiment does not limit the display of information relating to the position of the attachment AT operated by the operating device R26 to shovel image information. Therefore, in the second embodiment, a case will be described in which the end of the rotatable range of the attachment AT is displayed as an example of information relating to the position of the attachment AT.

[0127] FIG. 8 is a diagram showing an example of a screen displayed on the display device RD1 under the control of the display control unit 356 according to this embodiment. In the example of the screen shown in FIG. 8, first end information 1801, second end information 1802, overhead image information 1602, status information 1603, and distance information 1604 are superimposed on image information 1800 showing the situation around the excavator 100 captured by the camera S6F provided inside the cabin 10. The overhead image information 1602, status information 1603, and distance information 1604 are the same as those in FIG. 6, and therefore will not be described. In this embodiment, as in the above-mentioned embodiment, this is an example of a screen displayed on the display device RD1 in the forward direction from the operating seat DS, and the first end information 1801 and the second end information 1802 are displayed in an area visible when the operator OP operates the excavator 100.

[0128] FIG. 8 is an example of a screen that is displayed when the determination unit 354 determines that the current rotation angle of the boom 4 is approaching within a predetermined angle (e.g., 10 degrees) of the end of the rotatable range.

[0129] 8 is image information for enabling recognition of the rotatable range of the boom 4, and indicates the end of the rotatable range of a side 1811 of the boom 4. The first end information 1801 is highlighted, for example by blinking. In this embodiment, the first end information 1801 is displayed in a display area in which the current boom 4 is displayed. By visually recognizing the boom 4 displayed in the image information 1800 and the first end information 1801, the operator OP can recognize the correspondence between the current position of the boom 4 and the rotatable range of the boom 4.

[0130] Furthermore, FIG. 8 is an example of a screen that is displayed when the determination unit 354 determines that the current rotation angle of the arm 5 is approaching within a predetermined angle (e.g., 10 degrees) of the end of the rotatable range.

[0131] 8 is image information for enabling recognition of the rotatable range of the arm 5, and indicates the end of the rotatable range of a side 1812 of the arm 5. The second end information 1802 is highlighted, for example by blinking. In this embodiment, the second end information 1802 is displayed in a display area in which the current arm 5 is displayed. By visually checking the arm 5 displayed in the image information 1800 and the second end information 1802, the operator OP can recognize the correspondence between the current position of the arm 5 and the rotatable range of the boom 4.

[0132] Furthermore, the remote controller R30 may output an alert sound from a speaker (not shown) provided in the remote control room RC.

[0133] The operator can perform work after intuitively recognizing the rotation range of the attachment AT by referring to the first end information 1801 and the second end information 1802. Therefore, the improvement of work efficiency is realized.

[0134] (Modification of the second embodiment) In the above-described embodiment, an example has been described in which the first end portion information 1801 and the second end portion information 1802 are superimposed and displayed on the display device RD1 on the image information 1700 captured by the camera S6F. However, the above-described embodiment is not limited to the method of superimposing and displaying on the display device RD1. Therefore, in the modified example, a case in which the operator OP wears AR glasses will be described.

[0135] Fig. 9 is a diagram illustrating an example of AR glasses 1900 according to a modified example of the second embodiment. The AR glasses 1900 shown in Fig. 9 are connected to a remote controller R30 so as to be able to transmit and receive information. A wireless communication device (not shown) for communicating with the AR glasses 1900 is connected to the remote controller R30.

[0136] The AR glasses 1900 are glasses-type wearable terminals that are worn on the head of an operator OP who operates the shovel 100. That is, in this modification, when an operator at an operator's seat wears the AR glasses 1900, an image is displayed on the AR glasses 1900 that are located in front of the operator's seat.

[0137] The AR glasses 1900 include a left lens portion 1902L and a right lens portion 1902R provided on a frame member 1901. The left lens portion 1902L and the right lens portion 1902R are optically transparent. Therefore, the operator OP can see the actual scene (for example, a screen displayed on the display device RD1) through the left lens portion 1902L and the right lens portion 1902R.

[0138] Furthermore, the AR glasses 1900 include a control unit 1911, a wireless communication unit 1912, an image display unit 1913, and cameras 1914R and 1914L.

[0139] The control unit 1911 controls the entire AR glasses 1900.

[0140] The wireless communication unit 1912 performs short-range wireless communication with the remote controller R30.

[0141] Furthermore, image display units 1913 are provided on both the left and right sides of the AR glasses 1900. The image display units 1913 output images to the left lens unit 1902L and the right lens unit 1902R, so that the images can be displayed on each of the left lens unit 1902L and the right lens unit 1902R.

[0142] Therefore, the operator OP can see a video in which the image output by the image display unit 1913 is superimposed on the actual scene that can be seen through each of the left lens unit 1902L and the right lens unit 1902R.

[0143] Cameras 1914L and 1914R are provided in the AR glasses 1900. From the images captured by the cameras 1914L and 1914R, the actual scenes captured in the left lens portion 1902L and the right lens portion 1902R can be identified.

[0144] Then, the control unit 1911 transmits image information captured by the cameras 1914L and 1914R to the remote controller R30. The remote controller R30 can recognize the range of the actual scene captured in each of the left lens unit 1902L and the right lens unit 1902R based on the received image information.

[0145] When the determination unit 354 determines that the current rotation angle of each of the attachments AT is approaching within a predetermined angle (e.g., 10 degrees) of the end of the rotatable range, the generation unit 353 identifies the position of the attachment AT shown in each of the left lens unit 1902L and the right lens unit 1902R based on the received image information, and generates end information based on the identified position of the attachment AT. Then, the communication control unit 351 transmits the generated end information to the wireless communication unit 1912 of the AR glasses 1900.

[0146] Then, the image display unit 1913 displays the end information (e.g., end information 1801, 1802) received by the wireless communication unit 1912 on each of the left lens unit 1902L and the right lens unit 1902R. In this modification, the end information is generated based on the position of the attachment AT. That is, the operator OP can visually recognize the end of the rotatable range of the attachment AT.

[0147] The AR glasses 1900 according to this modification superimposes and displays end information on an actual scene that the operator OP can refer to via the left lens unit 1902L and the right lens unit 1902R (for example, an example of the situation around the excavator 100 displayed on the display device RD1). Furthermore, the AR glasses 1900 superimposes and displays overhead image information 1602, status information 1603, distance information 1604, and the like on an actual scene that the operator OP can refer to via the left lens unit 1902L and the right lens unit 1902R. Therefore, the remote operation system SYS according to this modification can present to the operator OP an image substantially similar to that shown in FIG. 8 of the above-mentioned embodiment.

[0148] In this modification, the remote controller R30 displays image information showing the surroundings of the shovel 100 captured by the camera S6F on the display device RD1, and displays end information on the AR glasses 1900. Therefore, the remote controller R30 does not need to perform image processing for superimposing the end information when displaying the image information captured by the camera S6F on the display device RD1. Therefore, the delay when displaying the image information captured by the camera S6F on the display device RD1 is reduced. Therefore, the operator OP can perform the work without being aware of the discomfort caused by the delay.

[0149] In the present embodiment, an example has been described in which end information is superimposed on a real scene and displayed on the AR glasses 1900. However, the present embodiment is not limited to a method of displaying end information on the AR glasses 1900, and it is sufficient if information relating to the position of the attachment AT can be displayed. For example, the shovel image information described in the first embodiment may be displayed.

[0150] (Third embodiment) In the above-described embodiment and modified examples, a case has been described in which information is displayed to the operator OP in the remote control room RC of the shovel 100. However, the above-described embodiment and modified examples are not limited to cases in which information is displayed to the operator OP in the remote control room RC of the shovel 100. Therefore, in this embodiment, a case in which information is displayed to an operator aboard the cabin 10 of the shovel 100 will be described.

[0151] That is, in this embodiment, information is displayed on the AR glasses 1900 located in front of the operator's seat in the cabin 10.

[0152] <Excavator block configuration> Fig. 10 is a functional block diagram showing a configuration example of the shovel 100 according to this embodiment. In the example shown in Fig. 4, block configurations of the remote operation room RC and the shovel 100 included in the remote operation system SYS are shown.

[0153] In this embodiment, an operator who boards the cabin 10 wears the AR glasses 1900. A wireless communication device (not shown) for wirelessly communicating with the AR glasses 1900 is connected to the controller 30A in the cabin 10.

[0154] This allows the controller 30A of the shovel 100 to transmit image information to be displayed on the AR glasses 1900.

[0155] The following describes the functions of the controller 30A mounted on the shovel 100. As shown in Fig. 10, the controller 30A has, as functional blocks, an acquisition unit 301, a generation unit 311, a determination unit 312, a communication control unit 313, and an actuator driving unit 303. Note that the same components as those in the above-mentioned embodiment are assigned the same reference numerals, and descriptions thereof will be omitted.

[0156] The communication control unit 313 transmits and receives information to and from the AR glasses 1900. For example, the communication control unit 313 receives image information captured by the cameras 1914L and 1914R of the AR glasses 1900.

[0157] The generation unit 311 generates image information to be displayed on the AR glasses 1900 based on the current state of the shovel 100 acquired by the acquisition unit 301 and the image information received by the communication control unit 313.

[0158] The generating unit 311 according to this embodiment generates image information relating to the position of the attachment AT being operated by the operating device R26, for example, based on the boom angle, the arm angle, and the bucket angle. The generating unit 311 according to this embodiment generates end information indicating the end of the rotatable range of each attachment AT, as image information relating to the position of the attachment AT.

[0159] The generating unit 311 according to this embodiment identifies the position of the attachment AT appearing in the received image information, identifies the end positions of the rotatable range of the attachment AT in each of the left lens unit 1902L and the right lens unit 1902R, and generates end information showing end lines at the identified positions. Similarly to the above-mentioned embodiment, the generating unit 311 generates overhead image information, status information, distance information, and the like.

[0160] The determination unit 312 determines whether or not to display various information based on the detection results of various sensors provided in the shovel 100. For example, the determination unit 312 determines whether or not the current rotation angle of each attachment AT is approaching within a predetermined angle (e.g., 10 degrees) of the end of the rotatable range.

[0161] The communication control unit 313 transmits the image information etc. generated by the generation unit 311 to the AR glasses 1900 based on the determination result by the determination unit 312.

[0162] The AR glasses 1900 display the received edge information superimposed on the actual scene captured through the left lens unit 1902L and the right lens unit 1902R. Furthermore, the AR glasses 1900 display the received edge information, distance information, and the like superimposed on the actual scene captured through the left lens unit 1902L and the right lens unit 1902R. This allows the operator to view, for example, an image as shown in FIG. 8.

[0163] Furthermore, a display device D1 is attached to the cabin 10. The determination unit 312 determines whether or not the operator is referring to the display device D1 based on the image information received from the AR glasses 1900. When the determination unit 312 determines that the operator is referring to the display device D1, the generation unit 311 generates image information that enlarges the screen displayed on the display device D1. Then, the communication control unit 313 transmits the enlarged screen information to the AR glasses 1900.

[0164] As a result, the screen displayed on the display device D1 is displayed in an enlarged state on the left lens portion 1902L and the right lens portion 1902R. Therefore, the operator can refer to the enlarged screen of the display device D1. The screen displayed on the display device D1 may include, for example, overhead image information and status information.

[0165] The shovel 100 according to this embodiment performs the above-mentioned display when the attachment AT approaches the end of the movable range, thereby enabling the operator to recognize that the attachment AT is approaching the end of the movable range.

[0166] As another example, the controller 30A may output an alert sound from a speaker (not shown) provided in the cabin 10 when the attachment AT approaches the end of the movable range.

[0167] As in this embodiment, when an operator is present in the cabin 10, if the arm 5 reaches the end of its movable range during the opening operation, or if the bucket 6 reaches the end of its movable range during the opening operation, the shovel 100 vibrates and generates a loud noise. In this case, there is a possibility that the operator aboard the shovel 100 may also be subjected to the vibrations.

[0168] In contrast to this, in this embodiment, the operator can recognize the status of the attachment AT by displaying end information on the AR glasses 1900, and can therefore stop the operation of the attachment AT before it reaches the end of the movable range. Therefore, in this embodiment, the burden on the operator can be reduced.

[0169] Furthermore, in this embodiment, the AR glasses 1900 can display the edge information superimposed on the actual scene without the controller 30A performing a process of superimposing the edge information on the image information showing the periphery of the shovel 100. Therefore, compared to the above-mentioned embodiment, the controller 30A can suppress delays due to the process of superimposing images and reduce the processing load.

[0170] In the above-described embodiment and modified example, a remote control system for a shovel and a case where the system is applied to a shovel have been described as an example of a work machine operation system. However, the work machine operation system is not limited to a remote control system for a shovel or a shovel, and may be any system for operating a work machine.

[0171] In the above-described embodiment and modified examples, a case has been described in which the shovel 100 is used as the work machine. However, the embodiment and modified examples do not limit the work machine to the shovel 100, and the work machine may be a crawler crane, a forklift, or an overhead crane that can be moved according to the operation of an operator, or may be a fixed power crane. Furthermore, the work machine may be a road machine including an asphalt finisher or the like.

[0172] <effect> The operating system for operating the shovel 100 according to the above-described embodiment and modified example displays information regarding the position of the attachment AT, which is difficult for the operator to see, so that the operator can recognize the current state of the attachment AT. Therefore, the operator can stop the operation of the attachment AT before it reaches the end of the rotatable range. Therefore, the operator can perform work while recognizing the current status of the attachment AT, thereby improving work efficiency.

[0173] Although the embodiments of the work machine operation system and work machine remote control device according to the present invention have been described above, the present invention is not limited to the above-mentioned embodiments. Various changes, modifications, substitutions, additions, deletions, and combinations are possible within the scope of the claims. Naturally, these also fall within the technical scope of the present invention. [Explanation of symbols]

[0174] 100 Shovel S1 Boom Angle Sensor S2 Arm Angle Sensor S3 Bucket Angle Sensor S6 spatial recognition device S7, S8, S9 Cylinder pressure sensors T1, T2 communication equipment 1 Undercarriage 2 Swivel mechanism 3. Upper rotating body 4. Boom 5 Arm 6 Bucket 7 Boom cylinder 8 Arm Cylinder 9 Bucket Cylinder 11 Engine PS Positioning Device 26 Operating device 29 Operation Sensor 30, 30A Controller 301 Acquisition Department 302 Communication Control Unit 303 Actuator drive unit 311 Generation part 312 Judgment section 313 Communication Control Unit 31 Proportional valve RC Remote Control Room R26 operating device R29 Operation Sensor R30 Remote Controller 351 Communication Control Unit 352 Acquisition Department 353 Generation part 354 Judgment section 355 Overlap 356 Display control section 357 Operation signal generation section R26 operating device R29 Operation Sensor RS1 Indoor Imaging Device RD1 display device 1900 AR Glasses

Claims

1. A work machine including a lower traveling body, an upper rotating body rotatably mounted on the lower traveling body, an attachment attached to the upper rotating body, and a working tool provided at a tip of the attachment; An operating device for operating the attachment; A detection unit that detects a rotation angle of the attachment; a display device that displays, in a region forward from the operator's seat, information for allowing the operator to recognize a correspondence between a current position of the attachment based on the rotation angle and a range in which the attachment can be rotated; A work machine operation system comprising:

2. the display device is a glasses-type wearable terminal that is worn on the head of an operator at the operation seat, the information for allowing the operator to recognize a correspondence between the current position of the attachment based on the rotation angle and a range in which the attachment can be rotated is superimposed and displayed on the area showing the situation around the work machine, which can be viewed by the operator through a lens unit provided on the wearable terminal; 2. The operating system of claim 1 for a work machine.

3. the work machine further includes an imaging device that images an image of the surroundings of the work machine, and a first communication device that transmits image information captured by the imaging device and information indicating the rotation angle, a second communication device that receives the image information and the rotation angle from the first communication device; The display device displays the image information received by the second communication device, and displays the information for recognizing a correspondence between a current position of the attachment based on the rotation angle received by the second communication device and a range in which the attachment can be rotated, superimposed on the image information.

3. The operating system for a work machine according to claim 2.

4. The display device displays the information indicating an end of a range in which the attachment can be rotated in a display area in which the current attachment is displayed.

4. An operating system for a work machine according to claim 2 or 3.

5. the display device displays image information of the work machine showing the current position of the attachment; 2. The operating system of claim 1 for a work machine.

6. The display device displays a distance from the ground to a predetermined portion of the attachment, or a load factor of an engine of the work machine.

4. An operating system for a work machine according to claim 2 or 3.

7. When the rotation angle of the attachment is farther than a predetermined angle from an end of a rotatable range, the display device suppresses the display of the information for allowing the user to recognize a correspondence between a current position of the attachment and a rotatable range of the attachment.

2. The operating system of claim 1 for a work machine.

8. an operation unit that generates an operation signal for operating a work machine having a lower traveling body, an upper rotating body that is rotatably mounted on the lower traveling body, an attachment attached to the upper rotating body, a working tool provided at the tip of the attachment, an imaging device that images the surroundings of the work machine, a detection unit that detects the rotation angle of the attachment, and a first communication device that transmits image information captured by the imaging device and information indicating the rotation angle; a communication unit that receives the image information and the rotation angle from the first communication device and transmits the operation signal to the first communication device; a display device that displays the image information received by the communication unit in an area forward from the operating seat, and displays information superimposed on the image information for allowing the user to recognize a correspondence between a current position of the attachment based on the rotation angle and a range in which the attachment can be rotated; A remote control device for a work machine comprising:

Images