Remote operation support system, remote operation support device

Abstract

To provide a technique capable of further improving the safety of a work machine when it is remotely operated.SOLUTION: A remote operation support system SYS according to one embodiment of the present disclosure comprises: an imaging device 40 for obtaining data concerning an object in the vicinity of a work machine 100; a communication device 60 for communicating with a remote operation support device 200; a communication device 220 for communicating with the work machine 100; a remote operation device 240A for receiving an input for remotely operating the work machine 100; a controller 30 for detecting an object to be monitored in the vicinity of the work machine 100 on the basis of an output by the imaging device 40; and an output device 230 for notifying a user who uses the remote operation device 240A of the fact that the object to be monitored is detected in a notification range in the vicinity of the work machine 100. If a communication delay between the work machine 100 and the remove operation support device 200 is relatively large, the remote operation support system SYS deforms an outer edge of the notification range more outward than when the communication delay is relatively small.SELECTED DRAWING: Figure 8

Description

[Technical field]

[0001] The present disclosure relates to a remote operation support system for a work machine. [Background technology]

[0002] 2. Description of the Related Art Conventionally, techniques relating to remote control of a work machine are known (see, for example, Patent Document 1).

[0003] In Patent Document 1, an imaging device and a distance sensor are installed on a work machine (shovel), and when a monitored object such as a person (hereinafter referred to as a "monitored object") is detected within a specified range around the work machine from the output of these, an alert is sent to an operator of an external device that remotely controls the work machine. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] International Publication No. 2020 / 218455 Summary of the Invention [Problem to be solved by the invention]

[0005] However, a communication delay may occur between the work machine and the external device. Therefore, if the communication delay becomes relatively large, the time lag between when an image capture device, a distance sensor, or the like captures the entry of a person or the like to be monitored into a predetermined range and when an operator or the like of the external device that remotely operates the work machine is notified may become relatively large. As a result, when an operator or the like of the external device is notified, the distance between the person or the like to be monitored and the work machine may become even closer. Therefore, there is room for improvement in terms of the safety of the excavator.

[0006] In view of the above problems, an object of the present invention is to provide a technique that can further improve the safety of a work machine when remote operation is performed. [Means for solving the problem]

[0007] In order to achieve the above object, in one embodiment of the present disclosure, An acquisition device provided on a work machine for acquiring data relating to objects in the vicinity of the work machine; a work machine communication device that is provided in the work machine and communicates with an external device that assists in remote operation of the work machine; an external device communication device provided in the external device and configured to communicate with the work machine; a remote operation device provided in the external device and configured to receive an input for remotely operating the work machine; an object detection device that is provided on the work machine or the external device and detects a predetermined object around the work machine based on an output of the acquisition device; a notification device that is provided in the external device and that, when the predetermined object is detected by the object detection device within a predetermined range based on the work machine, notifies a user of the remote control device of that fact, when a communication delay between the work machine and the external device is relatively large, the outer edge of the predetermined range is changed in a direction away from the work machine compared to when the communication delay is relatively small. A remote operation support system is provided.

[0008] In another embodiment of the present disclosure, a communication device that communicates with a work machine having an acquisition device that acquires data related to objects around the work machine; a remote control device that receives an input for remotely controlling the work machine; a notification device that notifies a user of the remote control device when a predetermined object is detected within a predetermined range based on the work machine based on the output of the acquisition device, when a communication delay between the work machine and the work machine is relatively large, the outer edge of the predetermined range is changed in a direction away from the work machine compared to when the communication delay is relatively small. A remote operation support device is provided. Effect of the Invention

[0009] According to the above-described embodiment, it is possible to further improve the safety of a work machine when remote control is performed. [Brief description of the drawings]

[0010] [Figure 1] FIG. 1 is a schematic diagram illustrating an example of a remote operation support system. [Diagram 2] FIG. 2 is a side view showing an example of a shovel. [Diagram 3] FIG. 2 is a top view showing an example of a shovel. [Figure 4] FIG. 1 is a diagram illustrating an example of a crawler crane. [Diagram 5] FIG. 1 illustrates an example of a configuration of a remote operation support system. [Figure 6] FIG. 1 is a diagram illustrating an example of a configuration of a shovel. [Figure 7] FIG. 1 is a diagram illustrating an example of a configuration of a crawler crane. [Figure 8] 4 is a flowchart illustrating a first example of a process for setting a notification range by a control device. [Figure 9] 10 is a flowchart illustrating a second example of a process for setting a notification range by a control device. [Figure 10] 10 is a flowchart illustrating a third example of a process for setting a notification range by a control device. [Figure 11] FIG. 13 is a diagram showing an example of an image of the shovel's surroundings displayed on the display device. [Figure 12] FIG. 2 is a diagram showing an example of an image of the surroundings of a crawler crane displayed on a display device. [Figure 13] FIG. 11 is a diagram showing another example of the image of the shovel's surroundings displayed on the display device. [Figure 14] FIG. 13 is a diagram showing another example of the image of the vicinity of the crawler crane displayed on the display device. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] Hereinafter, an embodiment will be described with reference to the drawings.

[0012] [Outline of the remote operation support system] First, an overview of a remote operation support system SYS according to this embodiment will be described with reference to FIGS.

[0013] Fig. 1 is a schematic diagram showing an example of a remote operation support system SYS according to this embodiment. Figs. 2 to 4 are diagrams showing a specific example of a work machine 100 included in the remote operation support system SYS. Specifically, Figs. 2 and 3 are a side view and a top view showing an example of an excavator 100A, and Fig. 4 is a diagram showing an example of a crawler crane 100B. In the following description, corresponding components of the excavator 100A and the crawler crane 100B (for example, components having the same function) are assigned the same reference numerals.

[0014] As shown in FIG. 1, the remote operation support system SYS includes a work machine 100 and a remote operation support device 200.

[0015] The remote operation support system SYS may include one or more work machines 100. Furthermore, when the remote operation support system SYS includes multiple work machines 100, the multiple work machines 100 may all be of the same type, or at least some of them may be of different types. For example, the remote operation support system SYS may include multiple units of either one of the shovels 100A and crawler cranes 100B described below, or may include both the shovels 100A and the crawler cranes 100B.

[0016] Furthermore, the remote operation support system SYS may include a plurality of remote operation support devices 200. That is, the plurality of remote operation support devices 200 may execute processing related to the remote operation support system SYS in a distributed manner. For example, each of the plurality of remote operation support devices 200 may communicate with a portion of the work machines 100 that it is responsible for among all the work machines 100 included in the remote operation support system SYS, and execute processing targeted at the portion of the work machines 100.

[0017] The remote operation support system SYS supports the remote operation of the work machine 100 in a remote operation support device 200.

[0018] Furthermore, as described below, when the work machine 100 performs work by fully automatic driving, the remote operation support system SYS may, for example, support remote monitoring of work performed by the fully automatic driving of the work machine 100 in the remote operation support device 200. In this case, the remote operation support system SYS may support the operation of the work machine 100 in the remote operation support device 200 by a monitor's intervention in the fully automatic driving function.

[0019] <Overview of the work machine> 2 to 4, the work machine 100 includes a lower traveling body 1, an upper rotating body 3 rotatably mounted on the lower traveling body 1 via a rotating mechanism 2, an attachment AT attached to the upper rotating body 3, and a cabin 10 for an operator to ride in. Hereinafter, the front of the work machine 100 (upper rotating body 3) corresponds to the direction in which the attachment for the upper rotating body 3 extends when the excavator 100A is viewed in a plan view (top view) from directly above along the rotation axis of the upper rotating body 3.

[0020] The work machine 100 may be a machine other than the excavator 100A or the crawler crane 100B. For example, the work machine 100 may be a forklift, a lifting magnet machine having a lifting magnet attached as an end attachment, a bulldozer, a wheel loader, an asphalt finisher, a forestry machine, or the like.

[0021] The work machine 100 can perform various types of work by operating at least one of the lower traveling body 1, the upper rotating body 3, and the attachment AT.

[0022] Furthermore, the work machine 100 is equipped with a communication device 60, and can communicate with the remote operation support device 200 via a predetermined communication line NW. This allows the work machine 100 to transmit (upload) various types of information to the remote operation support device 200 and receive various types of signals (for example, information signals and control signals) from the remote operation support device 200.

[0023] The communication line NW includes, for example, a wide area network (WAN). The wide area network may include, for example, a mobile communication network with a base station as a terminal. The wide area network may also include, for example, a satellite communication network that uses a communication satellite above the work machine 100. The wide area network may also include, for example, an Internet network. The communication line NW may also include, for example, a local network (LAN) of a facility in which the remote operation support device 200 is installed. The local network may be a wireless line, a wired line, or a line including both. The communication line NW may also include, for example, a short-distance communication line based on a predetermined wireless communication method such as WiFi or Bluetooth (registered trademark).

[0024] The work machine 100 is configured to be capable of being remotely operated from the remote operation support device 200. When the work machine 100 is remotely operated, the inside of the cabin 10 may be unmanned.

[0025] Specifically, the work machine 100 is operated by input from a user (operator) regarding the actuator of the work machine 100 performed by the remote operation support device 200. In this case, the work machine 100 transmits data of an image (hereinafter, "peripheral image") showing the surroundings of the work machine 100, such as an image captured by the imaging devices 40, 42 described below and a processed image (e.g., a viewpoint conversion image) based on the captured image, to an external device. The peripheral image is then displayed on a display device 230A described below that is provided in the remote operation support device 200. In addition, various information images (information screens) displayed on the output device 50 (display device) in the cabin 10 of the work machine 100 may also be displayed on the display device 230A of the remote operation support device 200. This allows the operator to remotely operate the work machine 100 while checking the display contents, such as the peripheral image showing the surroundings of the work machine 100 and various information images displayed on the display device 230A. The work machine 100 may operate actuators and drive driven elements such as the lower running body 1, the upper rotating body 3, and the attachment AT in response to a remote control signal indicating the content of the remote control received from the remote control support device 200.

[0026] Furthermore, the work machine 100 may operate an actuator (for example, a hydraulic actuator) in response to operation by an operator inside the cabin 10 to drive driven elements such as the lower traveling body 1, the upper rotating body 3, and the attachment AT. In the following, the explanation will be given on the premise that the operation by the operator includes both operation of the operating device 26 by the operator inside the cabin 10 and remote operation by an external operator.

[0027] Furthermore, the work machine 100 may automatically operate the actuators regardless of the contents of the operation by the operator. This allows the work machine 100 to realize a function for automatically operating at least some of the driven elements such as the lower traveling body 1, the upper rotating body 3, the attachment AT, etc., that is, a so-called "automatic driving function" or "machine control (MC) function."

[0028] The automatic driving function may include a function of automatically operating a driven element (actuator) other than the driven element (actuator) to be operated in response to the operation of the operator, i.e., a so-called "semi-automatic driving function" or "operation support type MC function". The automatic driving function may also include a function of automatically operating at least a part of a plurality of driven elements (hydraulic actuators) on the premise that there is no operation by the operator, i.e., a so-called "fully automatic driving function" or "fully automatic type MC function". When the fully automatic driving function is enabled in the work machine 100, the inside of the cabin 10 may be unmanned. The semi-automatic driving function, the fully automatic driving function, etc. may also include a mode in which the operation content of the driven element (actuator) to be the target of the automatic driving is automatically determined according to a rule that is defined in advance. The semi-automatic driving function, the fully automatic driving function, etc. may also include a so-called "autonomous driving function" in which the work machine 100 autonomously makes various judgments and autonomously determines the operation content of the driven element (hydraulic actuator) to be the target of the automatic driving in accordance with the judgment results.

[0029] The work machine 100 is, for example, a shovel 100A.

[0030] As shown in Figures 2 and 3, the excavator 100A comprises a lower running body 1, an upper rotating body 3 mounted on the lower running body 1 so as to be freely rotatable via a rotating mechanism 2, an attachment AT for performing various tasks, and a cabin 10.

[0031] The lower traveling structure 1 includes, for example, a pair of left and right crawlers 1C. The lower traveling structure 1 causes the excavator 100A to travel by hydraulically driving each of the crawlers 1C by a left traveling hydraulic motor 1ML and a right traveling hydraulic motor 1MR.

[0032] The upper rotating body 3 rotates relative to the lower traveling body 1 as a result of the rotating mechanism 2 being hydraulically driven by the rotating hydraulic motor 2M.

[0033] The attachment AT includes a boom 4A, an arm 5A, and a bucket 6A as driven elements.

[0034] The boom 4A is attached to the front center of the upper rotating body 3 so as to be able to be tilted up and down, an arm 5A is attached to the tip of the boom 4A so as to be able to rotate up and down, and a bucket 6A is attached to the tip of the arm 5A so as to be able to rotate up and down.

[0035] The bucket 6A is an example of an end attachment. The bucket 6A is used, for example, for excavation work. In addition, instead of the bucket 6A, another end attachment may be attached to the tip of the arm 5A depending on the work content, etc. The other end attachment may be, for example, another type of bucket such as a large bucket, a slope bucket, or a dredging bucket. The other end attachment may be a type of end attachment other than a bucket such as an agitator, a breaker, or a grapple. In addition, a spare attachment such as, for example, a quick coupling or a tiltrotator may be interposed between the arm 5A and the end attachment.

[0036] A hook HK for crane work may be attached to the bucket 6A. The hook HK has a base end rotatably connected to a bucket pin that connects the arm 5A and the bucket 6A. As a result, the hook HK is stored in the space formed between the two bucket links when work other than crane work (hoisting work), such as excavation work, is performed.

[0037] The boom 4A, the arm 5A, and the bucket 6A are hydraulically driven by a boom cylinder 7A, an arm cylinder 8A, and a bucket cylinder 9A, which serve as hydraulic actuators, respectively.

[0038] In addition, in the shovel 100A, some or all of the various hydraulic actuators may be replaced with electric actuators, that is, the shovel 100A may be a hybrid shovel or an electric shovel.

[0039] The cabin 10 is a cockpit where an operator sits, and is mounted, for example, on the front left side of the upper rotating body 3.

[0040] In addition, in the case where an operator does not board and operate the excavator 100A, and the work machine 100 operates mainly by remote control or a fully automatic driving function, the cabin 10 may be omitted. The same applies to the case of the crawler crane 100B below.

[0041] Moreover, the work machine 100 is, for example, a crawler crane 100B.

[0042] As shown in FIG. 4, the crawler crane 100B comprises a lower running body 1, an upper rotating body 3 mounted on the lower running body 1 so as to be freely rotatable via a rotating mechanism 2, an attachment AT, a mast 5B, a backstop 6B, a hook HK, a counterweight 9B, and a cabin 10.

[0043] The lower traveling body 1 includes, for example, a pair of left and right crawlers 1C. The lower traveling body 1 causes the crawler crane 100B to travel by hydraulically driving each of the crawlers 1C by a left traveling hydraulic motor 1ML and a right traveling hydraulic motor 1MR (see FIG. 7).

[0044] The upper rotating body 3 rotates relative to the lower traveling body 1 as the rotating mechanism 2 is hydraulically driven by a rotating hydraulic motor 2M (see FIG. 7).

[0045] The attachment AT (an example of a working device) includes a boom 4B and a main hoisting rope 7B.

[0046] The boom 4B is movably attached to the center of the front of the upper rotating body 3. A main hoisting rope 7B hangs down from the tip of the boom 4B, and a hook HK is attached to the tip of the main hoisting rope 7B. That is, the hook HK is attached to the tip of the boom 4B via the main hoisting rope 7B.

[0047] The main hoisting rope 7B has its base end attached to a main hoisting winch 7Ba that is attached to the rear portion between the base end and tip of the boom 4B, and its tip is attached to the hook HK. The main hoisting rope 7B can be wound up and unwound by the main hoisting winch 7Ba that is hydraulically driven by a main hoisting hydraulic motor 7M (see FIG. 7), thereby raising and lowering the hook HK.

[0048] The mast 5B is attached to the upper rotating body 3 slightly rearward of the base end of the boom 4B so as to be rotatable about a rotation axis parallel to the rotation axis of the boom 4B. The tip of the mast 5B is connected to the tip of the boom 4B via a pendant rope 5Ba, and the boom 4B is raised or lowered via the mast 5B by winding in and out the boom hoist rope 5Bb using a boom hoist winch 5Bc hydraulically driven by a hoist hydraulic motor 5M (see FIG. 7).

[0049] Incidentally, in the crawler crane 100B, similarly to the case of the excavator 100A, some or all of the various hydraulic actuators may be replaced with electric actuators.

[0050] The backstop 6B has a base end attached to a portion of the upper rotating body 3 rearward of the base end of the boom 4B so as to be rotatable about a rotation axis parallel to the rotation axis of the boom 4B, and a tip end attached to a rear portion between the base end and tip end of the boom 4B so as to be rotatable about a rotation axis parallel to the rotation axis of the boom 4B. The backstop 6B expands and contracts in response to the raising and lowering operation of the boom 4B, and has a function of supporting the boom 4B from the rear when the boom 4B is in a substantially upright state, for example.

[0051] The hook HK is attached to the end of the main hoisting rope 7B and is used to suspend a load.

[0052] The counterweight 9B is provided at the rear end of the upper rotating body 3 and has the function of balancing the weight of the boom 4B and the suspended load SL.

[0053] The cabin 10 is attached, for example, to the right front end portion of the upper rotating body 3. Inside the cabin 10, a cockpit and an operating device 26 (see FIG. 7) for operating various actuators are provided.

[0054] <Outline of remote operation support device> The remote operation support device 200 (an example of an external device) supports the remote operation of the work machine 100. In addition, the remote operation support device 200 may support the remote monitoring of the work machine 100 performing work in a fully automatic manner.

[0055] The remote operation support device 200 may be, for example, an on-premise server or a cloud server installed in a management center or the like outside the work site where the work machine 100 performs work. The remote operation support device 200 may also be, for example, an edge server installed in the work site where the work machine 100 performs work, or in a location relatively close to the work site. The remote operation support device 200 may also be a stationary terminal device or a portable terminal device (mobile terminal) installed in a management office or the like in the work site of the work machine 100. The stationary terminal device may include, for example, a desktop computer terminal. The portable terminal device may include, for example, a smartphone, a tablet terminal, a laptop computer terminal, or the like.

[0056] The remote operation support device 200 has, for example, a communication device 220 (see Figs. 5 to 7), and as described above, communicates with the work machine 100 via the communication line NW. This allows the remote operation support device 200 to receive various information uploaded from the work machine 100 and transmit various signals to the work machine 100. Therefore, the user of the remote operation support device 200 can check various information related to the work machine 100 through the output device 230 (see Figs. 2 and 3). In addition, the remote operation support device 200 can, for example, transmit an information signal to the work machine 100 to provide information required for work, or transmit a control signal to control the work machine 100. The user of the remote operation support device 200 may include, for example, the owner of the work machine 100, the manager of the work machine 100, an engineer of the manufacturer of the work machine 100, the operator of the work machine 100, the manager, supervisor, worker, etc. of the work site of the work machine 100.

[0057] For example, the remote operation support device 200 has a remote operation device 240A through which the operator makes inputs for remote operation, and a display device 230A that displays an image (peripheral image) of the surroundings of the work machine 100, etc. A signal input from the remote operation device 240A is transmitted from the remote operation support device 200 to the work machine 100 as a remote operation signal. This allows a user (operator) of the remote operation support device 200 to remotely operate the work machine 100 using the remote operation device 240A while checking the surroundings of the work machine 100 on the display device 230A.

[0058] For example, a user (monitor) of the remote operation support device 200 can monitor the state of work of the work machine 100 on the display device 230A. Also, for example, when an abnormality occurs in the work machine 100 or when the operation of the work machine 100 is inappropriate, the user (monitor) of the remote operation support device 200 can use the remote operation device 240A to perform an intervention operation on the automatic driving function of the work machine 100. Therefore, the monitor can use the remote operation device 240A to perform an emergency stop of the work machine 100 or to perform a remote operation that causes the work machine 100 to perform an appropriate operation.

[0059] [Configuration of remote operation support system] Next, the configuration of the remote operation support system SYS will be described with reference to FIGS.

[0060] Fig. 5 is a block diagram showing an example of the configuration of the remote operation support system SYS according to this embodiment. Fig. 6 is a diagram showing an example of the configuration of a shovel 100A. Specifically, Fig. 6 is a diagram showing a specific example of the configuration of the shovel 100A corresponding to Fig. 5. Fig. 7 is a diagram showing an example of the configuration of a crawler crane 100B. Specifically, Fig. 7 is a diagram showing a specific example of the configuration of the crawler crane 100B corresponding to Fig. 5.

[0061] In Figures 5 to 7, paths through which mechanical power is transmitted are indicated by double lines, paths through which high-pressure hydraulic oil that drives the hydraulic actuator flows are indicated by solid lines, paths through which pilot pressure is transmitted are indicated by dashed lines, and paths through which electrical signals are transmitted are indicated by dotted lines.

[0062] <Work machine configuration> The work machine 100 includes various components, such as a hydraulic drive system for hydraulically driving the driven elements, an operation system for operating the driven elements, a user interface system for exchanging information with the user, a communication system for communicating with the outside, and a control system for various controls.

[0063] <<Hydraulic drive system>> 5, the hydraulic drive system of the work machine 100 according to this embodiment includes the hydraulic actuator HA that hydraulically drives each of the driven elements such as the lower traveling body 1 (left and right crawlers 1C), the upper rotating body 3, and the attachment AT, as described above. The hydraulic drive system of the work machine 100 according to this embodiment also includes an engine 11, a regulator 13, a main pump 14, and a control valve 17.

[0064] As shown in FIG. 6, when the work machine 100 is an excavator 100A, the hydraulic actuator HA includes traveling hydraulic motors 1ML, 1MR, a swing hydraulic motor 2M, a boom cylinder 7A, an arm cylinder 8A, and a bucket cylinder 9A.

[0065] Furthermore, as shown in FIG. 7, when the work machine 100 is a crawler crane 100B, the hydraulic actuator HA includes traveling hydraulic motors 1ML, 1MR, a swing hydraulic motor 2M, a hoisting hydraulic motor 5M, and a main hoist hydraulic motor 7M.

[0066] The engine 11 is a prime mover of the work machine 100 and a main power source in the hydraulic drive system. The engine 11 is, for example, a diesel engine that uses light oil as fuel. The engine 11 is mounted, for example, on the rear of the upper rotating body 3. The engine 11 rotates at a constant speed at a preset target speed under direct or indirect control by a controller 30 (described later), and drives the main pump 14 and the pilot pump 15.

[0067] Incidentally, instead of or in addition to the engine 11, the work machine 100 may be equipped with another prime mover (for example, an electric motor) or the like.

[0068] The regulator 13 controls (adjusts) the discharge amount of the main pump 14 under the control of the controller 30. For example, the regulator 13 adjusts the angle of the swash plate of the main pump 14 (hereinafter, the “tilt angle”) in response to a control command from the controller 30.

[0069] The main pump 14 supplies hydraulic oil to the control valve 17 through a high-pressure hydraulic line. The main pump 14 is mounted, for example, on the rear of the upper rotating body 3, similar to the engine 11. As described above, the main pump 14 is driven by the engine 11. The main pump 14 is, for example, a variable displacement hydraulic pump, and as described above, under the control of the controller 30, the tilt angle of the swash plate is adjusted by the regulator 13 to adjust the stroke length of the piston, thereby controlling the discharge flow rate (discharge pressure).

[0070] The control valve 17 is a hydraulic control device that controls the hydraulic actuators HA in response to the contents of the operator's operation of the operating device 26 or the remote operation, or in response to an operation command related to the automatic operation function output from the controller 30. The operation command corresponding to the automatic operation function may be generated by the controller 30, or may be generated by another control device (arithmetic device) that performs control related to the automatic operation function. The control valve 17 is mounted, for example, in the center of the upper rotating body 3. As described above, the control valve 17 is connected to the main pump 14 via a high-pressure hydraulic line, and selectively supplies hydraulic oil supplied from the main pump 14 to each hydraulic actuator in response to the operator's operation or an operation command corresponding to the automatic operation function. Specifically, the control valve 17 includes a plurality of control valves (also referred to as "directional control valves") that control the flow rate and flow direction of hydraulic oil supplied from the main pump 14 to each hydraulic actuator HA.

[0071] <<Operation system>> As shown in FIG. 5, the operating system of the work machine 100 according to this embodiment includes a pilot pump 15, an operating device 26, a hydraulic control valve 31, a shuttle valve 32, and a hydraulic control valve 33.

[0072] The pilot pump 15 supplies pilot pressure to various hydraulic devices via a pilot line 25. The pilot pump 15 is mounted on the rear of the upper rotating body 3, for example, similar to the engine 11. The pilot pump 15 is, for example, a fixed displacement hydraulic pump, and is driven by the engine 11 as described above.

[0073] The pilot pump 15 may be omitted. In this case, the relatively high pressure hydraulic oil discharged from the main pump 14 is reduced in pressure by a predetermined pressure reducing valve, and the resulting relatively low pressure hydraulic oil is supplied to various hydraulic devices as pilot pressure.

[0074] The operating device 26 is provided near the cockpit of the cabin 10, and is used by the operator to operate the various driven elements. In other words, the operating device 26 is used by the operator to operate the hydraulic actuators HA that drive the respective driven elements. The operating device 26 includes a pedal device and a lever device for operating the respective driven elements (hydraulic actuators HA).

[0075] For example, as shown in Fig. 4, the operating device 26 is of a hydraulic pilot type. Specifically, the operating device 26 uses hydraulic oil supplied from the pilot pump 15 through a pilot line 25 and a pilot line 25A branched therefrom, and outputs a pilot pressure corresponding to the operation content to a secondary pilot line 27A. The pilot line 27A is connected to one inlet port of a shuttle valve 32, and is connected to the control valve 17 through a pilot line 27 connected to an outlet port of the shuttle valve 32. Thus, pilot pressure corresponding to the operation content related to various driven elements (hydraulic actuators) in the operating device 26 can be input to the control valve 17 through the shuttle valve 32. Therefore, the control valve 17 can drive each hydraulic actuator HA according to the operation content of the operating device 26 by an operator or the like.

[0076] The operating device 26 may be of an electric type. In this case, the pilot line 27A, the shuttle valve 32, and the hydraulic control valve 33 are omitted. Specifically, the operating device 26 outputs an electric signal (hereinafter, "operation signal") corresponding to the operation content, and the operation signal is taken in by the controller 30. The controller 30 then outputs a control command corresponding to the content of the operation signal, that is, a control signal corresponding to the operation content for the operating device 26, to the hydraulic control valve 31. As a result, a pilot pressure corresponding to the operation content of the operating device 26 is input from the hydraulic control valve 31 to the control valve 17, and the control valve 17 can drive each hydraulic actuator HA according to the operation content of the operating device 26. In addition, the control valves (directional control valves) built into the control valve 17 for driving each hydraulic actuator may be of an electromagnetic solenoid type. In this case, the operation signal output from the operating device 26 may be directly input to the control valve 17, that is, to the electromagnetic solenoid type control valve. In addition, when the work machine 100 is remotely operated or operates by an automatic driving function, the operating device 26 may be omitted.

[0077] The hydraulic control valve 31 is provided for each driven element (hydraulic actuator HA) to be operated by the operating device 26. The hydraulic control valve 31 may be provided, for example, in the pilot line 25B between the pilot pump 15 and the control valve 17, and configured to be able to change its flow path area (i.e., a cross-sectional area through which the hydraulic oil can flow). This allows the hydraulic control valve 31 to output a predetermined pilot pressure to the secondary pilot line 27B using the hydraulic oil of the pilot pump 15 supplied through the pilot line 25B. Therefore, as shown in FIG. 4, the hydraulic control valve 31 can indirectly apply a predetermined pilot pressure corresponding to a control signal from the controller 30 to the control valve 17 through the shuttle valve 32 between the pilot line 27B and the pilot line 27. Also, as shown in FIG. 5, the hydraulic control valve 31 can directly apply a predetermined pilot pressure corresponding to a control signal from the controller 30 to the control valve 17 through the pilot line 27B and the pilot line 27. Therefore, the controller 30 can supply pilot pressure to the control valve 17 from the hydraulic control valve 31 according to the operation of the electric operating device 26, thereby realizing operation of the work machine 100 based on the operation of the operator.

[0078] Furthermore, the controller 30 may, for example, control the hydraulic control valve 31 to realize an automatic driving function. Specifically, the controller 30 outputs a control signal corresponding to an operation command related to the automatic driving function to the hydraulic control valve 31, regardless of whether the operating device 26 is operated or not. As a result, the controller 30 can cause the hydraulic control valve 31 to supply a pilot pressure corresponding to the operation command related to the automatic driving function to the control valve 17, thereby realizing the operation of the work machine 100 based on the automatic driving function.

[0079] Furthermore, the controller 30 may, for example, control the hydraulic control valve 31 to realize remote operation of the work machine 100. Specifically, the controller 30 outputs a control signal corresponding to the content of remote operation specified in a remote operation signal received from the remote operation support device 200 via the communication device 60 to the hydraulic control valve 31. As a result, the controller 30 can supply a pilot pressure corresponding to the content of the remote operation from the hydraulic control valve 31 to the control valve 17, thereby realizing the operation of the work machine 100 based on the remote operation by the operator.

[0080] The shuttle valve 32 has two inlet ports and one outlet port, and outputs hydraulic oil having a higher pilot pressure of the two pilot pressures input to the two inlet ports to the outlet port. The shuttle valve 32 is provided for each driven element (hydraulic actuator HA) to be operated by the operating device 26. One of the two inlet ports of the shuttle valve 32 is connected to a pilot line 27A on the secondary side of the operating device 26 (specifically, the above-mentioned lever device or pedal device included in the operating device 26), and the other is connected to a pilot line 27B on the secondary side of the hydraulic control valve 31. The outlet port of the shuttle valve 32 is connected to a pilot port of a corresponding control valve of the control valve 17 through the pilot line 27. The corresponding control valve is a control valve that drives a hydraulic actuator that is the operating target of the above-mentioned lever device or pedal device connected to one inlet port of the shuttle valve 32. Therefore, each of these shuttle valves 32 can apply the higher of the pilot pressure in the pilot line 27A on the secondary side of the operating device 26 and the pilot pressure in the pilot line 27B on the secondary side of the hydraulic control valve 31 to the pilot port of the corresponding control valve. In other words, the controller 30 can control the corresponding control valve regardless of the operation of the operating device 26 by the operator by outputting a pilot pressure higher than the pilot pressure on the secondary side of the operating device 26 from the hydraulic control valve 31. Therefore, the controller 30 can control the operation of the driven elements (the lower traveling body 1, the upper rotating body 3, and the attachment AT) regardless of the operating state of the operating device 26 by the operator, thereby realizing an automatic driving function.

[0081] The hydraulic control valve 33 is provided in the pilot line 27A that connects the operating device 26 and the shuttle valve 32. The hydraulic control valve 33 is configured to be able to change its flow path area, for example. The hydraulic control valve 33 operates in response to a control signal input from the controller 30. As a result, the controller 30 can forcibly reduce the pilot pressure output from the operating device 26 when the operating device 26 is operated by the operator. Therefore, even when the operating device 26 is being operated, the controller 30 can forcibly suppress or stop the operation of the hydraulic actuator corresponding to the operation of the operating device 26. In addition, for example, even when the operating device 26 is being operated, the controller 30 can reduce the pilot pressure output from the operating device 26 to make it lower than the pilot pressure output from the hydraulic control valve 31. Therefore, the controller 30 can reliably apply a desired pilot pressure to the pilot port of the control valve in the control valve 17, for example, regardless of the operation content of the operating device 26, by controlling the hydraulic control valve 31 and the hydraulic control valve 33. Therefore, for example, by controlling the hydraulic control valve 33 in addition to the hydraulic control valve 31, the controller 30 can more appropriately realize the automatic driving function and remote control function of the work machine 100.

[0082] <<User interface system>> As shown in FIG. 5, the user interface system of the work machine 100 according to this embodiment includes an operation device 26, an output device 50, and an input device 52.

[0083] The output device 50 outputs various types of information to a user of the work machine 100 (for example, an operator of the cabin 10, a work vehicle in the vicinity of the work machine 100, or the like).

[0084] For example, the output device 50 includes lighting equipment, a display device, etc. that output various types of information in a visual manner. The lighting equipment is, for example, a warning light, etc. The display device is, for example, a liquid crystal display, an organic EL (Electroluminescence) display, etc. The lighting equipment and the display device may be provided, for example, inside the cabin 10, and output various types of information to an operator, etc. inside the cabin 10 in a visual manner. The lighting equipment and the display device may also be provided, for example, on a side surface of the upper rotating body 3, and output various types of information to workers, etc. around the work machine 100 in a visual manner.

[0085] Furthermore, for example, the output device 50 includes a sound output device that outputs various information by an auditory method. The sound output device includes, for example, a buzzer, a speaker, etc. The sound output device may be provided, for example, inside the cabin 10, and may output various information to an operator, etc., inside the cabin 10 by an auditory method.

[0086] Also, for example, the output device 50 may include a device that outputs various information in a tactile manner, such as by vibration of the cockpit.

[0087] The input device 52 receives various inputs from a user of the work machine 100, and a signal corresponding to the received input is taken into the controller 30. The input device 52 is provided, for example, inside the cabin 10, and receives inputs from an operator or the like inside the cabin 10. The input device 52 may also be provided, for example, on the side surface of the upper rotating body 3, and receives inputs from a worker or the like in the vicinity of the work machine 100.

[0088] For example, the input device 52 is an operation input device that accepts operation input. The operation input device may include a touch panel mounted on the display device, a touch pad installed around the display device, a button switch, a lever, a toggle, a knob switch provided on the operation device 26 (lever device), and the like.

[0089] Furthermore, for example, the input device 52 may be a voice input device that accepts voice input from a user. The voice input device includes, for example, a microphone.

[0090] Furthermore, for example, the input device 52 may be a gesture input device that accepts a gesture input by a user. The gesture input device includes, for example, an imaging device that captures an image of a gesture made by the user.

[0091] <<Communications>> As shown in FIG. 5, the communication system of the work machine 100 according to this embodiment includes a communication device 60.

[0092] The communication device 60 (an example of a communication device for a work device) is connected to the communication line NW and communicates with a device (for example, a remote operation support device 200) provided separately from the work machine 100. The device provided separately from the work machine 100 may include a device external to the work machine 100, as well as a portable terminal device brought into the cabin 10 by the user of the work machine 100. The communication device 60 may be, for example, 4G (4 th Generation) and 5G (5 th The communication device 60 may include a mobile communication module conforming to a standard such as IEEE 802.11 Generation. The communication device 60 may also include a satellite communication module. The communication device 60 may also include a WiFi communication module, a Bluetooth (registered trademark) communication module, or the like.

[0093] <<Control system>> 5, the control system of the work machine 100 according to this embodiment includes a controller 30. The control system of the work machine 100 according to this embodiment also includes an operating pressure sensor 29, image capturing devices 40, 42, and an acquisition device SX.

[0094] The controller 30 (an example of an object detection device) performs various controls related to the work machine 100. The functions of the controller 30 may be realized by any hardware or any combination of hardware and software. For example, the controller 30 is configured mainly with a computer including a CPU (Central Processing Unit), a memory device such as RAM (Random Access Memory), a non-volatile auxiliary storage device such as ROM (Read Only Memory), and various input / output interface devices.

[0095] The controller 30 includes, as functional units, an operation control unit 301, an image transmission unit 302, an object detection unit 303, and a safety control unit 304. The functions of the operation control unit 301, the image transmission unit 302, the object detection unit 303, and the safety control unit 304 are realized, for example, by loading a program installed in an auxiliary storage device into a memory device and executing the program on a CPU.

[0096] Note that some of the functions of the controller 30 may be realized by another controller (control device). That is, the functions of the controller 30 may be realized in a distributed manner by a plurality of controllers.

[0097] The operating pressure sensor 29 detects the pilot pressure on the secondary side (pilot line 27A) of the hydraulic pilot type operating device 26, i.e., the pilot pressure corresponding to the operating state of each driven element (hydraulic actuator) in the operating device 26. A detection signal of the pilot pressure by the operating pressure sensor 29 corresponding to the operating state of each driven element (hydraulic actuator HA) in the operating device 26 is input to the controller 30.

[0098] In addition, when the operating device 26 is an electric type, the operating pressure sensor 29 is omitted. This is because the controller 30 can grasp the operating state of each driven element through the operating device 26 based on the operating signal received from the operating device 26.

[0099] The imaging device 40 (an example of an acquisition device) acquires images for interpolating blind spots and places that are difficult to view from the operator's perspective around the shovel 100A. The output of the imaging device 40 is input to the controller 30.

[0100] The imaging device 40 includes, for example, a monocular camera, a stereo camera, a depth camera, etc. Furthermore, the imaging device 40 may acquire three-dimensional data (for example, point cloud data or surface data) representing the positions and external shapes of objects around the work machine 100 within a predetermined imaging range (angle of view) based on the captured image.

[0101] 2 and 3, the imaging device 40 of the excavator 100A includes a camera 40B that images the rear of the upper rotating body 3, a camera 40L that images the left side of the upper rotating body 3, and a camera 40R that images the right side of the upper rotating body 3. This allows the operator to visually recognize the captured images of the imaging device 40 and peripheral images such as processing images generated based on the captured images through the output device 50 and the display device 230A, and to check the conditions to the left, right, and rear of the upper rotating body 3.

[0102] In addition, the two-dot chain lines in FIG. 3 represent the angles of view (imaging ranges) of the cameras 40B, 40L, and 40R when viewed from above.

[0103] 4, the imaging device 40 of the crawler crane 100B is attached to the boom 4B, for example, and acquires captured images of the surroundings of the hook HK and the load suspended by the hook HK viewed from above. This allows the operator to visually recognize the captured images of the imaging device 40 and surrounding images such as processed images generated based on the captured images through the output device 50 (display device) and the display device 230A, and to confirm the surroundings of the load suspended by the hook HK from a perspective viewed from above.

[0104] The imaging device 40 in the form shown in FIG. 4 may be provided in the excavator 100A, and the imaging device 40 (cameras 40B, 40L, 40R) in the form shown in FIGS. 2 and 3 may be provided in the crawler crane 100B.

[0105] The imaging device 42 captures images of the surroundings of the work machine 100 so that a remote operator or a remote monitor can check the state of work being performed by the attachment AT. The output of the imaging device 42 is input to the controller 30.

[0106] The imaging device 42 includes, for example, a monocular camera, a stereo camera, a depth camera, etc. Furthermore, the imaging device 42 may acquire three-dimensional data (for example, point cloud data or surface data) representing the positions and external shapes of objects around the work machine 100 within a predetermined imaging range (angle of view) based on the captured image.

[0107] 2 and 3, the imaging device 42 of the shovel 100A is provided at the front of the upper rotating body 3, such as at the front top surface of the cabin 10, and captures an image of the state in front of the upper rotating body 3 including the tip of the attachment AT (bucket 6A). This allows the operator to operate the shovel 100A while checking the operation of the attachment AT including the bucket 6A by visually checking the image captured by the imaging device 42 and peripheral images such as processing images generated based on the imaging device 42 through the display device 230A.

[0108] 4, the imaging device 42 of the crawler crane 100B is provided in the front of the upper rotating body 3, such as the front of the upper surface of the cabin 10, and captures an image of the state in front of the upper rotating body 3 including the hook HK. Specifically, the imaging device 42 includes a camera 42A that captures an image of a relatively high space in the height direction away from the ground diagonally above and in front of the upper rotating body 3, and a camera 42B that captures an image of a relatively low range in the height direction near the ground diagonally below and in front of the upper rotating body 3. This allows the operator to operate the crawler crane 100B while checking the movement of the hook HK by visually checking the captured image of the imaging device 42 and peripheral images such as processed images generated based on the captured image through the display device 230A.

[0109] The acquisition device SX acquires information relating to the state of the shovel 100A, the state of the periphery of the shovel 100A, etc. The output of the acquisition device SX is input to the controller 30.

[0110] As shown in FIG. 6, when the work machine 100 is a shovel 100A, the acquisition device SX includes a boom angle sensor S1, an arm angle sensor S2, a bucket angle sensor S3, a machine body attitude sensor S4, and a swing angle sensor S5.

[0111] Furthermore, as shown in FIG. 7, when the work machine 100 is a crawler crane 100B, the acquisition device SX includes a machine attitude sensor S4 and a turning angle sensor S5.

[0112] The boom angle sensor S1 acquires detection information related to the attitude angle of the boom 4A (hereinafter, "boom angle") with respect to a predetermined reference (for example, a horizontal plane or a state at either end of a movable angle range of the boom 4A, etc.). The boom angle sensor S1 may include, for example, a rotary encoder, an acceleration sensor, an angular velocity sensor, a six-axis sensor, an IMU (Inertial Measurement Unit), etc. The boom angle sensor S1 may also include a cylinder sensor capable of detecting the extension / retraction position of the boom cylinder 7A.

[0113] The arm angle sensor S2 acquires detection information related to the posture angle of the arm 5A (hereinafter, "arm angle") relative to a predetermined reference (for example, a straight line connecting the connection points at both ends of the boom 4A or a state at either end of the movable angle range of the arm 5A). The arm angle sensor S2 may include, for example, a rotary encoder, an acceleration sensor, an angular velocity sensor, a six-axis sensor, an IMU, etc. The arm angle sensor S2 may also include a cylinder sensor capable of detecting the extension / retraction position of the arm cylinder 8A.

[0114] The bucket angle sensor S3 acquires detection information related to the attitude angle of the bucket 6A (hereinafter, "bucket angle") with respect to a predetermined reference (for example, a straight line connecting the connection points at both ends of the arm 5A or a state at either end of the movable angle range of the bucket 6A). The bucket angle sensor S3 may include, for example, a rotary encoder, an acceleration sensor, an angular velocity sensor, a six-axis sensor, an IMU, etc. The bucket angle sensor S3 may also include a cylinder sensor capable of detecting the extension / retraction position of the bucket cylinder 9A.

[0115] The machine body attitude sensor S4 acquires detection information relating to the attitude state of the machine body of the work machine 100 including the undercarriage 1 and the upper rotating body 3. The machine body attitude sensor S4 is mounted, for example, on the upper rotating body 3, and acquires detection information relating to the inclination angle of the upper rotating body 3 with respect to the horizontal plane and the attitude angle around the rotation axis (i.e., the orientation of the upper rotating body 3 with the ground as a reference). The machine body attitude sensor S4 may include, for example, an acceleration sensor (tilt sensor), an angular velocity sensor, a six-axis sensor, an IMU, etc.

[0116] The turning angle sensor S5 acquires detection information related to the turning angle of the upper turning body 3 (i.e., the orientation of the upper turning body 3) with respect to the lower traveling body 1. The turning angle sensor S5 includes, for example, a potentiometer, a rotary encoder, a resolver, or the like.

[0117] Furthermore, for example, the acquisition device SX may include a positioning device capable of measuring the absolute position of the own machine. The positioning device is, for example, a Global Navigation Satellite System (GNSS) sensor. This makes it possible to improve the estimation accuracy of the attitude state of the work machine 100.

[0118] Furthermore, for example, the acquisition device SX may include a distance sensor (an example of an acquisition device) that detects the distance to an object in the vicinity of the work machine 100. The distance sensor includes, for example, a LIDAR (Light Detecting and Ranging), a millimeter wave radar, an ultrasonic sensor, an infrared sensor, a distance image sensor, etc. This allows the controller 30 (object detection unit 303) to detect objects in the vicinity of the work machine 100 using the output of the distance sensor instead of or in addition to the output of the image capture devices 40, 42.

[0119] Incidentally, the sensors S1-S5 of the shovel 100A and the sensors S4, S5 of the crawler crane 100B may be omitted. For example, the information about the surroundings of the shovel 100A acquired by the imaging devices 40, 42 and distance sensors described below may include information about the positions and shapes of surrounding objects and attachments as viewed from the machine body (upper rotating body 3). In this case, for example, the controller 30 may be able to estimate the attitude state of the attachment AT and the machine body (upper rotating body 3) from that information depending on the required accuracy.

[0120] The operation control section 301 controls the operation of the hydraulic actuator of the work machine 100, with the hydraulic control valve 31 as the control object.

[0121] The operation control section 301, for example, controls the hydraulic control valve 31 as a control target and performs control related to the operation of the hydraulic actuators HA (driven elements) of the work machine 100 based on the operation of the electric operating device 26. Specifically, the operation control section 301 may selectively control the hydraulic control valves 31 corresponding to each of the multiple hydraulic actuators HA (driven elements) in accordance with the content of the operation signal output from the operating device 26.

[0122] Furthermore, the operation control section 301 performs control related to remote operation of the hydraulic actuator HA (driven element) of the work machine 100, for example, with the hydraulic control valve 31 as the control target. That is, the operation of the hydraulic actuator HA (driven element) of the work machine 100 includes remote operation of the hydraulic actuator HA from outside the work machine 100 (remote operation support device 200). Specifically, the operation control section 301 may selectively control the hydraulic control valves 31 corresponding to each of the multiple hydraulic actuators HA (driven elements) in accordance with the content of a remote operation signal received from the remote operation support device 200 via the communication device 60.

[0123] Furthermore, the operation control section 301 performs control related to the automatic driving function of the work machine 100, for example, with the hydraulic control valve 31 as the control target. Specifically, the operation of the hydraulic actuators HA of the work machine 100 may include an operation command for the hydraulic actuators HA of the work machine 100 that is output based on the automatic driving function. Specifically, the operation control section 301 may selectively control the hydraulic control valves 31 corresponding to each of the multiple hydraulic actuators HA (driven elements) according to the content of the operation command.

[0124] The image transmission unit 302 transmits the captured image of the imaging device 40 or a peripheral image such as a processed image generated based on the captured image of the imaging device 40 to the remote operation support device 200 via the communication device 60.

[0125] For example, the image transmission unit 302 of the shovel 100A may generate a processed image (composite image) by combining images captured by multiple cameras (at least two of cameras 40B, 40L, and 40R) based on the image captured by the imaging device 40, and transmit the processed image to the remote operation support device 200.

[0126] Specifically, the image transmission unit 302 may generate a composite image (viewpoint conversion image) viewed from a virtual viewpoint by performing known viewpoint conversion processing and synthesis processing, etc. based on the images captured by the cameras 40B, 40L, and 40R, and transmit the generated composite image to the remote operation support device 200.

[0127] Also, for example, the image transmission unit 302 of the crawler crane 100B may generate a viewpoint conversion image based on the image captured by the imaging device 40, and transmit the image to the remote operation support device 200.

[0128] Similarly, the image transmission unit 302 transmits the captured image by the imaging device 42 or a peripheral image such as a processed image generated by the controller 30 based on the captured image by the imaging device 42 to the remote operation support device 200 via the communication device 60.

[0129] For example, the image transmission unit 302 of the shovel 100A may generate a viewpoint conversion image based on the image captured by the imaging device 42, and transmit the image to the remote operation support device 200. The image transmission unit 302 of the shovel 100A may also perform known viewpoint conversion processing and synthesis processing, etc. on the images captured by the imaging devices 40, 42, to generate a synthetic image in which 360 degrees horizontally around the shovel 100A are viewed from a virtual viewpoint, and transmit the synthetic image to the remote operation support device 200.

[0130] The object detection section 303 detects a predetermined object to be monitored in the periphery of the work machine 100 (hereinafter, a “monitored object”) based on the output of the imaging device 40.

[0131] The monitored objects may include people such as workers working around the work machine 100 and work site supervisors. The monitored objects may also include any object (obstacle) other than people at the work site. Obstacles other than people at the work site may include, for example, specific topography such as holes, ditches, and piles of earth and sand, as well as fixed (non-moving) obstacles such as road cones, fences, utility poles, temporarily stored materials, and temporary offices at the work site. The monitored objects may also include, for example, movable obstacles such as other work machines and work vehicles.

[0132] The object detection unit 303 detects a monitored object in a predetermined monitoring area (hereinafter, for convenience, referred to as "monitoring area") around the work machine 100 (upper rotating body 3) based on the output of the imaging device 40, i.e., the captured image captured by the imaging device 40. The following description of the object detection unit 303 will be made on the assumption that the work machine 100 is located on a horizontal plane. Specifically, the object detection unit 303 of the excavator 100A may detect a monitored object in a monitoring area extending to the rear, left, and right of the upper rotating body 3. Also, the object detection unit 303 of the crawler crane 100B may detect a monitored object in a monitoring area around a load suspended by the hook HK.

[0133] In addition, when the above-mentioned distance sensor is mounted on the upper rotating body 3, the object detection unit 303 may detect a monitored object based on the output of the distance sensor instead of or in addition to the output of the imaging device 40. Furthermore, the object detection unit 303 may detect an object around the upper rotating body 3 based on the output of the imaging device 42 in addition to the output of the imaging device 40. That is, the object detection unit 303 may detect a monitored object in a monitoring area including the imaging range of the imaging device 42 based on the outputs of the imaging devices 40 and 42. Furthermore, when the work machine 100 is used exclusively for remote operation, the function of the object detection unit 303 may be transferred to the remote operation support device 200 (for example, the control device 210 (an example of an object detection device)). In this case, the output of the imaging device 40 or the distance sensor is transmitted to the remote operation support device 200 through the communication device 60.

[0134] The object detection unit 303 detects a monitored object, for example, in a horizontal direction as seen from the work machine 100 (hereinafter simply referred to as the "horizontal direction"), that is, in a monitoring area extending in a direction along a plane on which the work machine 100 is working (the undercarriage 1 is on the ground) (hereinafter for convenience, referred to as the "work plane"). Specifically, the object detection unit 303 may detect a monitored object in a monitoring area in which the distance D from a predetermined portion of the work machine 100 is within a threshold value Dth1. In this case, the threshold value Dth1 may be constant regardless of the direction as seen from the predetermined portion of the work machine 100, or may change depending on the direction as seen from the predetermined portion of the work machine 100. For example, the object detection unit 303 of the excavator 100A detects a monitored object in a monitoring area in which the horizontal distance D from the upper rotating body 3 is within a threshold value Dth1 (for example, 10 meters). Moreover, the object detection unit 303 of the crawler crane 100B detects a monitored object within a monitoring area whose horizontal distance from the hook HK (or the suspended load) is within a threshold value Dth1.

[0135] For example, the object detection unit 303 recognizes a surveillance object in a captured image by arbitrarily applying a machine learning-based classifier including various known image processing methods and artificial intelligence (AI), etc. Furthermore, when the surveillance object is a person such as a worker, the object detection unit 303 may specify whether the recognized person corresponds to any one of a plurality of pre-registered workers.

[0136] In addition, the object detection unit 303 may determine (estimate) the position (e.g., the position of a person's feet) (hereinafter referred to as the "actual position") of a recognized surveillance object captured in an image captured by the monocular imaging device 40 by applying various known techniques.

[0137] For example, the object detection unit 303 of the shovel 100A may estimate the horizontal position (hereinafter, "horizontal position") as seen from the upper revolving structure 3 based on the size of the recognized monitored object in the captured image (for example, the size in the height direction on the captured image). This is because there is a correlation in which the size of the recognized monitored object in the captured image becomes smaller as the monitored object moves away from the shovel 100A (upper revolving structure 3). Specifically, since there is a range of expected sizes for the monitored object (for example, the range of expected human heights), the correlation between the horizontal position of the monitored object as seen from the shovel 100A included in the expected range of sizes and the size on the captured image can be specified in advance. Therefore, the object detection unit 303 can estimate the actual position (horizontal position from the upper revolving structure 3) of the recognized monitored object based on, for example, a map or a conversion formula that indicates the correlation between the size of the monitored object on the captured image and the horizontal position as seen from the shovel 100A, which is stored in advance in an internal memory such as an auxiliary storage device of the controller 30.

[0138] Furthermore, for example, the object detection unit 303 may estimate the actual position (e.g., foot position) of a monitored object by projective transformation (homography) of a captured image onto the plane, on the assumption that the monitored object exists on the same plane as the work machine 100 (specifically, the undercarriage 1). In this case, a certain portion (e.g., a certain pixel) constituting the captured image is associated with a certain position on the same plane as the work machine 100.

[0139] Furthermore, the function of the object detection unit 303 may be switched between ON (enabled) and OFF (disabled) in response to a predetermined operation on the input device 52 by an operator or the like.

[0140] The safety control unit 304 performs control related to the functional safety of the work machine 100. Specifically, when the object detection unit 303 detects a monitoring target, the safety control unit 304 may activate a safety function.

[0141] The safety functions may include, for example, a notification function that outputs an alarm or the like to at least one of the inside of the cabin 10, the outside of the cabin 10, and a remote operator or manager of the work machine 100, and notifies the detection of a monitored object. This makes it possible to alert the operator inside the cabin 10, workers around the work machine 100, and an operator or monitor who remotely operates or remotely monitors the work machine 100 that a monitored object exists within a predetermined range around the work machine 100. Hereinafter, the notification function to the inside of the cabin 10 (operator, etc.) is sometimes referred to as an "internal notification function", the notification function to the outside of the work machine 100 (worker, etc.) is sometimes referred to as an "external notification function", and the notification function to the operator or monitor who remotely operates or remotely monitors the work machine 100 is sometimes referred to as a "remote notification function".

[0142] The safety function may also include, for example, an operation limiting function that limits the operation of the work machine 100 in response to an operation command corresponding to the operation of the operating device 26, remote operation, or the automatic driving function. This makes it possible to forcibly limit the operation of the work machine 100 and reduce the possibility of the work machine 100 or the suspended load approaching or coming into contact with surrounding objects. The operation limiting function may include an operation deceleration function that makes the operating speed of the work machine 100 slower than normal in response to an operation command corresponding to the operation of the operating device 26, remote operation, or the automatic driving function. The operation limiting function may also include an operation stop function that stops the operation of the work machine 100 and maintains the stopped state regardless of an operation command corresponding to the operation of the operating device 26, remote operation, or the automatic driving function.

[0143] The safety control unit 304 activates the alarm function, for example, when the object detection unit 303 detects a monitoring target within a predetermined range (hereinafter, "alert range") included in the monitoring area. The alert range is set to the same range as the monitoring area, or a range whose outer edge is relatively closer to the upper rotating body 3 or the hook HK (or the suspended load) than the monitoring area. Specifically, the safety control unit 304 may activate the alarm function when a monitored object is detected within the alert range where the distance D from a predetermined portion of the work machine 100 is within a threshold value Dth2 (≦Dth1). In this case, the threshold value Dth2 may be constant regardless of the direction as viewed from the predetermined portion of the work machine 100, or may change depending on the direction as viewed from the predetermined portion of the work machine 100.

[0144] The safety control unit 304, for example, controls the output device 50 (sound output device) to activate an internal notification function or an external notification function using sound (i.e., an auditory method) to at least one of the inside and the outside of the cabin 10. At this time, the safety control unit 304 may vary the pitch, sound pressure, tone color, blowing period when blowing a sound periodically, content of the sound, etc. of the output sound according to various conditions.

[0145] Also, the safety control unit 304 activates an internal notification function by, for example, a visual method. Specifically, the safety control unit 304 may control the output device 50 (display device) inside the cabin 10 to display an image indicating that a monitored object has been detected on the display device. Also, the safety control unit 304 may highlight the monitored object shown in the peripheral image displayed on the display device inside the cabin 10 or the position on the peripheral image corresponding to the detected monitored object. More specifically, the safety control unit 304 may superimpose and display a frame surrounding the detected monitored object on the peripheral image displayed on the display device, or superimpose and display a marker on a position on the peripheral image corresponding to the actual position of the detected monitored object. In this way, the output device 50 (display device) can realize a visual notification function for the operator. Also, the safety control unit 304 may notify the operator inside the cabin 10 that a monitored object has been detected by using a warning light, a lighting device, or the like inside the cabin 10.

[0146] The safety control section 304 may also activate an external notification function by a visual method, for example, by controlling an output device 50 (for example, a lighting device such as a headlight or a display device) provided on the side of the house section of the upper rotating body 3. The safety control section 304 may also activate an internal notification function by a tactile method, for example, by controlling a vibration generating device that vibrates the cockpit in which the operator sits. This allows the controller 30 to make the operator, workers and supervisors around the work machine 100 aware of the presence of a monitored object (for example, a person such as a worker) in the vicinity of the work machine 100. Therefore, the controller 30 can urge the operator to check the safety situation around the work machine 100, and can urge workers in the monitored area to evacuate from the monitored area.

[0147] Furthermore, the safety control unit 304 may activate the remote notification function, for example, by transmitting a command signal indicating activation of the notification function to the remote operation support device 200 via the communication device 60. In this case, when the remote operation support device 200 (control device 210) receives a command signal from the work machine 100 via the communication device 220, it may output an alarm by a visual or audio method via the output device 230. This allows an operator or monitor who remotely operates or remotely monitors the work machine 100 via the remote operation support device 200 to know that a monitored object has entered within the notification range around the work machine 100.

[0148] The remote notification function of the safety control unit 304 may be transferred to the remote operation support device 200 (for example, a notification control unit 2103 described later). In this case, the remote operation support device 200 receives information on the detection status of a monitored object by the object detection unit 303 from the work machine 100, determines whether or not a monitored object has entered the notification range based on the received information, and activates an external notification function when a monitored object is present within the notification range.

[0149] Furthermore, the safety control section 304 may vary the notification mode (that is, the method of notification) depending on the positional relationship between the monitoring object detected within the notification range and the work machine 100.

[0150] For example, when a monitored object detected within the notification range by the object detection unit 303 is located relatively far from a predetermined part of the work machine 100, such as the upper rotating body 3 or the hook HK (or the suspended load), the safety control unit 304 may output a relatively low-urgency alarm (hereinafter, "alarm of caution level") that calls attention to the monitored object. Hereinafter, for convenience, a range of the notification range in which the distance to a predetermined part of the work machine 100, such as the upper rotating body 3 or the hook HK (or the suspended load), is relatively far, that is, a range corresponding to an alarm of the caution level, may be referred to as an "alarm notification range." On the other hand, when a monitored object detected within the notification range by the object detection unit 303 is located relatively close to a predetermined part of the work machine 100, such as the upper rotating body 3 or the hook HK (or the suspended load), the safety control unit 304 may output a relatively high-urgency alarm (hereinafter, "alarm of caution level") that notifies that the monitored object is approaching a predetermined part of the work machine 100 and the danger is increasing. Hereinafter, the range within the notification range that is relatively close to a specific part of the work machine 100, such as the upper rotating body 3 or the hook HK (or the suspended load), i.e., the range corresponding to an alert level warning, may be referred to as the "alert notification range."

[0151] In this case, the safety control unit 304 may change the pitch, sound pressure, tone, sound cycle, etc. of the sound output from the output device (sound output device) between the caution level alarm and the alert level alarm. Also, the safety control unit 304 may change the color, shape, size, presence or absence of blinking, blinking cycle, etc. of the image indicating that a monitored object is detected and the monitored object or the image emphasizing the position of the monitored object on the peripheral image displayed on the display device (for example, a frame or marker, etc.) between the caution level alarm and the alert level alarm. In this way, the controller 30 can allow the operator, etc. to grasp the degree of urgency, in other words, the proximity of the monitored object to a predetermined part of the work machine 100, based on the difference in the notification sound (alarm sound) output from the sound output device and the notification image displayed on the display device.

[0152] After the start of the operation of the notification function, the safety control unit 304 may stop the notification function when the monitored object detected by the object detection unit 303 is no longer detected within the monitoring area. Further, after the start of the operation of the notification function, the safety control unit 304 may also stop the notification function when a predetermined operation for canceling the operation of the notification function is received through the input device 52.

[0153] In addition, for example, when the safety control unit 304 detects a monitored object within a predetermined range (hereinafter, "operation restriction range") included in the monitoring area by the object detection unit 303, the safety control unit 304 activates the operation restriction function. The operation restriction range is set to a range such that its outer edge is relatively closer to a predetermined part of the working machine 100 such as the upper swing body 3 or the hook HK (or the suspended load) than the above-described notification range. Thereby, for example, when the monitored object enters the notification range from the outside, the safety control unit 304 can first activate the notification function, and then, when the monitored object enters the inner operation restriction range, further activate the operation restriction function. Therefore, the controller 30 can gradually activate the notification function and the operation restriction function in accordance with the inward movement of the monitored object within the monitoring area. Specifically, when the safety control unit 304 detects a monitored object within the notification range where the distance D from a predetermined part of the working machine 100 is within the threshold Dth3 (<Dth2), the safety control unit 304 may activate the notification function. At this time, the threshold Dth3 may be constant regardless of the direction seen from the predetermined part of the working machine 100, or may vary depending on the direction seen from the predetermined part of the working machine 100.

[0154] The motion limit range also includes at least one of an operational deceleration range in which the motion speed of the work machine 100 is slower than normal in response to an operation command corresponding to the operation of the operating device 26, remote operation, or the automatic driving function, and an operational stop range in which the operation of the work machine 100 is stopped and the stopped state is maintained regardless of an operation command corresponding to the operation of the operating device 26, remote operation, or the automatic driving function. For example, when the motion limit range includes both the operational deceleration range and the operational stop range, the operational stop range is a range within the motion limit range that is close to a predetermined part of the work machine 100, such as the upper rotating body 3 or the hook HK (or a suspended load). The operational deceleration range is a range within the motion limit range that is set outside the operational stop range.

[0155] The safety control section 304 controls the hydraulic control valve 31 to activate an operation limiting function that limits the operation of the work machine 100. In this case, the safety control section 304 may limit the operation of all driven elements (i.e., corresponding hydraulic actuators) or may limit the operation of some of the driven elements (hydraulic actuators). This allows the controller 30 to slow down or stop the operation of the work machine 100 when a monitored object is present around the work machine 100. Therefore, the controller 30 can suppress the occurrence of contact between the monitored object around the work machine 100 and the work machine 100 or the suspended load. In addition, the safety control section 304 may activate the operation limiting function (operation stop function) by controlling an electromagnetic switching valve (not shown) of the pilot line 25 to cut off the pilot line 25.

[0156] Furthermore, the safety control unit 304 may stop the operation restriction function when a monitored object that had been detected by the object detection unit 303 is no longer detected after the operation of the operation restriction function starts. Furthermore, the safety control unit 304 may stop the operation restriction function when a predetermined operation for canceling the operation of the operation restriction function is received through the input device 52 after the operation of the operation restriction function starts. The operation for canceling the operation of the notification function on the input device 52 and the operation for canceling the operation of the operation restriction function may be the same or different.

[0157] Furthermore, the function of the safety control section 304 may be switched between ON (enabled) and OFF (disabled) in response to a predetermined operation on the input device 52 by an operator or the like.

[0158] <Configuration of remote operation support device> As shown in FIG. 5, the remote operation support device 200 includes a control device 210, a communication device 220, an output device 230, and an input device 240.

[0159] The control device 210 performs various controls related to the remote operation support device 200. The functions of the control device 210 are realized by any hardware or a combination of any hardware and software. The control device 210 is mainly configured with a computer including, for example, a CPU, a memory device such as a RAM, a non-volatile auxiliary storage device such as a ROM, and various input / output interface devices.

[0160] The control device 210 includes, as functional units, a remote operation support unit 2101, a display control unit 2102, a notification control unit 2103, a communication status monitoring unit 2104, and an operation status monitoring unit 2105. The functions of the remote operation support unit 2101, the display control unit 2102, the notification control unit 2103, the communication status monitoring unit 2104, and the operation status monitoring unit 2105 are realized, for example, by loading a program installed in an auxiliary storage device into a memory device and executing the program on a CPU.

[0161] The communication device 220 (an example of a communication device for an external device) is connected to the communication line NW and communicates with the outside of the remote operation support device 200 (for example, the work machine 100).

[0162] The output device 230 (an example of a notification device) outputs various information to a user of the remote operation support device 200. The output device 230 includes a display device 230A and a sound output device 230B.

[0163] The display device 230A outputs various information related to the remote operation support device 200 in a visual manner to a user of the remote operation support device 200 under the control of the control device 210. Specifically, the display device 230A displays an information image related to the remote operation support device 200. The display device 230A includes, for example, a liquid crystal display or an organic EL display.

[0164] The output device 230 may include a lighting device that visually outputs various information to the user of the remote operation support device 200. The lighting device includes, for example, a warning lamp.

[0165] The sound output device 230B includes a sound output device that outputs various information by an auditory means to the user of the remote operation support apparatus 200. The sound output device includes, for example, a buzzer, a speaker, and the like.

[0166] The input device 240 accepts input from a user of the remote operation support device 200, and outputs a signal representing the content of the input (for example, operation input, voice input, gesture input, etc.). The signal representing the content of the input is taken into the control device 210. The user of the remote operation support device 200 includes an administrator of the remote operation support device 200, an operator of the work machine 100, a monitor of the work machine 100, etc.

[0167] The input device 240 includes, for example, a remote control device 240A.

[0168] The remote control device 240A is used for remote control of the work machine 100 by an operator or for remote control of the work machine 100 by an observer intervening in the automatic driving function. The remote control device 240A receives inputs from the operator or observer for remote control of a plurality of actuators (driven elements), and outputs a signal indicating the content of the input to the control device 210.

[0169] The remote operation support unit 2101 performs control related to the remote operation of the work machine 100. Specifically, the remote operation support unit 2101 may take in an input signal related to the remote operation of the work machine 100 received by the remote operation device 240A, and transmit a remote operation signal indicating the content of the remote operation of the work machine 100 to the work machine 100 using the communication device 220.

[0170] The display control unit 2102 controls the display device 230A to display various information images on the display device 230A.

[0171] The display control section 2102, for example, causes an image (peripheral image) showing the state of the surroundings of the work machine 100, which is uploaded from the work machine 100, to be displayed on the display device 230A.

[0172] Specifically, the display control unit 2102 causes the display device 230A to display a peripheral image corresponding to the imaging range of the imaging device 42. This allows the user (operator) of the remote operation support device 200 to remotely operate the work machine 100 while checking the movement of the attachment AT by visually checking the peripheral image corresponding to the imaging range of the imaging device 42. In addition, the user (monitor) of the remote operation support device 200 can monitor the work status of the work machine 100 while checking the movement of the attachment AT by visually checking the peripheral image corresponding to the imaging range of the imaging device 42.

[0173] Furthermore, the display control unit 2102 causes the display device 230A to display a peripheral image corresponding to the imaging range of the imaging device 40. This allows the user of the remote operation support device 200 to visually confirm the peripheral image corresponding to the imaging range of the imaging device 40, thereby enabling him / her to confirm the situation around the work machine 100, which is difficult to confirm only from the peripheral image corresponding to the imaging range of the imaging device 42. Therefore, the remote operation support device 200 can improve the safety of the work machine 100 and the work site where the work machine 100 is used.

[0174] The display control section 2102 may generate a processed image as a peripheral image based on an image captured by the imaging device 40 uploaded from the work machine 100, and display the processed image on the display device 230A.

[0175] When a command signal indicating the activation of the notification function (remote notification function) is received via the communication device 220, the notification control unit 2103 notifies the operator or supervisor via the output device 230 that a monitored object has been detected within the notification range around the work machine 100.

[0176] The notification control unit 2103 controls the sound output device 230B, for example, to auditorily notify the operator or supervisor that a monitored object has been detected within the notification range around the work machine 100. At this time, the notification control unit 2103 may vary the pitch, sound pressure, tone color, the sound cycle when the sound is blown periodically, the content of the sound, and the like of the output sound according to various conditions.

[0177] Furthermore, the notification control unit 2103 notifies the operator or the supervisor that a monitored object has been detected within the notification range around the work machine 100, for example, by a visual method. Specifically, the notification control unit 2103 may control the display device 230A via the display control unit 2102 to cause the display device 230A to display an image indicating that a monitored object has been detected. Furthermore, the notification control unit 2103 may cause the display device 230A to highlight the monitored object shown in the peripheral image displayed on the display device 230A or the position on the peripheral image corresponding to the detected monitored object via the display control unit 2102. More specifically, the notification control unit 2103 may cause the display device 230A to display a frame surrounding the detected monitored object superimposed on the peripheral image displayed on the display device 230A, or to display a marker superimposed on a position on the peripheral image corresponding to the actual position of the detected monitored object. In this way, the display device 230A can realize a visual notification function for the operator or the supervisor. Furthermore, the notification control unit 2103 may use a lighting device such as a warning light to notify an operator or a supervisor that a monitored object has been detected within the notification range.

[0178] The communication status monitoring unit 2104 monitors the communication status between the work machine 100 and the remote operation support device 200.

[0179] Specifically, the communication status monitoring unit 2104 monitors the communication status between the work machine 100 and the remote operation support device 200 based on the presence or absence of a predetermined signal received from the work machine 100, the time required from transmission to reception of the predetermined signal, and the like. The communication status between the work machine 100 and the remote operation support device 200 includes, for example, the occurrence of communication interruption between the work machine 100 and the remote operation support device 200. In addition, the communication status between the work machine 100 and the remote operation support device 200 includes, for example, the occurrence of communication delay (the magnitude of communication delay) when a signal is transmitted from the work machine 100 to the remote operation support device 200. The predetermined signal is, for example, a response signal from the work machine 100 to a signal requesting a response that is periodically transmitted to the work machine 100 by the communication status monitoring unit 2104 via the communication device 220. In addition, the predetermined signal may be, for example, a signal corresponding to data (for example, a peripheral image) periodically transmitted (uploaded) from the work machine 100. This allows the communication status monitoring unit 2104 to monitor whether a communication interruption has occurred based on whether a specific signal has been received. Also, the communication status monitoring unit 2104 can measure the magnitude of a communication delay based on data (time stamp) indicating the transmission timing included in the specific signal and time data when the specific signal is actually received through the communication device 220.

[0180] The operation state monitoring unit 2105 monitors the operation state of the actuator (driven element) of the work machine 100 based on the input state received by the remote operation device 240A.

[0181] [Setting the notification range] Next, the process of setting the notification range by the control device 210 will be described with reference to FIGS.

[0182] <First example of notification range setting process> Fig. 8 is a flowchart showing in outline a first example of the processing for setting the notification range by the control device 210. This flowchart is repeatedly executed at a predetermined cycle while the work machine 100 is in operation, that is, from when the work machine 100 is started (for example, when the key switch is turned on) to when the work machine 100 is stopped (for example, when the key switch is turned off). The same may be applied to the flowcharts in Figs. 9 and 10 described below.

[0183] As shown in FIG. 8, in step S102, the communication status monitoring unit 2104 monitors the communication status between the work machine 100 and the remote operation support device 200 based on the reception status of the above-mentioned specified signal and the time lag from transmission to reception of the specified signal.

[0184] When the process of step S102 is completed, the control device 210 proceeds to step S104.

[0185] The communication status monitoring unit 2104 determines whether or not the communication delay between the work machine 100 and the remote operation support device 200 exceeds a predetermined standard based on the monitoring result. If the communication delay between the work machine 100 and the remote operation support device 200 does not exceed the predetermined standard, the communication status monitoring unit 2104 proceeds to step S106, and if it exceeds the predetermined standard, the communication status monitoring unit 2104 proceeds to step S108.

[0186] In step S106, the communication status monitoring unit 2104 sets the notification range of the notification function by the safety control unit 304 to the default state.

[0187] When the process of step S106 is completed, the control device 210 proceeds to step S110.

[0188] On the other hand, in step S108, the communication status monitoring unit 2104 sets the notification range of the notification function by the safety control unit 304 to a state expanded outward from the default.

[0189] For example, if the communication delay between the work machine 100 and the remote operation support device 200 becomes large, the time lag from when a monitored object is captured in the notification range around the work machine 100 by the imaging device 40 or the like to when the notification function of the remote operation support device 200 is activated increases. Therefore, when the notification function of the remote operation support device 200 is activated and the operator or monitor confirms the presence of the monitored object, the monitored object may move inside the notification range and may come even closer to the work machine 100 or the suspended load, etc. This may decrease the safety of the work machine 100 and the work site.

[0190] On the other hand, for example, in the case of an automobile (see, for example, Patent Publication 2016-71585, etc.), when communication delay between the work machine 100 and the remote operation support device 200 becomes large, it is possible to automatically activate the operation restriction function to ensure safety.

[0191] However, while the main function of an automobile is to move at a relatively high speed, and deceleration or stopping is essential to ensure safety, the main function of a work machine is to work in a relatively small area, and there may be cases where deceleration or stopping is not required across the board. Therefore, if the operation restriction function is activated across the board, there is a possibility that the operation of the work machine 100 will be decelerated or stopped even in situations where safety has not decreased significantly. As a result, there is a problem in terms of achieving both the safety of the work machine 100 and the operability of the work machine 100.

[0192] In addition, for example, when communication delay between the work machine 100 and the remote operation support device 200 becomes large, it is possible to activate the operation restriction function only when a monitored object is detected in the monitoring area around the work machine 100.

[0193] However, even when a monitored object is detected in the monitoring area around the work machine 100, there may be cases where safety is not significantly reduced, such as when the monitored object is simply moving near the outer edge of the monitoring area. Therefore, simply activating the operation restriction function of the work machine 100 only when a monitored object is detected in the monitoring area around the work machine 100 may not be sufficient in terms of ensuring the operability of the work machine 100.

[0194] In response to these, in this example, the control device 210 can expand the notification range outward from the default. Therefore, even in a situation where there is a large communication delay between the work machine 100 and the remote operation support device 200, the control device 210 can activate the notification function in the remote operation support device 200 before the monitored object approaches within a relatively close range of the work machine, a suspended load, or the like. Therefore, the control device 210 can improve the safety of the work machine 100 while suppressing a decrease in the workability (work efficiency) of the work machine 100.

[0195] For example, the communication status monitoring unit 2104 sets the outer edge of the notification range to be expanded outward by a predetermined amount or ratio. In this case, the expansion amount or expansion ratio of the outer edge of the notification range may be uniform or may differ depending on the location, and there may be some locations that are expanded and some locations that are not expanded.

[0196] Also, for example, the communication status monitoring unit 2104 may set the outer edge of the notification range to be expanded outward so that the amount of change (expansion) becomes larger as the communication delay between the work machine 100 and the remote operation support device 200 increases.

[0197] When the process of step S108 is completed, the control device 210 proceeds to step S110.

[0198] In step S110, the communication status monitoring unit 2104 transmits a signal to the work machine 100 via the communication device 220 requesting that the notification range be set to the range set in step S106 or step S108. As a result, the controller 30 (safety control unit 304) sets the notification range in response to the signal received from the remote operation support device 200 via the communication device 60. This allows the control device 210 to reflect the settings made in steps S106 and S108 in the control relating to the actual notification function of the work machine 100.

[0199] When the process of step S110 is completed, the control device 210 ends the process of this flowchart.

[0200] <Second example of notification range setting process> FIG. 9 is a flowchart illustrating a second example of the notification range setting process performed by the control device 210. In the example shown in FIG.

[0201] As shown in FIG. 9, steps S202 and S204 are the same as steps S102 and S104 in FIG. 8, and therefore a description thereof will be omitted.

[0202] In step S204, if the communication delay between the work machine 100 and the remote operation support device 200 does not exceed the predetermined standard, the communication status monitoring unit 2104 proceeds to step S206, and if it exceeds the predetermined standard, the communication status monitoring unit 2104 proceeds to step S208.

[0203] Step S206 is the same as the process in step S106 in FIG. 8, and therefore a description thereof will be omitted.

[0204] When the process of step S206 is completed, the control device 210 proceeds to step S212.

[0205] On the other hand, in step S208, the operation state monitoring unit 2105 monitors the operation state of the work machine 100 based on the input state of the remote operation device 240A.

[0206] When the process of step S208 is completed, the control device 210 proceeds to step S210.

[0207] In step S210, the communication status monitoring unit 2104 sets the notification range of the notification function by the safety control unit 304 to a state expanded outward from the default in accordance with the operation state of the work machine 100 based on the monitoring result in step S202.

[0208] For example, when the traveling operation of the undercarriage 1 is being performed via the remote operation device 240A, the communication status monitoring unit 2104 may set the outer edge portion of the notification range in the traveling direction of the work machine 100 to a state in which it is relatively expanded outward compared to the outer edge portions in other directions. This is because, as the work machine 100 moves, the distance between the work machine 100 and a monitored object that exists in the moving direction (traveling direction) of the work machine 100 becomes relatively small.

[0209] Also, for example, when the upper rotating body 3 is being rotated through the remote control device 240A, the communication status monitoring unit 2104 may set the outer edge portion of the notification range in the rotation direction around the rotation axis based on the orientation of the attachment AT to a state in which it is expanded relatively farther outward than the outer edge portion in the opposite direction to the rotation direction. This is because, as the work machine 100 rotates, the distance between the attachment AT and a monitored object existing in the rotation direction around the rotation axis becomes relatively small.

[0210] When the process of step S210 is completed, the control device 210 proceeds to step S212.

[0211] In step S212, the communication status monitoring unit 2104 transmits a signal to the work machine 100 via the communication device 220 requesting that the notification range be set to the range set in step S206 or step S210. As a result, the controller 30 (safety control unit 304) sets the notification range in response to the signal received from the remote operation support device 200 via the communication device 60. This allows the control device 210 to reflect the setting contents of steps S206 and S210 in the control related to the actual notification function of the work machine 100.

[0212] <Third example of notification range setting process> FIG. 10 is a flowchart illustrating a third example of the notification range setting process performed by the control device 210. In the example shown in FIG.

[0213] As shown in FIG. 10, steps S302 and S304 are the same as the processes in steps S102 and S104 in FIG. 8, and therefore a description thereof will be omitted.

[0214] In step S304, if the communication delay between the work machine 100 and the remote operation support device 200 does not exceed the predetermined standard, the communication status monitoring unit 2104 proceeds to step S306, and if it exceeds the predetermined standard, the communication status monitoring unit 2104 proceeds to step S308.

[0215] Step S306 is the same as the process in step S106 in FIG. 8, and therefore a description thereof will be omitted.

[0216] When the process of step S306 is completed, the control device 210 proceeds to step S314.

[0217] On the other hand, in step S308, the communication status monitoring unit 2104 determines whether or not a monitored object has been detected by the object detection unit 303 of the work machine 100. The communication status monitoring unit 2104 can determine whether or not a monitored object has been detected, based on data indicating the object detection status successively received from the work machine 100 via the communication device 220. If a monitored object has been detected by the object detection unit 303, the communication status monitoring unit 2104 proceeds to step S310, and if a monitored object has not been detected, the communication status monitoring unit 2104 proceeds to step S312.

[0218] In step S310, the communication status monitoring unit 2104 sets the outer edge of the notification range to a state expanded outward from the default in accordance with at least one of the position and the movement status of the detected monitored object.

[0219] For example, the communication status monitoring unit 2104 may shift the outer edge of the notification range in the direction in which the detected monitored object is located farther outward than the outer edge in other directions, because there is a possibility that the monitored object may move toward the inside of the notification range.

[0220] Also, for example, when a monitored object is moving inward into the notification range, the communication status monitoring unit 2104 may change the notification range to a larger extent outside the notification range than when the monitored object is not moving inward into the notification range, because it is necessary to notify the operator or monitor of the presence of a monitored object moving inward into the notification range at an earlier timing.

[0221] Also, for example, the communication status monitoring unit 2104 may change the outer edge of the notification range more outward when the moving speed of the monitored object is relatively fast than when the moving speed of the monitored object is relatively slow. This is because it is necessary to notify the operator or monitor of a monitored object that is moving faster into the notification range at an earlier timing.

[0222] When the process of step S310 is completed, the control device 210 proceeds to step S314.

[0223] On the other hand, in step S312, the communication status monitoring unit 2104 sets the outer edge of the notification range to a state expanded outward from the default so that the degree of change (amount of change or rate of change) is relatively smaller than when a monitored object is detected.

[0224] When the process of step S312 is completed, the control device 210 proceeds to step S314.

[0225] In step S314, the communication status monitoring unit 2104 transmits a signal to the work machine 100 via the communication device 220 requesting that the notification range be set to the range set in step S306, step S310, or step S312. As a result, the controller 30 (safety control unit 304) sets the notification range in response to the signal received from the remote operation support device 200 via the communication device 60. This allows the control device 210 to reflect the settings made in steps S306, S310, and S312 in the control of the actual notification function of the work machine 100.

[0226] [Example of monitoring screen] Next, specific examples of a screen (hereinafter, "monitoring screen") including an image of the surroundings of the work machine 100, which is displayed on the display device 230A, will be described with reference to Figs. 11 to 14.

[0227] <Monitoring screen when communication delay is relatively small> 11 and 12 are diagrams showing an example of a monitoring screen including a peripheral image of the work machine 100, displayed on the display device 230A. Specifically, FIGS. 11 and 12 are diagrams showing a specific example of a monitoring screen when the communication status monitoring unit 2104 determines that the communication delay between the work machine 100 and the remote operation support device 200 is equal to or less than a predetermined standard. More specifically, FIG. 11 is a diagram showing an example of a monitoring screen (monitoring screen 1100) displayed on the display device 230A, including a peripheral image of the excavator 100A corresponding to the imaging range of the imaging device 40. FIG. 12 is a diagram showing an example of a monitoring screen (monitoring screen 1200) displayed on the display device 230A, including a peripheral image of the crawler crane 100B corresponding to the imaging range of the imaging device 40.

[0228] In this example, apart from the monitoring screens 1100 and 1200 in Fig. 11 and Fig. 12, the captured image of the imaging device 42 and the processed image generated based on the captured image are displayed in another display area of ​​the display device 230A, or on another display device 230A among the multiple display devices 230A. The same applies to the case where the monitoring screens 1300 and 1400 in Fig. 13 and Fig. 14 described below are displayed on the display device 230A.

[0229] 11, under the control of the display control unit 2102, the display device 230A displays a monitoring screen 1100 including a shovel image CG and a viewpoint converted image EP arranged around the shovel image CG as a peripheral image of the shovel 100A. The shovel image CG and the viewpoint converted image EP are arranged in accordance with the positional relationship between the shovel 100A (upper rotating body 3) and the imaging range of the imaging device 40 (cameras 40B, 40L, 40R). This allows the operator to properly grasp the positional relationship between the shovel 100A and objects around the shovel 100A shown in the viewpoint converted image EP through the monitoring screen 1100.

[0230] In this example, the viewpoint conversion image EP is composed of a combination of an overhead image BVP of the surrounding area adjacent to the shovel 100A viewed from directly above, and a horizontal image HVP of the surrounding area viewed horizontally from the shovel 100A, which is arranged around the overhead image BVP. The viewpoint conversion image EP is obtained by projecting the captured images of the cameras 40B, 40L, and 40R onto a spatial model, and then reprojecting the projected images onto the spatial model onto another two-dimensional plane. The spatial model is a projection target of the captured images in the virtual space, and is composed of one or more planes or curved surfaces, including planes or curved surfaces other than the plane on which the captured images are located.

[0231] Furthermore, in this example, a line LN1 at a certain distance from the shovel 100A is displayed superimposed on the viewpoint converted image EP of the monitoring screen 1100. The line LN1 may represent, for example, the outer edge of the range of notification by the safety control unit 304. This allows the operator or supervisor to appropriately grasp the relationship of the distance between the shovel 100A and surrounding objects shown in the viewpoint converted image EP.

[0232] In this example, the worker W is shown in a location corresponding to the rear of the shovel image CG in the viewpoint conversion image EP of the monitoring screen 1100. This allows the operator or a supervisor to recognize that the worker W is located (exists) behind the shovel 100A (upper rotating body 3).

[0233] In this example, the worker W is outside the line LN1 corresponding to the outer edge of the notification range. Therefore, the worker W is not the target of the visual notification function, and an image emphasizing the presence of the worker W (for example, a frame FRM1 described below) is not displayed.

[0234] The display control unit 2102 may display a monitoring screen including at least one captured image of the cameras 40B, 40L, and 40R on the display device 230A. That is, the display control unit 2102 may display all captured images of the cameras 40B, 40L, and 40R or images captured by two cameras side by side on the display device 230A, or may display an image captured by any one camera on the display device 230A. Typically, the display control unit 2102 may display images captured by the cameras 40B and 40R side by side on the display device 230A. In this case, the display control unit 2102 may superimpose lines representing the outer edges of a monitoring area, a notification range, and the like on a monitoring screen including captured images of the cameras 40B, 40L, and 40R on the display device 230A, similar to the case of the viewpoint conversion image EP.

[0235] Also, for example, as shown in FIG. 12, under the control of the display control section 2102, the display device 230A displays a monitoring screen 1200 including an image captured by the imaging device 40 as an image of the surroundings of the crawler crane 100B.

[0236] The monitoring screen 1200 (image captured by the imaging device 40) shows the hook HK at the tip of the main hoisting rope 7B and the load SL slinging on the hook HK.

[0237] Furthermore, in this example, a line LN2 at a certain distance from the load SL is superimposed and displayed on the monitoring screen 1200. The line LN2 represents, for example, the outer edge of a range of notification by the safety control unit 304. This allows the operator or monitor to properly grasp the positional relationship between the load SL and surrounding objects shown on the monitoring screen 1200 (the image captured by the imaging device 40).

[0238] Furthermore, in this example, two obstacles OB around the load SL are displayed on the monitoring screen 1200. This allows the operator or observer to recognize that there are two obstacles OB around the load SL.

[0239] In this example, both of the two obstacles OB are located outside the line LN1 that indicates the outer edge of the notification range. Therefore, neither of the two obstacles OB is the target of activation of the visual notification function, and no image (e.g., a frame FRM2 described below) that emphasizes the presence of the obstacle OB is displayed.

[0240] The display control unit 2102 may cause the display device 230A to display, as the peripheral image, a viewpoint conversion image generated based on an image captured by the imaging device 40 of the crawler crane 100B.

[0241] <Monitoring screen when communication delay is relatively large> 13 and 14 are diagrams showing another example of a monitoring screen including a peripheral image of the work machine 100, displayed on the display device 230A. Specifically, FIGS. 13 and 14 are diagrams showing a specific example of a monitoring screen when the communication status monitoring unit 2104 judges that the communication delay between the work machine 100 and the remote operation support device 200 exceeds a predetermined standard. More specifically, FIG. 13 is a diagram showing another example (monitoring screen 1300) of a monitoring screen including a peripheral image of the excavator 100A corresponding to the imaging range of the imaging device 40, displayed on the display device 230A. FIG. 14 is a diagram showing an example (monitoring screen 1400) of a monitoring screen including a peripheral image of the crawler crane 100B corresponding to the imaging range of the imaging device 40, displayed on the display device 230A.

[0242] For example, as shown in FIG. 13, the display device 230A displays a monitoring screen 1300 including a shovel image CG and a viewpoint conversion image EP as a peripheral image of the shovel 100A under the control of the display control unit 2102, as in the example described above (FIG. 11).

[0243] Furthermore, in this example, the line LN1 corresponding to the outer edge of the notification range is displayed superimposed on the viewpoint conversion image EP of the monitoring screen 1300, similar to the above-mentioned example.

[0244] In this example, the line LN1 is uniformly changed to the outside, that is, in the direction away from the shovel image CG, compared to the above-mentioned example. This is because the communication delay between the shovel 100A and the remote operation support device 200 exceeds a predetermined standard. This allows the control device 210 to understand, via the display device 230A, that the outer edge of the notification range has changed to the outside compared to normal (when the communication delay is relatively small). In addition, the control device 210 allows the user to understand, via the display device 230A (monitoring screen 1300) that the line LN1 has moved outward compared to normal, that the communication delay between the shovel 100A and the remote operation support device 200 has become relatively large.

[0245] Also, in this example, the viewpoint converted image EP of the monitoring screen 1300 shows the worker W at a position corresponding to the rear of the shovel image CG, as in the above example.

[0246] In this example, a frame FRM1 surrounding the worker W is displayed superimposed on the viewpoint conversion image EP of the monitoring screen 1300. This is because the outer edge of the notification range is shifted outward, so that the worker W is now inside the line LN1. This allows the operator or monitor to become aware of the presence of the worker W at an earlier timing in a situation where there is a relatively large time lag between when the worker W is captured by the imaging device 40 and when the notification function of the remote operation support device 200 is activated.

[0247] The manner in which the line LN1 corresponding to the outer edge of the notification range changes from the above-mentioned example may be varied according to the operating state of the shovel 100A, as described above. For example, when the shovel 100A is operated to travel toward the rear of the upper rotating body 3, the degree of change of the line LN1 from the above-mentioned example may be made larger in the rear (lower) part (curved part) of the shovel image CG than in the left and right parts (straight parts) of the shovel image CG. Also, for example, when the upper rotating body 3 of the shovel 100A is operated to rotate to the right, the degree of change of the line LN1 from the above-mentioned example may be made larger in the right part than in the left part of the shovel image CG. Also, the manner in which the line LN1 corresponding to the outer edge of the notification range changes from the above-mentioned example may be varied according to the position and movement state of the monitored object (worker W) around the shovel 100A, as described above. For example, the degree of change of the line LN1 with respect to the above-mentioned example may be greater in the rear (lower) part of the shovel image CG where the worker W is present than in the left or right part of the shovel image CG where the worker W is not present. Also, for example, the degree of change of the line LN1 with respect to the above-mentioned example may be greater when the worker W is moving toward the shovel 100A than when the worker W is not moving toward the shovel 100A. Also, for example, the degree of change of the line LN1 with respect to the above-mentioned example may be greater when the moving speed of the worker W is relatively high than when it is relatively low.

[0248] Furthermore, in this example, an image MSG1 is displayed on the monitoring screen 1300, which displays a message indicating that a relatively large communication delay is occurring between the shovel 100A and the remote operation support device 200. This enables the operator or monitor to understand that a relatively large communication delay is occurring between the shovel 100A and the remote operation support device 200, and that the line LN1 has changed from normal due to the influence of the communication delay.

[0249] Also, for example, as shown in FIG. 14, the display device 230A displays a monitoring screen 1400 including an image captured by the imaging device 40 as an image of the surroundings of the crawler crane 100B under the control of the display control unit 2102, as in the example described above (FIG. 12).

[0250] The monitoring screen 1400 (image captured by the imaging device 40), like the above example, shows the hook HK at the tip of the main hoisting rope 7B and the load SL slinging on the hook HK.

[0251] Furthermore, in this example, the monitoring screen 1200 displays a line LN2 indicating the outer edge of the notification range superimposed thereon, as in the above example.

[0252] In this example, the line LN2 is moving outward from the above-mentioned example, that is, in a direction away from the load SL (hook HK). This is because the communication delay between the crawler crane 100B and the remote operation support device 200 exceeds a predetermined standard. This allows the control device 210 to understand, via the display device 230A, that the outer edge of the notification range has moved outward from normal (when the communication delay is relatively small). In addition, the control device 210 allows the user to understand, via the display device 230A (monitoring screen 1400) that the line LN2 has moved outward from normal, that the communication delay between the excavator 100A and the remote operation support device 200 has become relatively large.

[0253] Furthermore, in this example, the monitoring screen 1400 shows two obstacles OB around the suspended load SL, as in the above example.

[0254] In this example, a frame FRM2 surrounding each of the two obstacles OB is displayed in a superimposed manner on the monitoring screen 1400. This is because the outer edge of the notification range is shifted outward, and the obstacle OB is now present inside the line LN2. This allows the operator or observer to become aware of the presence of the obstacle OB at an earlier timing in a situation where there is a relatively large time lag between when the obstacle OB is captured by the imaging device 40 and when the notification function of the remote operation support device 200 is activated.

[0255] The manner in which the line LN2 corresponding to the outer edge of the notification range changes from the above-mentioned example may be varied according to the operating state of the crawler crane 100B, as described above. For example, when the crawler crane 100B is operated to travel toward the front of the upper rotating body 3, the degree of change of the line LN2 from the above-mentioned example may be greater in the front portion than in the left, right, and rear portions of the hook HK (suspended load SL). Also, for example, when the upper rotating body 3 of the crawler crane 100B is operated to rotate to the right, the degree of change of the line LN2 from the above-mentioned example may be greater in the right portion than in the left portion of the hook HK (suspended load SL). Also, the manner in which the line LN2 corresponding to the outer edge of the notification range changes from the above-mentioned example may be varied according to the position and movement state of the monitored object (obstacle OB) around the crawler crane 100B, as described above. For example, the degree of change of the line LN2 with respect to the above-mentioned example may be greater in the front or rear portion of the hook HK (load SL) where the obstacle OB is present than in the left or right portion of the hook HK (load SL) where the obstacle OB is not present. Also, for example, the degree of change of the line LN2 with respect to the above-mentioned example may be greater when the obstacle OB is moving relatively toward the hook HK (load SL) than when the obstacle OB is not moving relatively toward the hook HK (load SL). Also, for example, the degree of change of the line LN2 with respect to the above-mentioned example may be greater when the relative moving speed of the obstacle OB with respect to the hook HK (load SL) is relatively high than when it is relatively low.

[0256] Furthermore, in this example, an image MSG2 is displayed on the monitoring screen 1400, showing a message that a relatively large communication delay is occurring between the crawler crane 100B and the remote operation support device 200. This enables the operator or monitor to understand that a relatively large communication delay is occurring between the crawler crane 100B and the remote operation support device 200, and that the line LN2 has changed from normal due to the influence of the communication delay.

[0257] [Effect] Next, the operation of the remote operation support system SYS according to this embodiment will be described.

[0258] In this embodiment, the imaging device 40 of the work machine 100 acquires data (image data) related to objects in the vicinity of the work machine 100. Furthermore, the communication device 60 of the work machine 100 communicates with the remote operation support device 200. Furthermore, the communication device 220 of the remote operation support device 200 communicates with the work machine 100. The remote operation device 240A of the remote operation support device 200 accepts input for remotely operating the work machine 100. Furthermore, the controller 30 of the work machine 100 or the control device 210 of the remote operation support device 200 detects a monitored object in the vicinity of the work machine 100 based on the output of the imaging device 40. Furthermore, the output device 230 is provided in the remote operation support device 200, and when a monitored object is detected within a notification range in the vicinity of the work machine 100 by the controller 30 or the control device 210 under the control of the control device 210, the output device 230 notifies a user using the remote operation device 240A of that fact. When the communication delay between the work machine 100 and the remote operation support device 200 is relatively large, the remote operation support system SYS shifts the outer edge of the notification range outward more than when the communication delay is relatively small.

[0259] As a result, the remote operation support system SYS can set the monitored object as a target for activation of the notification function at a certain stage earlier than the monitored object entering a range inside the notification range where safety may be reduced. Therefore, even in a situation where the time lag from when the monitored object is captured by the imaging device 40 to when the notification function is activated by the remote operation support device 200 is relatively large due to communication delay, the remote operation support system SYS can notify the monitor or operator of the presence of the monitored object before the monitored object enters a range inside the notification range where safety may be relatively reduced. This can improve the safety of the work machine 100. In addition, the operator or monitor can check the monitoring screen displayed on the display device 230A in response to the activation of the notification function and determine for himself or herself whether to continue or stop the operation of the work machine 100. Therefore, it is possible to suppress a situation where the workability of the work machine 100 is significantly reduced, such as when the operation limiting function is uniformly activated or the operation limiting range is expanded outward when the communication delay between the work machine 100 and the remote operation support device 200 becomes relatively large. Therefore, the remote operation support system SYS can improve the safety of the work machine 100 while suppressing a decrease in the workability (work efficiency) of the work machine 100.

[0260] In addition, in this embodiment, when the above-mentioned communication delay is relatively large, the remote operation support system SYS may change the outer edge of the notification range outward in accordance with the contents of the input by the user to the remote operation device 240A.

[0261] As a result, in a situation where the above-mentioned communication delay is relatively large, the remote operation support system SYS can shift the outer edge part of the notification range that has an effect on safety outward from the center in accordance with the operation state of the work machine 100 via the remote operation device 240A. Therefore, the remote operation support system SYS can more appropriately achieve both the safety of the work machine 100 and the workability of the work machine 100.

[0262] Furthermore, in this embodiment, when the above-mentioned communication delay is relatively large, the remote operation support system SYS may, when the work machine 100 is being operated to travel through the remote operation device 240A, change the outer edge of the reporting range in the direction in which the work machine 100 is traveling significantly further outward than the outer edges of the reporting ranges in other directions.

[0263] As a result, the remote operation support system SYS can change the outer edge of the notification range, specifically, the outer edge part that has an effect on the safety of the work machine 100, outward from the center, in accordance with the operation state of the work machine 100 via the remote operation device 240A.

[0264] Furthermore, in this embodiment, when the above-mentioned communication delay is relatively large, the remote operation support system SYS changes the outer edge of the notification range in the rotation direction based on the orientation of the work attachment around the rotation axis more outward than the outer edge of the notification range in the direction opposite to the rotation direction when the work machine 100 is being operated via the remote operation device 240A.

[0265] As a result, the remote operation support system SYS can change the outer edge of the notification range, specifically, the outer edge part that has an effect on the safety of the work machine 100, outward from the center, in accordance with the operation state of the work machine 100 via the remote operation device 240A.

[0266] Furthermore, in this embodiment, when the above-mentioned communication delay is relatively large, the remote operation support system SYS changes the outer edge of the reporting range depending on at least one of the position and movement status of a monitored object present in the vicinity of the work machine 100.

[0267] This allows the remote operation support system SYS to shift the outer edge of the reporting range that has an effect on safety outward from the center in accordance with the position and movement of the monitored object. Therefore, the remote operation support system SYS can more appropriately achieve both the safety of the work machine 100 and the operability of the work machine 100.

[0268] In addition, in this embodiment, when communication delay is relatively large, the remote operation support system SYS may change the outer edge of the notification range in the direction in which the monitored object is located as viewed from the work machine 100 more outward than the outer edges of the notification range in other directions.

[0269] This allows the remote operation support system SYS to specifically shift the outer edge of the reporting range that has an impact on safety outward from the center in accordance with the position of the monitored object.

[0270] In addition, in this embodiment, when the communication delay is relatively large, the remote operation support system SYS may change the outer edge of the notification range more outward when the monitored object is moving toward the inside of the notification range than when the monitored object is not moving toward the inside of the notification range.

[0271] This allows the remote operation support system SYS to specifically shift the outer edge of the reporting range that has an impact on safety outward from the center in accordance with the movement status of the monitored object.

[0272] In addition, in this embodiment, when the communication delay is relatively large, the remote operation support system SYS may change the outer edge of the specified range more outward when the movement speed of the monitored object is relatively large than when the movement speed of the monitored object is relatively small.

[0273] This allows the remote operation support system SYS to specifically shift the outer edge of the reporting range that has an impact on safety outward from the center in accordance with the movement status of the monitored object.

[0274] Furthermore, in this embodiment, the display device 230A may display information related to the notification range such that the display content changes according to changes in the outer edge of the notification range.

[0275] This allows the remote operation support system SYS to allow the operator or observer to grasp, via the display device 230A, the change in the outer edge of the notification range that accompanies the increase in the communication delay described above.

[0276] Although the embodiments have been described in detail above, the present disclosure is not limited to such specific embodiments, and various modifications and variations are possible within the scope of the gist described in the claims. [Explanation of symbols]

[0277] 30 Controller (object detection device) 40 Imaging device (acquisition device) 40B Camera 40L Camera 40R Camera 42 Imaging device 42A Camera 42B Camera 50 Output Device 52 Input Device 60 Communication device (communication device for work equipment) 100 Work Machines 100A Shovel 100B Crawler Crane 200 Remote operation support device (external device) 210 Control device (object detection device) 220 Communication device (communication device for external device) 230 Output device (alarm device) 230A display device 230B Sound output device 240 Input Device 240A Remote Control Device 2101 Remote Operation Support Department 2102 Display control unit 2103 Notification control unit 2104 Communication Status Monitoring Department 2105 Operation status monitoring unit SYS Remote operation support system

Claims

1. An acquisition device provided on a work machine for acquiring data relating to objects in the vicinity of the work machine; a work machine communication device that is provided in the work machine and communicates with an external device that assists in remote operation of the work machine; an external device communication device provided in the external device and configured to communicate with the work machine; a remote operation device provided in the external device and configured to receive an input for remotely operating the work machine; an object detection device that is provided on the work machine or the external device and detects a predetermined object around the work machine based on an output of the acquisition device; an alarm device that is provided in the external device and that, when the predetermined object is detected by the object detection device within a predetermined range around the work machine, notifies a user of the remote control device of that fact, When a communication delay between the work machine and the external device is relatively large, the outer edge of the predetermined range is shifted outward compared to when the communication delay is relatively small. Remote operation support system.

2. When the communication delay is relatively large, the outer edge of the predetermined range is shifted outward in accordance with the input contents of the user to the remote control device. The remote operation support system according to claim 1 .

3. When the communication delay is relatively large, when the work machine is being operated to travel through the remote control device, the outer edge of the specified range in the traveling direction as seen from the work machine is significantly shifted outward compared to the outer edges of the specified range in other directions. The remote operation support system according to claim 2 .

4. When the communication delay is relatively large, when a swing operation of the work machine is being performed through the remote control device, an outer edge of the predetermined range in a swing direction based on a direction of the work attachment about a swing axis is largely shifted outward from an outer edge of the predetermined range in a direction opposite to the swing direction. The remote operation support system according to claim 2 or 3.

5. when the communication delay is relatively large, an outer edge of the predetermined range is changed in accordance with at least one of a position and a movement state of the predetermined object present in the periphery of the work machine. The remote operation support system according to any one of claims 1 to 4.

6. when the communication delay is relatively large, the outer edge of the predetermined range in the direction in which the predetermined object is located as viewed from the work machine is shifted more outward than the outer edges of the predetermined range in other directions; The remote operation support system according to claim 5 .

7. When the communication delay is relatively large, when the predetermined object is moving toward the inside of the predetermined range, the outer edge of the predetermined range is changed more outwardly than when the predetermined object is not moving toward the inside of the predetermined range.

7. The remote operation support system according to claim 5 or 6.

8. When the communication delay is relatively large, and the moving speed of the predetermined object is relatively large, the outer edge of the predetermined range is changed outwardly to a greater extent than when the moving speed of the predetermined object is relatively small. The remote operation support system according to any one of claims 5 to 7.

9. a display device provided in the external device and configured to display information about the predetermined range such that the display content changes in response to a change in the outer edge of the predetermined range; The remote operation support system according to any one of claims 1 to 8.

10. a communication device that communicates with a work machine having an acquisition device that acquires data related to objects around the work machine; a remote control device that receives an input for remotely controlling the work machine; a notification device that, when a predetermined object is detected within a predetermined range around the work machine based on the output of the acquisition device, notifies a user of the remote control device of that fact, when a communication delay between the work machine and the work machine is relatively large, the outer edge of the predetermined range is changed in a direction away from the work machine compared to when the communication delay is relatively small. Remote operation support device.

Images