Work machines, surrounding monitoring systems for work machines

Abstract

The present invention provides a construction machine or the like that allows users such as workers or supervisors to check the status of a work site from outside. To this end, a construction machine (e.g., shovel 100) according to one embodiment of the present invention is equipped with a sensor (e.g., imaging device 40) that acquires information about the status around the construction machine, and a transmitter (e.g., transmitter 305C) that transmits information about the status around the construction machine acquired by the sensor or generated based on the information acquired by the sensor to an external device (e.g., support device 200 or management device 300).

Description

Technical Field

[0001] The present invention relates to a working machine and the like.

Background Art

[0002] Working machines such as excavators are known (for example, see Patent Document 1).

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] However, workers, supervisors, etc. may want to check the situation at the work site from outside the work site. For example, it is a case where it rains early in the morning on the work day and it is desired to know whether the work of the day can be carried out at the work site. Therefore, it is desirable to use a working machine to obtain information regarding the situation at the work site and enable workers, supervisors, etc. to check the information outside the work site.

[0005] Therefore, in view of the above problems, an object is to provide a working machine or the like that enables users such as workers and supervisors to check the situation at the work site from the outside.

Means for Solving the Problems

[0006] In one embodiment of the present invention, a sensor that acquires information regarding the situation around the working machine, a transmission unit that transmits information regarding the situation around the working machine, which is acquired by the sensor or generated based on the information acquired by the sensor, to an external device, are provided. A working machine is provided.

[0007] Also, in another embodiment of the present invention, an information processing device that is communicable with the working machine and includes a monitoring unit that acquires information regarding the situation around the working machine and transmits it outside the working machine, A transmitting unit that transmits a command to the work machine requesting information about the surrounding conditions of the work machine in response to a predetermined operation by the user, A first acquisition unit that acquires information regarding the surrounding conditions of the work machine transmitted from the work machine in response to the command, The system includes a first notification unit that notifies the user of information regarding the surrounding conditions of the work machine, which has been acquired by the first acquisition unit, An information processing device is provided.

[0008] Furthermore, in yet another embodiment of the present invention, An information processing method performed by an information processing device that can communicate with a work machine, the device being equipped with a monitoring unit that acquires information about the surrounding conditions of the work machine and transmits it to the outside of the work machine, A transmission step of sending a command to the work machine requesting information about the surrounding conditions of the work machine in response to a predetermined operation from the user, A first acquisition step of acquiring information about the surrounding conditions of the work machine transmitted from the work machine in response to the command, The process includes a first notification step of notifying the user of information regarding the surrounding conditions of the work machine, which was obtained in the first acquisition step, Information processing methods are provided.

[0009] Furthermore, in yet another embodiment of the present invention, An information processing device that can communicate with a work machine is provided, which includes a monitoring unit that acquires information about the surrounding conditions of the work machine and transmits it to the outside of the work machine, A transmission step of sending a command to the work machine requesting information about the surrounding conditions of the work machine in response to a predetermined operation from the user, A first acquisition step of acquiring information about the surrounding conditions of the work machine transmitted from the work machine in response to the command, A first notification step is performed to notify the user of information regarding the surrounding conditions of the work machine, which was obtained in the first acquisition step. A program will be provided. [Effects of the Invention]

[0010] According to the above-described embodiment, it is possible to provide work machinery and the like that allows users such as workers and supervisors to check the status of the work site from an external location. [Brief explanation of the drawing]

[0011] [Figure 1] This is a schematic diagram showing an example of a field monitoring system. [Figure 2] This is a diagram showing an example of the configuration of a field monitoring system. [Figure 3] This diagram shows an example of a power supply system for an excavator. [Figure 4] This figure shows an example of a remote control permission screen displayed on a display device. [Figure 5A] This figure shows an example of a monitoring image (through image) displayed on a display device. [Figure 5B] This figure shows another example of a monitoring image (surrounding image) displayed on a display device. [Figure 6] This diagram schematically shows an example of a monitoring area. [Figure 7A] This figure shows an example of the command transmission operation screen for a support device. [Figure 7B] This figure shows an example of the command transmission operation screen for a support device. [Figure 8] This figure shows an example of a screen displaying the surrounding conditions of a support device. [Figure 9] This figure shows an example of the arrangement of parked excavators at a work site. [Figure 10A] This flowchart provides a schematic overview of one example of how a field monitoring system operates. [Figure 10B] This flowchart provides a schematic overview of one example of how a field monitoring system operates. [Figure 10C] This flowchart provides a schematic overview of one example of how a field monitoring system operates. [Figure 10D] It is a flowchart schematically showing an example of the operation of the on-site monitoring system.

Figure 11

Figure 12

Figure 13A

Figure 13B

Figure 14

Figure 15A

Figure 15B

Figure 15C

Figure 15D

Figure 16

[0012] The following describes embodiments for carrying out the invention with reference to the drawings.

[0013] [First Embodiment] First, the first embodiment will be described.

[0014] [Overview of the Field Monitoring System] First, with reference to Figure 1, the field monitoring system SYS according to this embodiment will be described.

[0015] Figure 1 is a schematic diagram showing an example of the configuration of the SYS field monitoring system.

[0016] The site monitoring system SYS includes a shovel 100, a support device 200, and a management device 300. It acquires information regarding the surrounding conditions of the shovel 100 (hereinafter referred to as "surrounding conditions information" for convenience), that is, information regarding the conditions of the work site where the shovel 100 is working, and provides this surrounding conditions information to the user through the support device 200. This allows the user of the support device 200 to check the conditions of the work site even when they are outside the work site. Hereinafter, the operation of the shovel 100 to acquire surrounding conditions information and transmit it to the support device 200 will be referred to as "site monitoring" for convenience. The site monitoring system SYS may include one shovel 100 or multiple shovels 100. Similarly, the site monitoring system SYS may include one support device 200 or multiple support devices 200.

[0017] In Figures 1 and 2, for convenience, one excavator 100 and one support device 200 are shown. Furthermore, the field monitoring system SYS may include other types of work machinery in place of, or in addition to, the excavator 100. For example, the field monitoring system SYS may include a lifting magnet machine with a lifting magnet attached as an end attachment, a bulldozer, a wheel loader, an asphalt finisher, forestry machinery, a crawler crane, etc.

[0018] <Overview of an excavator> Excavator 100 (an example of a work machine) comprises a lower traveling body 1, an upper rotating body 3 mounted on the lower traveling body 1 so as to be rotatable via a slewing mechanism 2, a boom 4, an arm 5, and a bucket 6 as attachments (working equipment), a cabin 10, and an engine 11.

[0019] The lower traveling body 1 includes, for example, a pair of left and right crawlers, and each crawler is hydraulically driven by a travel hydraulic motor (not shown), thereby enabling it to move under its own power.

[0020] The upper rotating body 3 rotates relative to the lower traveling body 1 when driven by a rotating hydraulic motor or the like.

[0021] The boom 4 is pivotally attached to the front center of the upper slewing body 3 so as to be able to move up and down. The arm 5 is pivotally attached to the tip of the boom 4 so as to be able to rotate up and down. The bucket 6 is pivotally attached to the tip of the arm 5 so as to be able to rotate up and down. The boom 4, arm 5, and bucket 6 are hydraulically driven by the boom cylinder 7, arm cylinder 8, and bucket cylinder 9, respectively.

[0022] Cabin 10 is the cockpit where the operator sits and is mounted on the front left side of the upper rotating body 3.

[0023] The engine 11 is the power source for the shovel 100 and is mounted, for example, at the rear of the upper slewing body 3. The engine 11 is, for example, a diesel engine that uses light oil as fuel. The engine 11 operates to maintain a predetermined rotational speed under the control of, for example, an engine control module (ECM) 30B, which will be described later. The rotation shaft of the engine 11 is connected to a main pump 14 that supplies hydraulic fluid to hydraulic actuators, including a travel hydraulic motor, a slewing hydraulic motor, a boom cylinder 7, an arm cylinder 8, and a bucket cylinder 9, as well as a pilot pump that generates the source pressure for pilot pressure to operate the hydraulic actuators.

[0024] Furthermore, Shovel 100 may also be powered by an external commercial power source connected via a battery or cable. In other words, Shovel 100 may be an electric shovel, such as a battery-powered shovel or cable shovel.

[0025] The shovel 100 can communicate with the management device 300 via a predetermined communication network NW, including, for example, a mobile communication network with a base station as its endpoint or the internet. This allows the shovel 100 to transmit (upload) various information to the management device 300 and provide (transmit) various information to the support device 200 via the management device 300. Further details will be described later.

[0026] Furthermore, the shovel 100 may be connected to the support device 200 via a communication network NW in a P2P (Peer to Peer) manner, and may directly exchange various information with each other without going through the management device 300.

[0027] <Overview of support device> The support device 200 (an example of an external device, information processing device, or user terminal) is a portable terminal carried by the user (for example, a worker or supervisor at the work site of the shovel 100). The support device 200 may be, for example, a general-purpose notebook PC, tablet terminal, or smartphone carried by the user. Alternatively, the support device 200 may be a dedicated terminal for checking surrounding situation information related to the shovel 100.

[0028] The support device 200 can communicate with the management device 300 via a communication network NW. This allows the support device 200 to send a command to the excavator 100 requesting surrounding condition information (hereinafter referred to as "site monitoring command") via the management device 300. The support device 200 then receives various information, such as surrounding condition information, transmitted from the excavator 100 in response to the site monitoring command, and can provide (notify) the user of the received information, such as surrounding condition information, through a display device 240, which is mounted on the support device. Further details will be provided later.

[0029] <Overview of the control device> The management device 300 (an example of an information processing device) is a terminal device installed at a geographically separate location from the shovel 100 and the support device 200. The management device 300 is installed, for example, in a management center located outside the work site where the shovel 100 operates, and is a server device consisting of one or more server computers. In this case, the server device may be a proprietary server operated by the operator of the site monitoring system SYS or a related company associated with that operator, or it may be a so-called cloud server. Alternatively, the management device 300 may be a stationary or portable computer terminal located in a management office or similar location within the work site of the shovel 100.

[0030] As described above, the management device 300 can communicate with the shovel 100 and the support device 200 via the communication network NW. This allows the management device 300 to receive and store various information uploaded from the shovel 100. The management device 300 can also receive various requests transmitted from the support device 200 and, in response to these requests, provide (transmit) various information (for example, surrounding condition information) obtained from the shovel 100.

[0031] [Configuration of the on-site monitoring system] Next, with reference to Figure 1, as well as Figures 2 and 3, the specific configuration of the field monitoring system SYS according to this embodiment will be described.

[0032] Figure 2 is a diagram showing an example of the configuration of the field monitoring system SYS. Figure 3 is a diagram showing an example of the power supply system of the shovel 100.

[0033] The key cylinder position of the Shovel 100 (hereinafter referred to as "key position") is set in four stages in the following order: OFF position, accessory position (hereinafter referred to as "ACC position"), ignition position (hereinafter referred to as "IG position"), and start position. The ACC position is linked to the accessory switch (hereinafter referred to as "ACC switch") 66, which will be described later, and is a key position for supplying power only to some of the Shovel 100's equipment without starting the Shovel 100 (i.e., without starting the engine 11). The IG position is linked to the ignition switch (hereinafter referred to as "IG switch") 68, which will be described later, and is a key position for supplying power to equipment that starts up in conjunction with the start of the Shovel 100 (start of the engine 11). The start position is a key position for activating the starter of the engine 11 and starting the engine 11, and in the start position, the ACC switch 66 is temporarily disconnected.

[0034] <Shovel configuration> The shovel 100 includes a shovel controller 30A, an ECM 30B, a peripheral monitoring controller 30C, an imaging device 40, a display device 50, an audio output device 52, a gate lock valve 54, a communication device 60, and an authorization operation unit 62.

[0035] The shovel controller 30A is mounted, for example, inside the cabin 10 and performs overall drive control of the shovel 100. Power is supplied to the shovel controller 30A from the battery 64 (a power source, an example of a secondary battery) through both the system via the ACC switch 66 (hereinafter referred to as the "ACC system") and the system via the IG switch 68 (hereinafter referred to as the "IG system"). As a result, the shovel controller 30A is started before the engine 11 is started and the power supply is not interrupted when the engine 11 is started. The same applies to the ECM 30B below.

[0036] Furthermore, the battery 64 is charged, for example, by the power of an alternator generated by the engine 11.

[0037] The functions of the shovel controller 30A may be realized by any hardware, software, or a combination thereof. The shovel controller 30A is mainly composed of a microcomputer including, for example, a CPU (Central Processing Unit), RAM (Random Access Memory), ROM (Read Only Memory), auxiliary storage device, RTC (Real Time Clock), and various communication interfaces. The same applies to the ECM 30B and peripheral monitoring controller 30C below. The same also applies to the control device 210 of the support device 200 and the control device 310 of the management device 300, which will be described later.

[0038] The ECM30B controls the engine 11 under the control of the shovel controller 30A. For example, in response to commands from the shovel controller 30A, the ECM30B controls various actuators (e.g., fuel injectors, etc.) mounted on the engine 11 so that the engine 11 maintains a predetermined rotational speed, as described above.

[0039] The peripheral monitoring controller 30C controls the monitoring of the area surrounding the shovel 100, including the on-site monitoring described above. For example, the peripheral monitoring controller 30C controls the imaging device 40 to acquire information about the conditions around the shovel 100, that is, captured images showing the conditions (appearance) around the shovel 100. Also, for example, based on the information (captured images) acquired from the imaging device 40, the peripheral monitoring controller 30C detects a predetermined monitoring target within a predetermined monitoring range around the shovel 100. In this case, the monitoring target may include not only people such as workers and supervisors working around the shovel 100, but also any objects such as stationary obstacles such as materials and soil at the work site, and mobile obstacles (i.e., moving objects) such as other work machinery and trucks at the work site. Furthermore, the monitoring target may include suspicious persons who have entered the work site outside of working hours.

[0040] The peripheral monitoring controller 30C is always electrically connected to the battery 64 and can operate by receiving power from the battery 64 even when the shovel 100 is stopped. Specifically, when the shovel 100 is in operation, all functions of the peripheral monitoring controller 30C are operational, while when the shovel 100 is stopped (when the engine 11 is stopped), a minimum number of functions are operational. Further details will be described later.

[0041] The peripheral monitoring controller 30C includes, for example, a monitoring image generation unit 301C, a detection unit 302C, an alarm output unit 303C, an operation limiting unit 304C, a transmission unit 305C, a command acquisition unit 306C, and a start / stop unit 307C, as functional units realized by executing one or more programs stored in ROM or a non-volatile auxiliary storage device on the CPU. The peripheral monitoring controller 30C also includes, for example, a storage unit 308C as a storage area defined in a non-volatile internal memory such as an auxiliary storage device.

[0042] Furthermore, when the shovel 100 is stopped (i.e., the engine 11 is stopped), the peripheral monitoring controller 30C wakes up only the command acquisition unit 306C among the monitoring image generation unit 301C, detection unit 302C, alarm output unit 303C, operation limiting unit 304C, transmission unit 305C, command acquisition unit 306C, and start / stop unit 307C, while stopping (putting into sleep mode) the other functional units. This allows the peripheral monitoring controller 30C to minimize power consumption while the shovel 100 is stopped.

[0043] The imaging device 40 (an example of a sensor) is mounted on the upper part of the upper rotating body 3 and images the area around the shovel 100. The imaging device 40 includes cameras 40B, 40L, and 40R.

[0044] Cameras 40B, 40L, and 40R are mounted on the upper rear end, upper left end, and upper right end of the upper rotating body 3, respectively, and capture images of the rear, left, and right sides of the upper rotating body 3. For example, cameras 40B, 40L, and 40R are each monocular wide-angle cameras with a very wide field of view. Specifically, cameras 40B, 40L, and 40R are mounted on the upper part of the upper rotating body 3 with their optical axes pointing diagonally downwards, and capture an image range in the vertical direction, including from the ground near the shovel to far beyond the shovel. During the operation of the shovel 100, cameras 40B, 40L, and 40R each output captured images at predetermined intervals (e.g., 1 / 30 second), and the output captured images are taken into the peripheral monitoring controller 30C.

[0045] The imaging device 40 is powered by the battery 64 via the IG system. The imaging device 40 can also be powered by the battery 64 via a branch system that branches off from the system that is always connected to the battery 64 (hereinafter referred to as the "always-connected system") and passes through relay 42. Specifically, cameras 40B, 40L, and 40R can each be powered by the battery 64 via branch systems that branch off from the always-connected system and pass through relays 42B, 42L, and 42R. As a result, the imaging device 40 can be started even when the shovel 100 (engine 11) is stopped. In addition, cameras 40B, 40L, and 40R can be started individually even when the shovel 100 (engine 11) is stopped.

[0046] Furthermore, the shovel 100 may be equipped with a microphone capable of collecting sounds from the surrounding area, either in place of or in addition to the imaging device 40. In this case, the microphone may be connected to a branching system that includes a relay similar to that of the imaging device 40. This allows the microphone to be activated even when the shovel 100 (engine 11) is stopped.

[0047] The display device 50 is installed in the cabin 10 around the cockpit, specifically in a position easily visible to the operator seated in the cockpit, and displays various image information to notify the operator. Power is supplied to the display device 50 from the battery 64 via the ACC system. The same applies to the audio output device 52. The display device 50 may be, for example, a liquid crystal display or an organic EL (electroluminescence) display, and may also be a touch panel that also serves as an operation unit. Specifically, as described later, the display device 50 displays images captured by the imaging device 40 (hereinafter sometimes referred to as "through images") as monitoring images showing the surroundings of the shovel 100, or composite images (for example, viewpoint transformation images described later) generated (combined) by the surrounding monitoring controller 30C based on the images captured by the imaging device 40.

[0048] The audio output device 52 is installed around the cockpit in the cabin 10 and outputs various audio information to notify the operator. The audio output device 52 is, for example, a speaker or a buzzer. Specifically, the audio output device 52 outputs an alarm sound.

[0049] The gate lock valve 54 is located at the uppermost part of the pilot line that supplies pilot pressure from the aforementioned pilot pump to the operating devices used by the operator to control various operating elements of the shovel 100 (i.e., the lower traveling body 1, the upper slewing body 3, the boom 4, the arm 5, and the bucket 6, etc.), and switches the connection / disconnection of the pilot line.

[0050] For example, the gate lock valve 54 switches the connection / disconnection of the pilot line in accordance with the output signal (ON / OFF) of a gate lock switch, which is usually linked to the operating state of a gate lock lever located in the area corresponding to the entrance to the cockpit inside the cabin 10. Specifically, the gate lock valve 54 connects the pilot line when the output signal of the gate lock switch corresponds to the state in which the gate lock lever is raised (i.e., the state in which an operator is seated in the cockpit) (hereinafter referred to as the "OFF signal" for convenience). On the other hand, the gate lock valve 54 disconnects the pilot line when the output signal of the gate lock switch corresponds to the state in which the gate lock lever is lowered (i.e., the state in which an operator has left the cockpit) (hereinafter referred to as the "ON signal" for convenience).

[0051] Furthermore, for example, the gate lock valve 54 is configured to also accept command signals input from the peripheral monitoring controller 30C. Specifically, the signal input section of the gate lock valve 54 may be connected to a logic circuit that can interrupt with command signals from the peripheral monitoring controller 30C. This allows the gate lock valve 54 to switch the connection / disconnection of the pilot line in response to command signals (OFF signals / ON signals) from the peripheral monitoring controller 30C. In other words, the gate lock valve 54 can disconnect the pilot line even when the gate lock lever is raised, under the control of the peripheral monitoring controller 30C.

[0052] The communication device 60 is any device that communicates with external devices such as the management device 300 via the communication network NW. The communication device 60 is always electrically connected to the battery 64 and can operate by receiving power from the battery 64 even when the shovel 100 is stopped. As a result, the communication device 60 can receive various information from the outside even when the shovel 100 is stopped. The communication device 60 is a mobile communication module that supports mobile communication standards such as LTE (Long Term Evolution), 4G (4th Generation), and 5G (5th Generation).

[0053] The authorization operation unit 62 receives an operation from an operator or the like to authorize remote operation related to monitoring the surrounding area (on-site monitoring) of the shovel 100 by a specific support device 200. The authorization operation unit 62 can be realized by a combination of hardware operation units such as buttons, toggles, levers, or a touch panel implemented on the display device 50, and software operation units such as button icons on the operation screen displayed on the display device 50.

[0054] For example, Figure 4 shows an example of the authorization operation unit 62. Specifically, Figure 4 shows an example of the operation screen (hereinafter referred to as the "remote operation authorization operation screen") displayed on the display device 50 (remote operation authorization operation screen 400).

[0055] As shown in Figure 4, the display device 50 includes a display unit 50A which is a display area on which various information images are displayed, and an operation unit 50B which is a hardware unit such as a button switch that can accept operations by an operator or the like regarding the various information images displayed on the display unit 50A.

[0056] The display unit 50A is, for example, a horizontally elongated rectangular display in the display device 50 (for example, a display with an aspect ratio of 4:3).

[0057] As shown in Figure 4, the remote operation permission screen 400 displays a candidate list 401 of support devices 200 that are authorized to remotely control the surrounding area of ​​the shovel 100. The candidate list 401 is displayed based, for example, on candidate terminal information 3081C stored in the memory unit 308C, which will be described later. The candidate terminal information 3081C in the memory unit 308C is information about support devices 200 that have been pre-registered in the management device 300, and is updated with the latest candidate terminal information distributed from the management device 300.

[0058] For example, an operator can scroll through the candidate list 401 via a touch panel or control unit 50B implemented on the display device 50 to position a support device 200 on the cursor 402. This allows the operator to select one support device 200 from among the multiple support devices 200 included in the candidate list 401.

[0059] The operator then makes a decision via the touch panel or control unit 50B as appropriate. This allows the authorization control unit 62 to authorize remote operation of the selected support device 200 for on-site monitoring of the shovel 100. At this time, the support device 200 authorized for remote operation of the shovel 100's surroundings is registered in the authorization terminal information 3082C of the storage unit 308C, which will be described later. Furthermore, the authorization control unit 62 (remote operation authorization screen 400) notifies the management device 300 via the communication device 60 of the support device 200 authorized for remote operation of the shovel 100's on-site monitoring. This allows the management device 300 to keep track of and manage the support device 200 authorized for remote operation of on-site monitoring for each shovel 100.

[0060] Furthermore, the authorized support devices 200 may be identified (selected) by other methods. For example, the support device 200 held by the operator may be identified (selected) as the authorized support device 200 by predetermined proximity communication such as Bluetooth® communication or WiFi® communication between the shovel 100 and the support device 200 held by the operator, etc. Also, the support devices 200 for which remote operation related to on-site monitoring of each shovel 100 is permitted may be determined (registered) by the management device 300. In other words, the authorization operation unit 62 is provided in the management device 300, and the administrator of the management device 300 may authorize remote operation related to on-site monitoring of each shovel 100 by a specific support device 200. This is because the on-site monitoring command from the support device 200 is transmitted to the shovel 100 via the management device 300, so it is sufficient for the management device 300 to determine whether or not remote operation related to on-site monitoring of the shovel 100 is permitted. For example, the administrator of the control device 300 may, for each excavator 100, grant permission for remote operation only to support devices 200 related to the specific site where the excavator 100 is working (for example, support devices 200 for supervisors or workers at that site), selected from a plurality of pre-registered support devices 200.

[0061] Returning to Figures 1 to 3, the monitoring image generation unit 301C displays the monitoring image on the display device 50 based on the image captured by the imaging device 40.

[0062] For example, the monitoring image generation unit 301C displays an image captured by at least one of the cameras 40B, 40L, and 40R on the display device 50 as a monitoring image.

[0063] Furthermore, for example, the monitoring image generation unit 301C generates a separate image (hereinafter referred to as "surrounding image") representing the surroundings of the shovel 100 as a monitoring image, based on the image captured by the imaging device 40. Specifically, the monitoring image generation unit 301C may generate the surrounding image by combining the images captured by cameras 40B, 40L, and 40R, or by performing processing to transform the viewpoint. The monitoring image generation unit 301C then displays the monitoring image, including the generated surrounding image, on the display device 50.

[0064] More specifically, the monitoring image generation unit 301C generates a viewpoint-transformed image as seen from a virtual viewpoint by performing known viewpoint transformation processing based on the images captured by cameras 40B, 40L, and 40R as surrounding images, and displays it on the display device 50. In addition, when the monitoring image generation unit 301C displays the surrounding images on the display device 50, it also displays a shovel image schematically representing the shovel 100 on the display device 50 in order to clearly show the relative positional relationship between the imaging range of the imaging device 40 and the shovel 100. That is, the monitoring image generation unit 301C generates a monitoring image that includes the shovel image and surrounding images arranged around the shovel image in accordance with the relative positional relationship between the shovel 100 and the imaging range of the imaging device 40, and displays it on the display device 50.

[0065] For example, Figures 5A and 5B show an example of a monitoring image displayed on the display device 50 (monitoring image MP1 including a through image) and another example (monitoring image MP2 including a peripheral image EP), respectively.

[0066] As shown in Figure 5A, in this example, the display unit 50A displays a pass-through image from one of the cameras 40B, 40L, or 40R as the monitoring image MP1, as described above.

[0067] In this example, the monitoring image MP1 (through image) is overlaid with a guideline GLa. The guideline GLa represents, for example, a position where the horizontal distance D from the shovel 100 is a predetermined distance D1. This allows operators to understand how far away the monitored object is from the shovel 100 when it is visible (included) in the through image.

[0068] Furthermore, the predetermined distance D1 can be set as appropriate within a range of less than or equal to the predetermined distance D2 corresponding to the monitoring area described later.

[0069] Furthermore, as shown in Figure 5B, in this example, the display unit 50A displays a monitoring image MP2, which includes the shovel image CG and the surrounding image EP arranged around the shovel image CG, as described above. This allows the operator to appropriately understand the positional relationship between the monitoring target shown in the surrounding image EP and the shovel 100.

[0070] The peripheral image EP in this example is a viewpoint transformation image that combines an overhead view image BVP of the surrounding area adjacent to the shovel 100 viewed from directly above, and a horizontal view image HVP of the surrounding area viewed horizontally from the shovel 100, which is positioned around the overhead view image BVP. The peripheral image EP, which is a viewpoint transformation image, is obtained by projecting the images captured by cameras 40B, 40L, and 40R onto a spatial model, and then reprojecting the projected images onto that spatial model onto another two-dimensional plane. The spatial model is the projection target of the captured images in virtual space, and consists of one or more planes or curved surfaces, including planes or curved surfaces other than the plane on which the captured images are located.

[0071] Furthermore, the guideline GLb is superimposed on the monitoring image MP2. Similar to the guideline GLa in the monitoring image MP1 (through image) in Figure 5A, the guideline GLb represents a position where the horizontal distance D from the shovel 100 is a predetermined distance D1. In other words, the guideline GLb is a line that offsets (magnifies) the contour shape of the shovel image CG, which corresponds to a plan view from directly above the shovel 100, by an amount equivalent to a predetermined distance D1 outside the shovel image CG. This allows operators, etc., to understand how far away the monitored object is from the shovel 100, if it is visible in the surrounding image EP, similar to the case of guideline GLa in Figure 5A.

[0072] Returning to Figures 1 to 3, the detection unit 302C detects the object to be monitored within the monitoring area around the shovel 100, specifically, within the monitoring area in the horizontal direction as seen from the shovel 100, that is, along the plane on which the shovel 100 is working (where the lower traveling body 1 is in contact with the ground), based on the image captured by the imaging device 40. Specifically, the detection unit 302C detects the object to be monitored within the monitoring area where the horizontal distance D from the shovel 100 is within a predetermined distance D2 (for example, 5 meters).

[0073] For example, Figure 6 is a schematic diagram showing an example of a monitoring area (monitoring area MA). Specifically, it shows monitoring area MA as viewed from directly above the shovel 100.

[0074] In the diagram, the dotted line L1 and the dashed line L2 correspond to positions where the horizontal distance D from the shovel 100 is a predetermined distance D1 and a predetermined distance D2, respectively.

[0075] As shown in Figure 6, the monitoring area MA (shaded area in the figure) is defined as the range within which the horizontal distance D from the shovel 100 is less than or equal to a predetermined distance D2, which is determined by the imaging ranges AVB, AVL, and AVR defined by the horizontal field of view of cameras 40B, 40L, and 40R, respectively.

[0076] Returning to Figures 1 to 3, for example, the detection unit 302C recognizes the target to be monitored in the captured image by arbitrarily applying various known image processing techniques or machine learning-based classifiers, including artificial intelligence (AI).

[0077] Furthermore, the detection unit 302C can determine (estimate) the location of the recognized surveillance target (hereinafter referred to as "actual location," for example, the foot position of a person being monitored) included in the image captured by the monocular imaging device 40 by applying various known methods.

[0078] For example, the detection unit 302C estimates the horizontal position (hereinafter referred to as "horizontal position") of the recognized monitored object based on its size on the captured image (for example, its height on the captured image). This is because the size of the recognized monitored object on the captured image is correlated to decreasing as the monitored object moves further away from the shovel 100. Specifically, a range of assumed size (for example, a range of assumed human height) can be defined for the monitored object, and therefore, a correlation between the horizontal position of the monitored object, which is within the assumed size range and can be viewed from the shovel 100, and its size on the captured image can be defined in advance. For this reason, the detection unit 302C can estimate the actual location of the recognized monitored object (horizontal position from the shovel 100) based on a map or conversion formula that represents the correlation between the size of the monitored object on the captured image and its horizontal position as viewed from the shovel 100, which is pre-stored in the internal memory of the peripheral monitoring controller 30C, such as an auxiliary storage device.

[0079] Furthermore, for example, the detection unit 302C can estimate the actual location (e.g., the position at the user's feet) of the monitored object by projecting the captured image onto the same plane (homography), assuming that the monitored object is on the same plane as the shovel 100 (or its lower traveling body 1). In this case, a certain part (a certain point) of the captured image is associated with a certain position on the same plane as the shovel 100.

[0080] Furthermore, the detection unit 302C may detect the target to be monitored within the monitoring area around the shovel 100 based on detection information from other sensors that may be mounted on the shovel 100, in addition to or in addition to the image captured by the imaging device 40. For example, the detection unit 302C may detect the target to be monitored within the monitoring area around the shovel 100 based on detection information from other sensors such as a stereo camera, millimeter-wave radar, or LIDAR (Light Detection and Ranging).

[0081] The alarm output unit 303C outputs an alarm to the inside or outside of the cabin 10 when the detection unit 302C detects a monitored object within the monitoring area. This allows the surrounding monitoring controller 30C to make the operator, workers around the shovel 100, supervisors, etc., aware that a monitored object has entered the monitoring area around the shovel 100. Therefore, the surrounding monitoring controller 30C can prompt the operator to check the safety conditions around the shovel 100 and prompt workers, etc., within the monitoring area to evacuate from the monitoring area.

[0082] For example, the alarm output unit 303C outputs an alarm using an auditory method, that is, by sound. Specifically, the alarm output unit 303C outputs a control command to the voice output device 52, causing it to output a warning sound.

[0083] Furthermore, the alarm output unit 303C may vary the pitch, sound pressure, and tone of the warning sound, as well as the sounding period when the warning sound (e.g., a buzzer sound) is sounded periodically, depending on various conditions.

[0084] Furthermore, for example, the alarm output unit 303C outputs an alarm in a visual manner, that is, by displaying it on the display device 50. Specifically, the alarm output unit 303C sends an alarm request to the monitoring image generation unit 301C. In response to the alarm request, the monitoring image generation unit 301C highlights the monitored object included in the monitoring image (through image or surrounding image) displayed on the display device 50, thereby enabling the display device 50 to output an alarm to the operator.

[0085] Further, the warning output unit 303C may output a warning by a tactile method, for example, through a vibration generator that vibrates the driver's seat on which the operator is seated.

[0086] Also, the warning output unit 303C may vary the type (warning level) of the warning according to the positional relationship between the monitoring target detected in the monitoring area by the detection unit 302C and the excavator 100.

[0087] For example, when the monitoring target detected in the monitoring area by the detection unit 302C is present at a position relatively far from the excavator 100, the warning output unit 303C outputs a warning with a relatively low warning level (hereinafter, "warning of attention level") that prompts an operator or the like to pay attention to the monitoring target. Specifically, when the horizontal distance D between the monitoring target detected by the detection unit 302C and the excavator 100 exceeds a predetermined distance D1 (when D1 < D ≤ D2), the warning output unit 303C may output a warning of attention level. Hereinafter, the area in the monitoring area where the distance D from the excavator 100 exceeds a predetermined distance D1 is conveniently referred to as the "attention area". On the other hand, when the monitoring target detected in the monitoring area by the detection unit 302C is present at a position relatively close to the excavator 100, the warning output unit 303C outputs a warning with a relatively high warning level (hereinafter, "warning of alert level") that notifies that the monitoring target is approaching the excavator 100 and the risk level is increasing. Specifically, when the horizontal distance D between the monitoring target detected by the detection unit 302C and the excavator 100 is less than or equal to a predetermined distance D1 (when D ≤ D1), the warning output unit 303C may output a warning of alert level. Hereinafter, the area in the monitoring area where the distance D from the excavator 100 is less than or equal to a predetermined distance D1 is referred to as the "alert area".

[0088] In this case, the alarm output unit 303C may differentiate the pitch, sound pressure, timbre, and sounding cycle of the sound output from the voice output device 52 between a caution level alarm and a warning level alarm. Furthermore, the alarm output unit 303C may differentiate the color, shape, size, presence or absence of flashing, and flashing cycle of markers that highlight the monitored object included in the monitoring image (through image or surrounding image) displayed on the display device 50 between a caution level alarm and a warning level alarm. As a result, the surrounding monitoring controller 30C can allow operators to understand the alarm level, that is, the degree of proximity to the monitored shovel 100, based on the differences in the alarm sound and markers that highlight the monitored object displayed on the display device 50.

[0089] Furthermore, the alarm output unit 303C cancels the alarm output if the monitored object that was detected by the detection unit 302C is no longer detected after the alarm output has started, or if a predetermined cancellation operation is performed by an operator or the like.

[0090] The operation limiting unit 304C restricts the operation of the excavator 100's operating elements when the detection unit 302C detects a target being monitored within the monitoring area. The operating elements subject to this restriction include some or all of the lower traveling body 1, upper slewing body 3, boom 4, arm 5, and bucket 6. For example, the operation limiting unit 304C transmits a command signal (ON signal) to the gate lock valve 54, thereby deconnecting the pilot line to the gate lock valve 54. This prevents the supply of hydraulic fluid (pilot pressure) from the pilot pump (not shown) to the operating device (not shown) that controls the operating elements of the excavator 100. Therefore, even if an operator inputs an operation to the operating device, the operating elements of the excavator 100 are restricted from operating. As a result, even if a target being monitored (e.g., a person) enters the monitoring area around the excavator 100, the operation of the excavator 100 is restricted, suppressing collisions between the excavator 100 and the target being monitored, and ultimately maintaining safety around the excavator 100.

[0091] For example, instead of the gate lock valve 54, a pressure reducing valve is provided to reduce the secondary pilot pressure output from the operating device that operates the operating element, and the operation limiting unit 304C may control the pressure reducing valve. In this case, the pressure reducing valve can disable the operator's operation of the operating device by reducing the secondary pilot pressure output from the operating device to a pressure value below that corresponding to zero operation amount. Therefore, by controlling the pressure reducing valve, the operation limiting unit 304C can, as a result, limit the operation of the operating element of the shovel 100. Alternatively, by controlling the pressure reducing valve, the operation limiting unit 304C may limit the operation of the shovel 100 in a manner that suppresses the operation of various operating elements in response to operator operation more gently than usual, without stopping the operating element of the shovel 100.

[0092] Furthermore, similar to the case of the alarm output unit 303C, the operation limiting unit 304C may set different levels of operation limiting depending on the positional relationship between the monitored object detected by the detection unit 302C and the shovel 100.

[0093] For example, when the monitored object detected within the monitoring area by the detection unit 302C is at a position relatively far from the excavator 100, the operation restriction unit 304C performs operation restriction on the excavator 100 in a mode with a relatively low degree of restriction. Specifically, when the horizontal distance D between the monitored object detected by the detection unit 302C and the excavator 100 exceeds a predetermined distance D1 (when D1 < D ≤ D2), the operation restriction unit 304C may perform operation restriction on the excavator 100 in a mode with a relatively low degree of restriction (for example, making the operating speed of the operating element with respect to the operation amount slower than normal). On the other hand, when the monitored object detected within the monitoring area by the detection unit 302C is at a position relatively close to the excavator 100, the operation restriction unit 304C performs operation restriction on the excavator 100 in a mode with a relatively high degree of restriction. Specifically, when the horizontal distance D between the monitored object detected by the detection unit 302C and the excavator 100 is less than or equal to the predetermined distance D1 (when D ≤ D1), the operation restriction unit 304C may perform operation restriction on the excavator 100 in a mode with a relatively high degree of restriction (for example, stopping the operation of the operating element).

[0094] Further, after the start of the operation restriction of the excavator 100, when the monitored object detected by the detection unit 302C disappears or when the above-described release operation is performed, the operation restriction unit 304C releases the operation restriction of the excavator 100.

[0095] In response to the on-site monitoring command from the support device 200, the transmission unit 305C transmits information regarding the situation around the excavator 100 (peripheral situation information) to the management device 300 through the communication device 60. Thereby, the transmission unit 305C can provide (transmit) the peripheral situation information to the support device 200 via the management device 300.

[0096] In addition, as described above, when the excavator 100 and the support device 200 are P2P-connected, the transmission unit 305C may directly transmit the peripheral situation information to the support device 200.

[0097] For example, the transmitting unit 305C transmits to the management device 300 information acquired by the imaging device 40 (cameras 40B, 40L, 40R), that is, captured images including the surroundings of the shovel 100, as surrounding situation information. Also, as described above, if other sensors such as a stereo camera, millimeter-wave radar, or LIDAR are mounted on the shovel 100 in place of or in addition to the imaging device 40, the transmitting unit 305C may transmit detection information from the other sensors to the management device 300 as surrounding situation information. Furthermore, the transmitting unit 305C may transmit to the management device 300 surrounding images generated by the monitoring image generation unit 301C as surrounding situation information.

[0098] Furthermore, for example, the transmitting unit 305C transmits to the management device 300 surrounding situation information, which is generated based on the information acquired by the imaging device 40 (i.e., the captured image). Specifically, the transmitting unit 305C transmits to the management device 300 information about the monitoring targets around the shovel 100, which is generated based on the captured image from the imaging device 40, and more specifically, information about whether or not the monitoring targets have been detected by the detection unit 302C. In this case, the transmitting unit 305C may transmit information about whether or not the monitoring targets have been detected to the management device 300 regardless of whether or not the monitoring targets have been detected by the detection unit 302C, or it may transmit information to the management device 300 indicating that the monitoring targets have been detected only when the monitoring targets have been detected.

[0099] Furthermore, for example, the transmitting unit 305C transmits information acquired by the aforementioned microphone (i.e., audio information about the surroundings of the shovel 100) to the management device 300 as ambient situation information.

[0100] Furthermore, if there are multiple types of information (for example, captured images and audio information) that are predetermined as ambient situation information, the transmitting unit 305C may transmit all of the multiple types of information to the management device 300, or it may transmit only some of the types of information to the management device 300. For example, the information to be transmitted among the multiple types of information may be set by a user such as an operator.

[0101] Furthermore, the transmission unit 305C may store the surrounding situation information transmitted to the support device 200 via the management device 300 through the communication device 60 as log information in the storage unit 308C or the like. This allows the operator of the shovel 100 to access the log information in a predetermined manner and confirm it afterward by displaying it on the display device 50, etc.

[0102] The command acquisition unit 306C acquires (receives) on-site monitoring commands from the support device 200 via the management device 300 through the communication device 60.

[0103] Furthermore, the command acquisition unit 306C determines whether the acquired field monitoring command is a legitimate field monitoring command. Specifically, the command acquisition unit 306C may determine whether it is a legitimate field monitoring command based on predetermined authentication information included in the field monitoring command. This is to prevent the excavator 100 from being remotely controlled by a malicious third party. The command acquisition unit 306C may also determine whether it is a legitimate field monitoring command based on whether the excavator 100 matches the excavator that is the target of remote control specified in the field monitoring command. This is to prevent the excavator 100 from being remotely controlled in situations such as when a field monitoring command that should have been sent to a different excavator is mistakenly sent to the excavator 100 due to a bug in the program of the management device 300. Furthermore, the command acquisition unit 306C may determine whether the source of the field monitoring command corresponds to (is included in) a support device 200 that has been authorized for remote control by the authorization operation unit 62, based on the authorization terminal information 3082C stored in the storage unit 308C. This is to prevent situations in which the shovel 100 could be remotely controlled through a support device 200 that is not authorized for remote control.

[0104] Furthermore, if the command acquisition unit 306C is a legitimate field monitoring command, it performs processing prior to the transmission of surrounding situation information by the transmission unit 305C.

[0105] Specifically, if the excavator 100 is stopped (i.e., the engine 11 is stopped), the command acquisition unit 306C activates the start / stop unit 307C, which is in sleep mode, and activates the functions related to on-site monitoring of the excavator 100 (hereinafter referred to as "on-site monitoring functions").

[0106] Furthermore, the command acquisition unit 306C (an example of an internal notification unit) notifies, via the display device 50, that when the shovel 100 is in operation (i.e., the engine 11 is running), that surrounding monitoring information of the shovel 100 will be transmitted externally (to the support device 200, the management device 300). This allows operators to proceed with their work on the shovel 100 while being aware that information about the surrounding conditions of the shovel 100 they are operating is being transmitted externally and can be viewed from outside the work site.

[0107] Then, once the processing in the previous stage is complete, the command acquisition unit 306C instructs the transmission unit 305C to transmit surrounding situation information based on the on-site monitoring command.

[0108] The start / stop unit 307C checks various states of the battery 64 that supplies power to the field monitoring function in response to a command from the command acquisition unit 306C. For example, the start / stop unit 307C acquires the charge state of the battery 64, specifically the remaining capacity C (an example of the amount of power that can be supplied), and determines whether the remaining capacity C exceeds a predetermined threshold Cth1 (an example of a first threshold). The predetermined threshold Cth1 is predetermined as the minimum remaining capacity necessary to ensure the start of the field monitoring function and its continuous operation for a certain period after start. The predetermined threshold Cth1 may also be variable depending on the period during which field monitoring is performed as included in the field monitoring command (for example, the period from the start time to the end time). In this case, the remaining capacity C of the battery 64 may be acquired from a battery sensor mounted on the terminals of the battery 64, or it may be calculated from the measured values ​​of a voltage sensor or current sensor that measures the voltage and current of the battery 64.

[0109] Furthermore, the field monitoring unit 110 may be configured to be able to receive power from another power source in place of, or in addition to, the battery 64 when the shovel 100 is stopped. The other power source may include a replaceable primary battery. The other power source may also include a capacitor that can be charged with power from a predetermined generator of the shovel 100 (for example, a generator that generates power using the power of the engine 11 while the shovel 100 is running). The other power source may also include a predetermined power generation means (for example, a fuel cell or an alternator that generates power using the power of the engine 11 which is temporarily started when the shovel 100 is stopped).

[0110] The start / stop unit 307C (an example of a start unit) starts up part or all of the field monitoring unit 110 corresponding to the stopped field monitoring function, according to the settings of the field monitoring command, when the remaining capacity C of the battery 64 exceeds a predetermined threshold Cth1. The field monitoring unit 110 (an example of a monitoring unit) includes, for example, an imaging device 40 (cameras 40B, 40L, 40R), a monitoring image generation unit 301C, a detection unit 302C, and a transmission unit 305C. Specifically, the start / stop unit 307C sends a command signal to the relay 42 (relays 42B, 42L, 42R) and connects it, thereby starting the power supply from the battery 64 to the imaging device 40 (cameras 40B, 40L, 40R) and starting the imaging device 40. In this case, the start / stop unit 307C may activate a portion of the imaging device 40 (one or two of the cameras 40B, 40L, and 40R) by connecting one or two of the relays 42B, 42L, and 42R according to the settings of the on-site monitoring command. The start / stop unit 307C also activates the monitoring image generation unit 301C, the detection unit 302C, and the transmission unit 305C by performing predetermined processing within the peripheral monitoring controller 30C.

[0111] For example, the start / stop unit 307C has a normal mode in which the entire field monitoring unit 110 is operated, and an energy-saving mode in which only a part of the field monitoring unit 110 is operated, limiting the power consumption of the field monitoring unit 110 to a relatively low level. The start / stop unit 307C may start the field monitoring unit 110 in normal mode when the remaining capacity C of the battery 64 is relatively high within a range exceeding a predetermined threshold Cth1, and may start the field monitoring unit 110 in energy-saving mode when it is relatively low. In addition, the start / stop unit 307C may switch from normal mode to energy-saving mode after starting the field monitoring unit 110 in normal mode if the remaining capacity C of the battery 64 becomes relatively low.

[0112] Furthermore, if the surrounding situation information transmitted from the shovel 100 to the support device 200 does not include surrounding images generated by the monitoring image generation unit 301C, the monitoring image generation unit 301C may be omitted from the site monitoring unit 110. For example, if the surrounding situation information only includes information regarding whether or not a monitored target has been detected by the detection unit 302C, the monitoring image generation unit 301C may be omitted from the site monitoring unit 110. In addition, the site monitoring unit 110 may include an alarm output unit 303C. This allows, for example, if a person (suspicious person) as a monitored target is detected by the detection unit 302C while the shovel 100 is stopped, an alarm can be output by the voice output device 52 to notify the surrounding area of ​​the presence of the suspicious person.

[0113] On the other hand, the start / stop unit 307C determines that the field monitoring function cannot be started if the remaining capacity C of the battery 64 is less than or equal to a predetermined threshold Cth1, and does not start the field monitoring function. The start / stop unit 307C (an example of the first external notification unit) may then send a notification indicating that the field monitoring function cannot be started (hereinafter referred to as the "startup failure notification") to the support device 200 via the management device 300, as information regarding whether or not the field monitoring unit 110 can be started. This allows the user to recognize, via the support device 200, that the field monitoring function of the shovel 100 cannot be started.

[0114] Furthermore, the start / stop unit 307C may attempt to start the field monitoring function while it is stopped without checking the various states of the battery 64. If the start / stop unit 307C is unable to start the field monitoring function, it may send a notification of failure to start to the support device 200 via the management device 300 through the communication device 60. Also, if the field monitoring function is able to be started, the start / stop unit 307C may send a notification to the support device 200 via the management device 300 through the communication device 60 indicating that the field monitoring function is able to be started. In addition, the start / stop unit 307C may send information regarding the various states of the battery 64 (for example, remaining capacity C corresponding to the charge state; hereinafter referred to as "power status information") to the support device 200 via the management device 300 as information regarding whether or not the field monitoring function (field monitoring unit 110) is able to be started. This is because the user may be able to determine whether or not the field monitoring function can be started by checking the information regarding the various states of the battery 64 through the support device 200.

[0115] Furthermore, the start / stop unit 307C (an example of a stop unit) stops the on-site monitoring unit 110, which was started while the shovel 100 was stopped, once predetermined conditions are met after the on-site monitoring unit 110 has been started.

[0116] For example, the start / stop unit 307C stops the field monitoring unit 110 when it is the predetermined termination time, specifically when a field monitoring termination command is received via the management device 300 by the communication device 60, or when the termination time set by the field monitoring command is reached.

[0117] Also, for example, the start / stop unit 307C determines whether or not the remaining capacity C of the battery 64 has decreased to a predetermined threshold value Cth2 (<Cth1). When the remaining capacity C of the battery 64 has decreased to a level below the predetermined threshold value Cth2 (an example of the second threshold value), the start / stop unit 307C stops the on-site monitoring unit 110 regardless of the end timing. This is because if the remaining capacity C of the battery 64 decreases too much, there is a possibility that the engine 11 cannot be started when the next excavator 100 is started. Then, the start / stop unit 307C (an example of the second external notification unit) transmits, via the communication device 60, a notification (hereinafter, "monitoring continuation impossible notification") indicating that the on-site monitoring function (on-site monitoring unit 110) is continuously inoperable, as information regarding whether or not the on-site monitoring unit 110 can continue to operate, to the support device 200 via the management device 300. As a result, the user can recognize, through the support device 200, that the on-site monitoring function of the excavator 100 cannot continue to operate and that the on-site monitoring function has been stopped.

[0118] Furthermore, the start / stop unit 307C may, via the communication device 60, send a notification to the support device 200 via the management device 300 indicating that the field monitoring function (field monitoring unit 110) is capable of continuing to operate, even if the field monitoring function is capable of continuing to operate. In addition, the start / stop unit 307C may send information regarding various states of the battery 64 (for example, remaining capacity corresponding to the charge state) to the support device 200 via the management device 300 as information regarding whether or not the field monitoring function (field monitoring unit 110) is capable of continuing to operate. This is because the user may be able to determine whether or not the field monitoring function is capable of continuing to operate by checking information regarding various states of the battery 64 via the support device 200. Furthermore, the start / stop unit 307C may notify the support device 200 via the management device 300 of information regarding whether or not the field monitoring unit 110 is operational, regardless of whether or not the field monitoring unit 110 is started or capable of continuing to operate. For example, the start / stop unit 307C may notify the support device 200 via the management device 300 of information regarding whether the field monitoring unit 110 is operational, in response to an acquisition request received from the support device 200 via the communication device 60. Specifically, the start / stop unit 307C may notify the support device 200 via the management device 300 of the remaining capacity C of the battery 64, the operating time of the field monitoring function (field monitoring unit 110) based on the remaining capacity C of the battery 64, etc. In this case, the support device 200 displays the information notified from the shovel 100 via the management device 300 on the display device 240. The user can request the display of the remaining capacity C of the battery 64 and the operating time of the field monitoring function via the operation device 230, thereby causing the support device 200 to send an acquisition request for this information to the shovel 100 via the management device 300. For example, the support device 200 may display the remaining capacity C of the battery 64 and the operating time of the field monitoring function on the display device 240 using a bar graph or the like. This allows the user to easily check information regarding whether or not the field monitoring function is operational through the support device 200.

[0119] <Configuration of support device> The support device 200 includes a control device 210, a communication device 220, an operating device 230, and a display device 240.

[0120] The control device 210 controls various operations of the support device 200. The control device 210 includes a command unit 2101, an information acquisition unit 2102, and a notification unit 2103, which are functional units realized by executing one or more programs stored in ROM or a non-volatile auxiliary storage device on the CPU. The control device 210 also includes a storage unit 2100, which is a storage area defined in a non-volatile internal memory such as an auxiliary storage device.

[0121] The communication device 220 is any device that communicates with the outside world of the support device 200, such as the management device 300, via the communication network NW. The communication device 220 is, for example, a mobile communication module that supports mobile communication standards such as LTE, 4G, and 5G.

[0122] The operating device 230 receives various operations from the user on the support device 200. The operating device 230 includes, for example, hardware-based operating parts such as buttons, keyboards, mice, touchpads, and touch panels implemented on the display device 50. Alternatively, the operating device 230 may be a combination of hardware-based operating parts such as touch panels implemented on the display device 240 and software-based operating parts such as button icons on the operation screen displayed on the display device 240.

[0123] The display device 240 displays various information images. The display device 240 is, for example, a liquid crystal display or an organic EL display.

[0124] The command unit 2101 (an example of a transmission unit) transmits a field monitoring command to the management device 300 via the communication device 220 when a predetermined operation is performed by the user, through the operating device 230, with the shovel 100 as the destination.

[0125] For example, Figure 7 (Figures 7A and 7B) shows an example of an operation screen (hereinafter referred to as the "command transmission operation screen") for sending field monitoring commands, which is displayed on the display device 240. Specifically, Figure 7A shows an example of an operation screen (command transmission operation screen 700) for selecting the excavator 100 to which the field monitoring command is to be sent, that is, the excavator 100 that will be remotely controlled. Figure 7B shows an example of an operation screen (command transmission operation screen 710) for setting specific field monitoring commands.

[0126] First, as shown in Figure 7A, the command transmission operation screen 700 displays a candidate list 701 of excavators 100 that can be remotely controlled for surrounding area monitoring (site monitoring) via the support device 200. The candidate list 701 is displayed based on the remotely controlled machine information 2100A stored in the storage unit 2100. The remotely controlled machine information 2100A in the storage unit 2100 contains the latest information on excavators 100 that can be remotely controlled for site monitoring by the support device 200, and is updated as appropriate by the latest remotely controlled machine information distributed from the management device 300 to the support device 200.

[0127] The user scrolls through the candidate list 701 via a touch panel or the like implemented on the display device 240. This allows the user to find the shovel 100 that they want to remotely control.

[0128] The user then places a checkmark (black circle in the diagram) on the far left of the item column corresponding to the shovel 100 to be remotely controlled from the candidate list 701 via a touch panel or similar device. This allows the user to select the shovel 100 to be remotely controlled, that is, the shovel 100 to which the on-site monitoring command will be sent. At this time, one shovel 100 may be selected, or multiple shovels 100 may be selected. In this example, two shovels 100, "Shovel A" and "Shovel B," have been selected.

[0129] Finally, the user operates a button icon 702 at the bottom of the command transmission operation screen 700, which is marked with the word "Confirm," via a touch panel or similar device. This confirms the selection of the shovel 100 to be remotely controlled, that is, the shovel 100 to which the field monitoring command is sent, and allows the display content of the display device 240 to transition to the command transmission operation screen 710.

[0130] Next, as shown in Figure 7B, the command transmission operation screen 710 includes various setting fields 711 to 715 related to on-site monitoring commands for the selected "Shovel A".

[0131] In the setting field 711, which is labeled "Time Setting," settings are made for the time period during which the shovel 100 (in this example, "Shovel A") will perform site monitoring. In other words, in the setting field 711, the user can reserve the shovel 100 to perform perimeter monitoring based on a site monitoring command. This allows the shovel 100 to automatically start perimeter monitoring according to the reservation. In this example, the start time ("6:00") and end time ("8:00") are set in the setting field 711. Alternatively, the setting field 711 may also contain information indicating that perimeter monitoring by the shovel 100 will start immediately upon transmission of a site monitoring command. In other words, in the setting field 711, the user can set the immediate start of perimeter monitoring by the shovel 100. Furthermore, multiple time periods during which the shovel 100 will perform site monitoring may be set in the setting field 711. For example, in the settings field 711, multiple specific time slots defined by start and end times may be set, or the start time of the first on-site monitoring, the period during which on-site monitoring is performed (e.g., 30 minutes), and the time interval of regular start timings (e.g., every hour) may be set. In addition, in the settings field 711, it may be possible to set the time slots during which the shovel 100 performs on-site monitoring for multiple days. For example, in the settings field 711, it may be possible to select multiple specific dates, or it may be possible to select multiple regularly repeating days such as every day, specific days of the week (e.g., Monday to Friday), or specific days of each month. This improves user convenience because the user does not need to perform the operation of sending an on-site monitoring command through the support device 200 each time they want the shovel 100 to perform on-site monitoring.

[0132] Furthermore, assuming that the shovel 100 immediately starts on-site monitoring (activates the on-site monitoring function) upon receiving the on-site monitoring command, the setting field 711 may be used to set the timing at which the on-site monitoring command is sent (for example, a specific date and time) and the period during which the surrounding area monitoring is performed (for example, 3 hours). In this case, as described above, multiple dates and times for which the on-site monitoring command is sent may be set in the setting field 711. For example, multiple specific dates and times for which the on-site monitoring command is sent may be specified in the setting field 711, or a time interval (for example, every hour) for the periodic timing at which the on-site monitoring command is sent may be set. As a result, as described above, the user does not need to perform the operation to send an on-site monitoring command through the support device 200 each time they want the shovel 100 to perform surrounding area monitoring, thus improving user convenience.

[0133] In setting field 712, which is labeled with the text information "Camera Image Acquisition," settings are made regarding the acquisition of images captured by the imaging device 40 of the shovel 100. Specifically, in setting field 712, it is possible to set whether or not to acquire images captured by the imaging device 40, and the type of image to acquire (video or still image). In this example, in setting field 712, the acquisition of images captured by the imaging device 40 is enabled ("ON"), and the type of image to acquire is set to "video."

[0134] In setting field 713, which is labeled with the text "Acquire detection information," information regarding the acquisition of information about the monitored objects around the shovel 100, that is, information regarding whether or not the monitored objects have been detected, is set. In this example, setting field 713 is set to "ON" for acquiring information regarding the detection of monitored objects.

[0135] In the setting field 714, which is labeled "Camera Selection," the camera selected from cameras 40B, 40L, and 40R is used to acquire the captured images. In this example, camera 40B ("rear") and camera 40R ("right") are selected ("ON") as the cameras to acquire the captured images, while camera 40L is not selected ("OFF") as the camera to acquire the captured images.

[0136] In the setting field 715, which is labeled with the text "Log," settings are made regarding whether or not to record the surrounding situation information transmitted from the shovel 100 to the support device 200 as a log in the shovel 100 or the management device 300. In this example, the setting field 715 is set to record the surrounding situation information transmitted from the shovel 100 to the support device 200 as a log ("ON"). In this case, the log may be stored, for example, in the storage unit 308C of the shovel 100 or in the storage unit 3100 of the management device 300, which will be described later.

[0137] Furthermore, the command transmission operation screen 710 may display setting fields 711 to 715 related to "Shovel B" when a scroll operation is performed.

[0138] The user, using the touch panel or similar device, configures the settings in fields 711 to 715 as appropriate for the selected shovels 100 (Shovel A and Shovel B in this example) on the command transmission operation screen 700. Then, the user operates the button icon 716 at the bottom of the command transmission operation screen 710, which is labeled "Send," using the touch panel or similar device. As a result, the command unit 2101 transmits a field monitoring command, including the identification information of the selected shovels 100 on the command transmission operation screen 700 and the settings in fields 711 to 715 for each selected shovel 100, to the management device 300 via the communication device 220. In this way, the user can transmit a field monitoring command from the support device 200 to the shovels 100 (Shovel A and Shovel B in this example) via the management device 300 through the command transmission operation screens 700 and 710.

[0139] Returning to Figures 1 to 3, the command unit 2101 may, via the operating device 230, send a field monitoring termination command to the management device 300 via the communication device 220 when a predetermined operation is performed by the user, with the excavator 100 as the destination. This allows field monitoring of the excavator 100 to be terminated regardless of the settings (time settings) of the field monitoring command described above.

[0140] The information acquisition unit 2102 (an example of the first acquisition unit) acquires (receives) surrounding condition information transmitted from the shovel 100 via the management device 300 through the communication device 220.

[0141] Furthermore, the information acquisition unit 2102 (an example of a second acquisition unit) acquires (receives) information transmitted from the shovel 100 via the management device 300 through the communication device 220 regarding whether the shovel 100's field monitoring function (field monitoring unit 110) can be started (for example, a notification that it cannot be started). The information acquisition unit 2102 also acquires information transmitted from the shovel 100 via the management device 300 through the communication device 220 regarding whether the shovel 100's field monitoring function (field monitoring unit 110) can continue to operate (for example, a notification that monitoring cannot be continued). In addition, the information acquisition unit 2102 may acquire information regarding various states of the battery 64, i.e., power status information (for example, remaining capacity C), as information regarding whether the shovel 100's field monitoring function can be started or whether the field monitoring function can continue to operate, as described above.

[0142] The notification unit 2103 (an example of the first notification unit and the second notification unit) notifies the user of various information, such as surrounding condition information, acquired by the information acquisition unit 2102. Specifically, the notification unit 2103 notifies (provides) the user of various information, such as surrounding condition information, by displaying it on the display device 240. The notification unit 2103 may also notify the user of various information, such as surrounding condition information, by voice. This allows the user to confirm various information, such as surrounding condition information, that has been returned from the excavator 100 via the management device 300 in response to the on-site monitoring command.

[0143] For example, Figure 8 shows an example of a screen (hereinafter referred to as the "site situation notification screen") (site situation notification screen 800) that is displayed on the display device 240 of the support device 200 to notify the user of surrounding situation information.

[0144] The site status notification screen 800 displays surrounding situation information acquired from the excavator 100 ("Excavator A") to which the site monitoring command was sent via the command transmission operation screens 700 and 710 in Figures 7A and 7B.

[0145] Specifically, the display area 811, which is accompanied by the text information "camera image," displays an image captured by the imaging device 40 of the shovel 100 ("Shovel A"). The image in the display area 811 may be, for example, an image representing the surroundings of the shovel 100 at the present time (approximately real-time), or an image representing the surroundings of the shovel 100 at a point in the past.

[0146] Furthermore, the display area 812, which includes the text information "Detection Information," displays information regarding whether or not a target of surveillance (specifically, a suspicious person) has been detected around the shovel 100. In this example, the captured image displayed in display area 811 does not show any suspicious persons, and the text information indicating that no suspicious persons were detected ("No suspicious persons detected") is displayed.

[0147] In this way, the user can understand the conditions of the work site outside of working hours by using the images captured by the imaging device 40 (display area 811) included in the surrounding conditions information. For example, if it rains before work begins, the user can have the support device 200 send a site monitoring command to the shovel 100 at the work site, allowing the support device 200 to confirm whether the work site is in a condition where work can be performed without having to go to the work site. In addition, the user can check whether a monitored target (suspicious person) has entered the work site outside of working hours by using the information (display area 812) regarding the detection of a monitored target included in the surrounding conditions information. For example, the user can have the support device 200 send a site monitoring command specifying a nighttime period to the shovel 100 at the work site, allowing the support device 200 to check whether a monitored target has been detected at night and to understand the intrusion of a suspicious person, etc.

[0148] Furthermore, for example, Figure 9 shows the parking situation of Shovel 100 (Shovel 100A, 100B) at a work site. Hereafter, in this example, the right direction in the figure will be referred to as the X1 direction, the left direction as the X2 direction, the upward direction as the Y1 direction, and the downward direction as the Y2 direction.

[0149] In this example, a shovel 100A is parked near buildings 902 and 903 located at the corners corresponding to the ends in the X1 and Y2 directions of a roughly rectangular site 901 of the work site 900, with the X1 and X2 directions being the longer sides, with the rear of the upper rotating body 3 facing in the X2 direction.

[0150] When the shovel 100A is stopped, cameras 40B and 40R are activated, and cameras 40B and 40R can acquire images that allow confirmation of the condition of approximately half of the area in the Y1 direction of the site 901, based on their imaging ranges AVB and AVR.

[0151] In this example, the shovel 100B is parked near the center of the X1 and X2 directions at the Y2 direction end of the site 901, with the rear of the upper rotating body 3 facing the Y1 direction.

[0152] When the shovel 100B is stopped, cameras 40B, 40L, and 40R are activated, and cameras 40B, 40L, and 40R can acquire images that allow confirmation of the condition of approximately half of the area in the Y2 direction of the site 901, according to their imaging ranges AVB, AVL, and AVR.

[0153] Therefore, in this example, the user can check the situation throughout the entire work site 900 by having the support device 200 send on-site monitoring commands to shovels 100A and 100B.

[0154] In this way, users such as workers and supervisors can check the overall status of the work site from the outside by having the support device 200 send site monitoring commands to the excavators 100 in accordance with the parking status of one or more excavators 100 at the work site. In addition, by appropriately arranging the placement of one or more excavators 100 at the work site, users can check the overall status of the work site from the outside based on the surrounding situation information transmitted from the excavators 100 to the support device 200 via the management device 300.

[0155] Returning to Figures 1-3, the user can also check the status of the work site in progress by using the images captured by the imaging device 40 included in the surrounding situation information. For example, a user such as a supervisor in charge of multiple work sites can have the support device 200 send a site monitoring command to the excavator 100 at a geographically distant work site, allowing the support device 200 to check the status of the work site in progress at that geographically distant location. In addition, the user can check the presence and frequency of intrusions of monitored objects (workers) into the monitoring area adjacent to the excavator 100 at the work site in progress by using the information regarding the presence or absence of detection of monitored objects included in the surrounding situation information. For example, a user such as a supervisor in charge of multiple work sites can have the support device 200 send a site monitoring command to the excavator 100 at a geographically distant work site, allowing the support device 200 to check the safety status of the work site in progress at that geographically distant location. Therefore, users can take measures such as sending warning notices to work sites where workers frequently enter the monitoring area adjacent to the shovel 100, or going to the work site to directly issue warnings.

[0156] <Configuration of the control device> The management device 300 includes a control device 310 and a communication device 320.

[0157] The control device 310 controls various operations of the management device 300. The control device 310 includes a relay unit 3101 as a functional unit realized by executing one or more programs stored in ROM or a non-volatile auxiliary storage device on the CPU. The control device 310 also includes a storage unit 3100 as a storage area defined in the non-volatile internal memory of an auxiliary storage device, for example.

[0158] The communication device 320 is any device that communicates with the outside world of the management device 300, such as the shovel 100 and the support device 200, via the communication network NW.

[0159] The relay unit 3101 relays communication between the shovel 100 and the support device 200 via the communication device 320. For example, when the relay unit 3101 receives a field monitoring command from the support device 200 via the communication device 320, it transmits (forwards) the field monitoring command to the shovel 100 designated as the destination in the field monitoring command. Also, for example, when the relay unit 3101 receives surrounding condition information, etc., from the shovel 100 via the communication device 320, it transmits (forwards) the surrounding condition information, etc., to the support device 200 designated as the destination in the surrounding condition information, etc.

[0160] Furthermore, if the control device 310 receives surrounding condition information transmitted from the shovel 100 to the support device 200 via the communication device 320, it may store the surrounding condition information as log information in the storage unit 3200. This allows the user to use the support device 200 to access the management device 300 in a predetermined manner and check the log information.

[0161] The memory unit 3200 stores candidate terminal information 3100A and machine / terminal compatibility information 3101B.

[0162] Candidate terminal information 3100A is information about candidate support devices 200 that perform remote control related to on-site monitoring of the excavator 100, either through the support device 200 or registered in the management device 300. When the candidate terminal information 3100A is updated, it is transmitted to the excavator 100 via the communication device 320. This ensures that the candidate terminal information 3081C of the excavator 100 is kept up-to-date.

[0163] The machine / terminal correspondence information 3100B is information regarding the correspondence between the excavator 100 and the support device 200 that is authorized to remotely control the site for monitoring purposes. The machine / terminal correspondence information 3100B is updated based on information about the authorized support device 200, which is notified by the excavator 100 each time the excavator 100 is authorized to remotely control the site for monitoring purposes by the support device 200. For example, based on the machine / terminal correspondence information 3100B, information about the excavator 100 authorized to remotely control the surrounding area for monitoring purposes for each support device 200, i.e., remotely controlled machine information, can be generated. Therefore, each time the machine / terminal correspondence information 3100B is updated, new remotely controlled machine information is generated and transmitted to the support device 200. As a result, the remotely controlled machine information 2100A stored in the memory unit 2100 of the support device 200 can be updated and kept up-to-date.

[0164] [Details of the operation of the on-site monitoring system] Next, we will explain the details of the operation of the field monitoring system SYS with reference to Figure 10 (Figures 10A to 10D).

[0165] <Processing flow by the shovel's peripheral monitoring controller> Figures 10A and 10B are flowcharts illustrating an example of the process related to on-site monitoring by the peripheral monitoring controller 30C of the shovel 100. The process shown in this flowchart is executed when an on-site monitoring command is received from the support device 200 via the management device 300 by the communication device 60.

[0166] In step S102, the command acquisition unit 306C acquires a field monitoring command from the receiving buffer, etc., and determines whether it is time to start peripheral monitoring based on the settings of the field monitoring command. Specifically, the command acquisition unit 306C determines whether the start time of the time setting included in the field monitoring command indicates an immediate start, or whether the current time has reached the start time of the time setting included in the field monitoring command. If it is time to start peripheral monitoring, the command acquisition unit 306C proceeds to step S104; otherwise, it waits until it is time to start peripheral monitoring (repeating the processing of this step).

[0167] In step S104, the command acquisition unit 306C determines whether the shovel 100 is in operation (i.e., whether the engine 11 is running). If the shovel 100 is not in operation, i.e., stopped, the command acquisition unit 306C proceeds to step S105; if the shovel 100 is in operation, the unit proceeds to step S116.

[0168] In step S105, the start / stop unit 307C starts up in response to a command from the command acquisition unit 306C and determines whether the remaining capacity C of the battery 64 exceeds a predetermined threshold Cth1. If the remaining capacity C of the battery 64 exceeds the predetermined threshold Cth1, the start / stop unit 307C proceeds to step S106. On the other hand, if the remaining capacity C of the battery 64 does not exceed the predetermined threshold Cth1, that is, if it is less than or equal to the predetermined threshold Cth1, the start / stop unit 307C proceeds to step S107 and sends a notification of inability to start to the support device 200 via the management device 300 through the communication device 60, and terminates the current process.

[0169] In step S106, the start / stop unit 307C starts the field monitoring unit 110 according to the settings of the field monitoring command acquired by the command acquisition unit 306C. For example, if the field monitoring command selects only some of the cameras 40B, 40L, and 40R as the cameras from which to acquire captured images (for example, only some of the setting fields 713 on the command transmission operation screen 710 in Figure 7B are set to "ON"), the start / stop unit 307C may start only the selected cameras. Also, for example, if the field monitoring command does not request the acquisition of information regarding the detection of a monitored object (for example, setting field 713 on the command transmission operation screen 710 in Figure 7B is set to "OFF"), the start / stop unit 307C may not start the detection unit 302C. Also, for example, if the field monitoring command does not request the acquisition of surrounding images generated by the monitoring image generation unit 301C, the start / stop unit 307C may not start the monitoring image generation unit 301C. As a result, only the functions necessary to acquire surrounding situation information and transmit it to the support device 200 are activated, thus suppressing unnecessary energy consumption of the shovel 100 (specifically, a decrease in the remaining capacity of the battery 64). Furthermore, even if the field monitoring unit 110 of the shovel 100 is activated, the shovel 100 remains stopped and the engine 11 is not started. Therefore, situations such as the engine 11 starting in the early morning or at night, causing noise problems, can be suppressed. In addition, situations in which various operating elements of the shovel 100 (specifically, the lower travel body 1, the upper slewing body 3, attachments, etc.) operate when no operator is present, can be suppressed, thereby ensuring the safety of the shovel 100.

[0170] Furthermore, in step S106, the start / stop unit 307C may activate not only the field monitoring unit 110 but also the entire function of the shovel 100. In other words, the start / stop unit 307C may activate the shovel 100 by starting the engine 11 or by starting the power supply from the battery or an external commercial power source. In particular, if the shovel 100 is an electric shovel, it is well-suited to activating the entire function because it produces very little noise and has relatively little energy loss, similar to the engine 11.

[0171] In step S108, the field monitoring unit 110 starts field monitoring according to the settings of the field monitoring command. Specifically, the field monitoring unit 110 acquires surrounding situation information corresponding to the settings of the field monitoring command and starts the process of transmitting surrounding situation information to the management device 300 for use with the support device 200 via the communication device 60.

[0172] In step S110, the start / stop unit 307C determines whether or not it is the end time for on-site monitoring. Specifically, the start / stop unit 307C determines whether or not the end time set in the on-site monitoring command has been reached, or whether or not an on-site monitoring end command has been received from the support device 200 via the management device 300 by the communication device 60. If it is not the end time for on-site monitoring, the start / stop unit 307C proceeds to step S112; if it is the end time, it proceeds to step S114.

[0173] In step S112, the start / stop unit 307C determines whether the battery 64 has stopped when its remaining capacity C falls below a predetermined threshold Cth2. If the battery 64's remaining capacity C has fallen below the predetermined threshold Cth2, the start / stop unit 307C proceeds to step S113; otherwise, it returns to step S110.

[0174] In step S113, the start / stop unit 307C transmits a notification that monitoring cannot be continued to the support device 200 via the management device 300 through the communication device 60.

[0175] In step S114, the field monitoring unit 110 completes field monitoring (i.e., acquisition of surrounding situation information and transmission to the support device 200), and the start / stop unit 307C stops the field monitoring unit 110, thus ending this process.

[0176] Meanwhile, in step S116, the command acquisition unit 306C notifies the operator, etc., via the display device 50, that it will begin transmitting surrounding situation information to the outside.

[0177] In step S118, the field monitoring unit 110 starts field monitoring according to the settings of the field monitoring command, similar to step S108.

[0178] In step S120, the start / stop unit 307C determines whether it is the end of the on-site monitoring time, similar to step S110. If it is the end of the on-site monitoring time, the start / stop unit 307C proceeds to step S122. If it is not the end of the monitoring time, the process of this step is repeated until the end of the monitoring time is reached.

[0179] In step S122, the field monitoring unit 110 completes field monitoring (i.e., acquiring surrounding situation information and transmitting it to the support device 200), and terminates this process.

[0180] <Processing flow by the control device of the support device> Figures 10C and 10D are flowcharts illustrating an example of the on-site monitoring process performed by the control device 210 of the support device 200. The process shown in this flowchart is executed when the on-site monitoring command transmission operation is performed.

[0181] In step S202, the command unit 2101 transmits a field monitoring command addressed to a specific excavator 100 to the management device 300 via the communication device 220.

[0182] In step S204, the notification unit 2103 determines whether it is the moment immediately before the start of on-site monitoring (for example, a few minutes before the start time set in the on-site monitoring command). If it is the moment immediately before the start of on-site monitoring, the notification unit 2103 proceeds to step S206; otherwise, it waits until the moment immediately before the start of on-site monitoring arrives (repeating the processing of this step).

[0183] In step S205, the notification unit 2103 determines whether or not a startup failure notification has been received from the support device 200 via the management device 300 through the communication device 220. If no startup failure notification has been received, the notification unit 2103 proceeds to step S206. On the other hand, if a startup failure notification has been received, the notification unit 2103 proceeds to step S207, where it notifies the user via the display device 240 that the field monitoring function of the shovel 100 cannot be activated, and terminates the process.

[0184] In step S206, the notification unit 2103 notifies the user via the display device 240 to confirm that the display of surrounding situation information on the display device 240 has started.

[0185] In step S208, if the information acquisition unit 2102 receives an operation via the operating device 230 agreeing to start displaying the surrounding situation information on the display device 240, it proceeds to step S210. If the operation is not performed, it waits until it is performed (the process of this step is repeated).

[0186] Furthermore, it is possible that the operation to consent to the start of display may not be performed until the reception of surrounding situation information has finished (i.e., the on-site monitoring of Shovel 100 has finished). Therefore, the process according to this flowchart may be forcibly terminated when the end time set in the on-site monitoring command arrives.

[0187] In step S210, the information acquisition unit 2102 transmits a signal to the management device 300 via the communication device 220 requesting information about the surrounding conditions from the shovel 100, which is the destination of the on-site monitoring command.

[0188] In step S212, the information acquisition unit 2102 determines whether or not the reception of surrounding condition information from the shovel 100 via the management device 300 through the communication device 220 has started. If the information acquisition unit 2102 has started receiving surrounding condition information from the shovel 100, it proceeds to step S214; otherwise, it waits until it starts (repeating the process of this step).

[0189] Furthermore, for example, the reception of surrounding condition information may not start for some reason, such as a communication failure. Therefore, the information acquisition unit 2102 may retry sending a signal requesting surrounding condition information from the shovel 100 to the management device 300 via the communication device 220 multiple times. In addition, the process according to this flowchart may be forcibly terminated after a certain amount of time has elapsed since the transmission of the signal requesting surrounding condition information from the shovel 100.

[0190] In step S214, the notification unit 2103 starts displaying the surrounding situation information acquired by the information acquisition unit 2102 on the display device 240.

[0191] In step S215, the notification unit 2103 determines whether or not a notification of inability to continue monitoring has been received from the support device 200 via the management device 300 through the communication device 220. If the notification unit 2103 has not received a notification of inability to continue monitoring, it proceeds to step S216; if it has received a notification of inability to continue monitoring, it proceeds to step S219.

[0192] In step S216, the information acquisition unit 2102 determines whether or not it has finished receiving surrounding condition information from the control device 300, that is, whether or not on-site monitoring of the shovel 100 has finished. If the information acquisition unit 2102 has finished receiving surrounding condition information from the control device 300, it proceeds to step S218; otherwise, it returns to step S215.

[0193] In addition, step S216 may determine whether or not the termination timing (e.g., termination time) set in the on-site monitoring command has been reached.

[0194] In step S218, the information acquisition unit 2102 notifies the user via the display device 240 that the period of on-site monitoring by the shovel 100 has ended, and proceeds to step S220.

[0195] Meanwhile, in step S219, the notification unit 2103 notifies the user via the display device 240 that the field monitoring function (field monitoring unit 110) of the shovel 100 is unable to continue operating, and proceeds to step S220.

[0196] In step S220, the notification unit 2103 finishes displaying the surrounding situation information on the display device 240 and terminates the current process.

[0197] [Transformation / Modification] Although the first embodiment has been described in detail above, various modifications and changes are possible within the scope of the gist of this embodiment.

[0198] For example, in this embodiment, a field monitoring command is transmitted from the support device 200 via the management device 300. However, the management device 300 (an example of an external device) may transmit a field monitoring command to the shovel 100 through an operation by the administrator of the management device 300. Alternatively, a timer function or the like pre-configured in the management device 300 may transmit a field monitoring command with predetermined settings from the management device 300 to the shovel 100. In this case, the shovel 100 transmits surrounding condition information to the management device 300, and the administrator of the management device 300 can check the surrounding condition information through the display device of the management device 300.

[0199] Furthermore, in this embodiment, surrounding situation information is transmitted from the shovel 100 to the support device 200 via the management device 300 in near real time, but the surrounding situation information may also be provided to the user in a manner that allows for retrospective confirmation. For example, instead of transmitting the surrounding situation information transmitted from the shovel 100 to the support device 200, or in addition to doing so, the management device 300 may also transmit it to the user's email address or a designated social networking service (SNS) account. For example, surrounding monitoring information transmitted directly to the support device 200 may normally be deleted without prior confirmation due to capacity limitations such as the ring buffer of the support device 200. In contrast, by transmitting a digest version of the surrounding monitoring information to the user's email address or SNS address, the user can retrospectively confirm the situation at the shovel 100's work site, albeit with some time lag.

[0200] Furthermore, in this embodiment, the shovel 100 (peripheral monitoring controller 30C) acquires surrounding situation information in response to the reception of each site monitoring command, but it may also acquire surrounding situation information at a predetermined timing regardless of the site monitoring command. The shovel 100 (peripheral monitoring controller 30C) may then record the acquired surrounding situation information as log information in its internal memory (storage unit 308C) or transmit it to the management device 300 as appropriate. This ensures that even if the user forgets to send (set) a site monitoring command from the support device 200 to the shovel 100, the management device 300 will still store the surrounding situation information of the work site. For example, it is conceivable that a user (supervisor) might forget to send a site monitoring command for a new site because their assigned site has changed, or that they might forget that the previous setting for the site monitoring command has expired. Therefore, even in such situations, the user can access the management device 300 through the support device 200 and download the most recent surrounding information of a specific excavator 100 to the support device 200, thereby confirming the conditions at the work site.

[0201] In this case, the timing for acquiring surrounding condition information by the shovel 100 (surrounding monitoring controller 30C) may be set by an operator or the like through a predetermined operation screen displayed on the display device 50 of the shovel 100. Alternatively, the timing for acquiring surrounding condition information by the shovel 100 may be set by the user of the support device 200 through a predetermined operation screen displayed on the display device 240 of the support device 200, and the settings may be reflected in the shovel 100 via the management device 300.

[0202] In this case, the timing for acquiring surrounding condition information by the shovel 100 can be set to be regular, such as once or multiple times a day at the same time, or at the same time every week from Monday to Friday. Furthermore, the timing for acquiring surrounding condition information can be set to acquire it at any time (for example, when the processing resources of the surrounding monitoring controller 30C are relatively free) or at random times (for example, when a trigger defined by a predetermined random function is output), such as when the shovel 100 is stopped.

[0203] In this case, the management device 300 may analyze the surrounding situation information received from the shovel 100 and monitor for the occurrence of predetermined events (for example, detection of suspicious persons during times when the work site is closed). When a predetermined event occurs, the management device 300 may send an alert via push notification to a support device 200 that can send a site monitoring command to the shovel 100. The alert may also be sent to the user's email address or a predetermined SNS account of the support device 200. This allows the user to be aware of the occurrence of predetermined events related to the work site, such as the detection of suspicious persons at the work site where the shovel 100 is parked, even if they have not sent a site monitoring command to the shovel 100.

[0204] In this case, the management device 300 may transmit the surrounding situation information received from the shovel 100 to the support device 200 in real time if a predetermined application is running on the support device 200 which has been registered in advance as a transmission target. This allows the user to check real-time surrounding situation information (for example, images captured by the imaging device 40 showing the surroundings of the shovel 100) simply by starting the predetermined application on the support device 200. At this time, the management device 300 may also transmit the surrounding situation information received from the shovel 100 to the support device 200 in real time if the predetermined application is started and a confirmation operation is performed by the user on the support device 200. This prevents situations where unnecessary communication costs are incurred due to the transmission of surrounding situation information that the user does not need in real time. In addition, the management device 300 may transmit the surrounding situation information received from the shovel 100 to the support device 200 when it receives an acquisition request from the support device 200. In this case, the support device 200 may, in response to user operation, send an acquisition request to acquire all unacquired surrounding condition information for the shovel 100 at once. The support device 200 may also, in response to user operation, send an acquisition request to the management device 300 that limits the surrounding condition information to be acquired from the management device 300 based on conditions such as time period, type of information, and associated metadata. This allows the user to access the management device 300 through the support device 200 and have the support device 200 acquire the necessary surrounding condition information, even when not sending a field monitoring command.

[0205] Furthermore, in this embodiment, the shovel 100 (peripheral monitoring controller 30C) includes both an alarm output unit 303C and an operation limiting unit 304C, but it may also include only one of them. In other words, the peripheral monitoring controller 30C of the shovel 100 may be configured to either output an alarm or limit the operation of the shovel 100 when it detects a monitored object within the horizontal monitoring area as seen from the shovel 100.

[0206] Furthermore, in this embodiment, the shovel 100 was configured to have all of its various operating elements, such as the lower traveling body 1, upper slewing body 3, boom 4, arm 5, and bucket 6, hydraulically driven; however, some of these elements may be electrically driven. In other words, the configurations disclosed in this embodiment may be applied to a field monitoring system that includes a hybrid shovel or the aforementioned electric shovel.

[0207] In this embodiment, the shovel 100 operates in response to the operations of an operator sitting in the cabin 10, but it may also be remotely controlled by operation inputs received from an external device (for example, a control device 300, etc.) via the communication device 60.

[0208] In this case, the shovel 100 (peripheral monitoring controller 30C) may automatically acquire surrounding condition information according to predetermined conditions set remotely from an external device and transmit it to the support device 200 via the management device 300. The predetermined conditions may include, for example, the same date and time conditions as described above, as well as conditions related to the operating status of the shovel 100, such as "the work of the shovel 100 has been completed and stopped," and conditions related to the environmental conditions around the shovel 100, such as "the illumination around the shovel 100 has decreased relatively (it has become nighttime)."

[0209] Furthermore, the shovel 100 may operate autonomously and perform predetermined tasks without relying on operator input or input from an external device.

[0210] In this case, the shovel 100 (peripheral monitoring controller 30C) may autonomously learn the timing at which it should acquire surrounding situation information. Then, the shovel 100 may autonomously acquire the surrounding situation information at the timing corresponding to the learned result and transmit it to the support device 200 via the management device 300.

[0211] Furthermore, the on-site monitoring system according to this embodiment may be applied to transport devices (e.g., transport robots) operating in factories, offices, etc. For example, since there are times when no people are present in factories, offices, etc., the transport device may acquire surrounding situation information during such times and transmit this information to a support device via a management device. This can improve security during times when there are no people present in factories, offices, etc.

[0212] Furthermore, the following is disclosed regarding this embodiment.

[0213] (1) A work machine capable of communicating with the outside world, A sensor that acquires information about the surrounding conditions of the work machine, The device comprises a transmitting unit that transmits to the external device information regarding the surrounding conditions of the work machine, which is acquired by the sensor or generated based on the information acquired by the sensor, in response to a command from the external device of the work machine. A type of machinery used for industrial work.

[0214] (2) The sensor includes an imaging device that captures images of the area around the work machine. The transmitting unit transmits the captured image of the area around the work machine, captured by the imaging device, to the external device in response to the command. (1) The work machine described above.

[0215] (3) If the command is received while the work machine is stopped, the system further includes a starting unit that activates the monitoring unit, which includes at least the sensor and the transmitting unit. (1) or (2) the work machine described above.

[0216] (4) If the command is received while the work machine is stopped, the starting unit will activate only the monitoring unit while maintaining the stopped state of the work machine. (3) The work machines described above.

[0217] (5) Based on the information acquired by the aforementioned sensor, the machine further comprises a detection unit that detects a predetermined target for monitoring within a predetermined range around the machine, The activation unit, while the work machine is stopped, activates the detection unit in addition to the sensor and the transmission unit as the monitoring unit. The transmitting unit transmits to the external device information regarding the monitored object in the vicinity of the work machine, which is detected by the detection unit. (3) or (4) The work machine described above.

[0218] (6) When the monitoring unit is activated by the activation unit while the work machine is stopped, and the monitoring target is detected by the detection unit, the transmission unit transmits information to the external device indicating that the monitoring target has been detected within the predetermined range around the work machine. (5) The work machines described above.

[0219] (7) The machine further includes a power supply that provides power to the equipment of the work machine, including the monitoring unit, The startup unit activates the monitoring unit when the command is received and the remaining capacity of the power supply exceeds a predetermined first threshold. (3) to (6) any one of the following working machines.

[0220] (8) The system further includes a first external notification unit that transmits a notification to the aforementioned external device, Even if the command is received, the startup unit will not start the monitoring unit if the remaining capacity of the power supply is below the first threshold. The first external notification unit transmits a notification to the external device indicating that the monitoring unit cannot be started if the monitoring unit is not started by the startup unit because the remaining capacity of the power supply does not exceed the first threshold. (7) The work machines described.

[0221] (9) If the monitoring unit is activated by the start unit while the work machine is stopped, the system further includes a stop unit that stops the monitoring unit when the remaining capacity of the power supply falls below a predetermined second threshold. (7) or (8) The work machine described above.

[0222] (10) The system further includes a second external notification unit that, when the remaining capacity of the power supply falls below the second threshold, sends a notification to the external device indicating that the monitoring unit is unable to continue operating. (9) The work machines described.

[0223] (11) The system further includes an internal notification unit that, if the command is received while the work machine is in operation, notifies the operator that the transmitting unit will transmit information regarding the surrounding conditions of the work machine to the external device. (1) to (10) any one of the working machines.

[0224] (12) The system further includes an authorization unit that accepts a predetermined operation to authorize remote operation of the work machine by a user terminal, The external device is the user terminal, or a management device capable of communicating with the user terminal and the work machine. When the authorization operation unit receives the predetermined operation, the transmission unit transmits information regarding the surrounding conditions of the work machine to the external device in response to the command received directly from the user terminal or via the management device. A machine used for work as described in any one of items (1) through (11).

[0225] (13) A support device that can communicate with a work machine, comprising a monitoring unit that acquires information about the surrounding conditions of the work machine and transmits it to the outside of the work machine, A transmitting unit that transmits a command to the work machine requesting information about the surrounding conditions of the work machine in response to a predetermined operation by the user, A first acquisition unit that acquires information regarding the surrounding conditions of the work machine transmitted from the work machine in response to the command, The system includes a first notification unit that notifies the user of information regarding the surrounding conditions of the work machine, which has been acquired by the first acquisition unit, Support equipment.

[0226] (14) A second acquisition unit acquires from the work machine, in response to the command, information regarding whether the monitoring unit in the stopped work machine can be started, or whether the monitoring unit can continue to operate after being started, The system includes a second notification unit that notifies the user of the information acquired by the second acquisition unit, (13) The support device described above.

[0227] (15) The second acquisition unit is mounted on the work machine and acquires information regarding the status of the power supply that supplies power to the monitoring unit. (14) The support device described above.

[0228] (16) A support method performed by a support device that can communicate with a work machine, the support device being equipped with a monitoring unit that acquires information about the surrounding conditions of the work machine and transmits it to the outside of the work machine, A transmission step of sending a command to the work machine requesting information about the surrounding conditions of the work machine in response to a predetermined operation from the user, A first acquisition step of acquiring information about the surrounding conditions of the work machine transmitted from the work machine in response to the command, The process includes a first notification step of notifying the user of information regarding the surrounding conditions of the work machine, which was obtained in the first acquisition step, How to help.

[0229] (17) The support device is instructed to perform the support method described in (16). Support program.

[0230] [Second Embodiment] Next, a second embodiment will be described.

[0231] A peripheral monitoring device is known that ensures safety around a work machine, such as an excavator, by outputting an alarm or restricting the operation of the work machine when a predetermined target (e.g., a person) is detected within the monitoring range of the work machine (for example, Japanese Patent Application Publication No. 2014-181508).

[0232] However, even if the monitored object is no longer detected within the monitoring range, it is conceivable that the monitored object may still be in a proximity area relatively close to the work machine, which includes the monitoring range. For example, if the monitored object is a person, while the work machine is idling and the operator is away from the machine, a worker may crawl under the machine to inspect the underside or inspect the engine room from the top of the slewing body. In this case, the area under and on top of the slewing body of the work machine is not usually included in the monitoring range, so the worker may enter the monitoring range and then leave it into a proximity area outside the monitoring range, potentially deactivating the alarm output and operation restrictions. As a result, the operator who returns to the work machine may, with the alarm output and operation restrictions deactivated, start work on the machine with confidence, even though the worker has been inspecting it.

[0233] Therefore, this embodiment provides a peripheral monitoring device that can improve the safety of a work machine when the object to be monitored is located in an area outside the monitoring range that is close to the work machine.

[0234] [Shovel Overview] First, with reference to Figure 11, we will describe the outline of the excavator 100 (an example of a work machine) on which the surrounding monitoring device 150 (see Figure 12) according to this embodiment is mounted.

[0235] Figure 11 is a diagram showing an example of an excavator 100 on which the surrounding monitoring device 150 according to this embodiment is mounted, and is specifically a side view of the excavator 100.

[0236] Furthermore, the surrounding monitoring device 150 according to this embodiment may be mounted on any work machine other than the shovel 100. For example, the surrounding monitoring device 150 may be mounted on a lifting magnet machine with a lifting magnet attached as an end attachment, a bulldozer, a wheel loader, an asphalt finisher, a forestry machine, a crawler crane, etc.

[0237] The shovel 100 comprises a lower traveling body 1, an upper rotating body 3 mounted on the lower traveling body 1 so as to be rotatable via a slewing mechanism 2, a boom 4, an arm 5, and a bucket 6 as attachments (working equipment), and a cabin 10.

[0238] The lower traveling body 1 includes, for example, a pair of left and right crawlers, and each crawler is hydraulically driven by a travel hydraulic motor (not shown), thereby enabling it to move under its own power.

[0239] The upper rotating body 3 rotates relative to the lower traveling body 1 by being driven by a rotating hydraulic motor or an electric motor (neither of which is shown in the illustration).

[0240] The boom 4 is pivotally attached to the front center of the upper slewing body 3 so as to be able to move up and down. The arm 5 is pivotally attached to the tip of the boom 4 so as to be able to rotate up and down. The bucket 6 is pivotally attached to the tip of the arm 5 so as to be able to rotate up and down. The boom 4, arm 5, and bucket 6 are hydraulically driven by the boom cylinder 7, arm cylinder 8, and bucket cylinder 9, respectively.

[0241] Cabin 10 is the cockpit where the operator sits and is mounted on the front left side of the upper rotating body 3.

[0242] [Configuration of the peripheral monitoring device] Next, with reference to Figure 11 and Figure 12, the specific configuration of the peripheral monitoring device 150 according to this embodiment will be described.

[0243] FIG. 12 is a block diagram showing an example of the configuration of the peripheral monitoring device 150 according to the present embodiment.

[0244] The peripheral monitoring device 150 monitors the intrusion of a predetermined object (hereinafter simply referred to as a "monitoring target") that is a monitoring target within a predetermined range around the excavator 100, and when the monitoring target is detected, outputs an alarm or restricts the operation of the excavator. The monitoring target may include not only a person such as an operator working around the excavator 100 or a supervisor at the work site, but also any object other than a person, such as a stationary obstacle such as materials temporarily placed at the work site or a moving obstacle such as a vehicle including a truck. Hereinafter, in the present embodiment, the case where the monitoring target is a person will be mainly described.

[0245] The peripheral monitoring device 150 includes a controller 30, an imaging device 40, a display device 50, an audio output device 52, a gate lock valve 54, a release switch 70, a monitoring function ON / OFF switch (hereinafter, for convenience, simply referred to as a "monitoring function switch") 72, a display content switching switch 74, and a monitoring area switching switch 76.

[0246] The controller 30 is a main control device that performs drive control of the excavator 100. For example, the controller 30 is mounted in the cabin 10 and performs various control processes related to the peripheral monitoring device 150.

[0247] The function of the controller 30 may be realized by any hardware, software, or a combination thereof. The controller 30 is configured mainly around a microcomputer including, for example, a CPU (Central Processing Unit), a RAM (Random Access Memory), a ROM (Read Only Memory), an auxiliary storage device, an RTC (Real Time Clock), and various communication interfaces. The controller 30 includes, as functional units realized by executing various programs stored in, for example, the ROM or the auxiliary storage device on the CPU, a display control unit 3011, a detection unit 3012, a tracking unit 3013, an alarm output unit 3014, and an operation restriction unit 3015.

[0248] The imaging device 40 (an example of a sensor) is attached to the upper part of the upper swing body 3, images the periphery of the excavator 100, and outputs an imaging image. The output imaging image may include an object including a monitoring target existing in the periphery of the excavator 100. That is, the imaging device 40 outputs an imaging image as detection information regarding an object existing in the periphery of the excavator 100. The imaging device 40 includes cameras 40B, 40L, and 40R.

[0249] The camera 40B, the camera 40L, and the camera 40R are respectively attached to the upper rear end, the upper left end, and the upper right end of the upper swing body 3, and image the rear, the left side, and the right side of the upper swing body 3. For example, the camera 40B, the camera 40L, and the camera 40R are each a single-eye wide-angle camera having a very wide angle of view. Specifically, the camera 40B, the camera 40L, and the camera 40R are each attached to the upper part of the upper swing body 3 such that the optical axis is directed obliquely downward, and image an imaging range in the vertical direction including from the ground near the excavator 100 to the distance from the excavator 100. The camera 40B, the camera 40L, and the camera 40R each output an imaging image at a predetermined period (for example, 1 / 30 seconds) during the operation of the excavator 100, and the output imaging image is taken into the controller 30.

[0250] The display device 50 is located around the cockpit in the cabin 10, specifically in a position easily visible to the operator seated in the cockpit, and displays various image information to be notified to the operator under the control of the controller 30. The display device 50 may be, for example, a liquid crystal display or an organic EL (electroluminescence) display, and may also be a touch panel that also serves as an operation unit. Specifically, as described later, the display device 50 displays images captured by the imaging device 40 (through images) and surrounding images (for example, viewpoint transformation images described later) generated by the controller 30 based on the images captured by the imaging device 40.

[0251] The voice output device 52 is installed around the cockpit in the cabin 10 and outputs various voice information to notify the operator under the control of the controller 30. The voice output device 52 is, for example, a speaker or a buzzer. Specifically, the voice output device 52 outputs an alarm sound based on a control command from the controller 30.

[0252] The gate lock valve 54 is located at the uppermost part of the pilot line that supplies pilot pressure from the pilot pump to an operating device (not shown) that operates the operating elements of the shovel 100 (i.e., the lower traveling body 1, the upper slewing body 3, the boom 4, the arm 5, and the bucket 6, etc.), and switches the connection / disconnection of the pilot line.

[0253] For example, the gate lock valve 54 switches the connection / disconnection of the pilot line in accordance with the output signal (ON / OFF) of a gate lock switch, which is usually linked to the operating state of a gate lock lever located in the area corresponding to the entrance to the cockpit inside the cabin 10. Specifically, the gate lock valve 54 connects the pilot line when the output signal of the gate lock switch corresponds to the state in which the gate lock lever is raised (i.e., the state in which an operator is seated in the cockpit) (hereinafter referred to as the "OFF signal" for convenience). On the other hand, the gate lock valve 54 disconnects the pilot line when the output signal of the gate lock switch corresponds to the state in which the gate lock lever is lowered (i.e., the state in which an operator has left the cockpit) (hereinafter referred to as the "ON signal" for convenience).

[0254] Furthermore, for example, the gate lock valve 54 is configured to also accept command signals input from the controller 30. Specifically, the signal input section of the gate lock valve 54 may be connected to a logic circuit that can interrupt with command signals from the controller 30. This allows the gate lock valve 54 to switch the pilot line between connected and disconnected in response to command signals (OFF signals / ON signals) from the controller 30. In other words, under the control of the controller 30, the gate lock valve 54 can disconnect the pilot line even when the gate lock lever is raised.

[0255] The release switch 70 is, for example, located near the cockpit in the cabin 10, and is an operating unit into which the operator or the like inputs their intention to release the alarm or the restrictions on the operation of the shovel 100, which will be described later. The release switch 70 may be, for example, a hardware operating unit such as a push button, lever, or rotary knob of the operating unit 50B, which will be described later. Alternatively, the release switch 70 may be a combination of a software operating unit such as a virtual button (icon) on the operation screen displayed on the display device 50 and a hardware operating unit such as a touch panel implemented on the display device 50. The same applies to the monitoring function switch 72, the display content switching switch 74, and the monitoring area switching switch 76, etc. Information regarding the operating status of the release switch 70 is received by the controller 30.

[0256] In addition, instead of the release switch 70, a camera that recognizes specific poses or gestures corresponding to the operator's intention to release, or a gaze sensor that recognizes specific eye movements corresponding to the operator's intention to release, may be used. The same applies to the monitoring function switch 72, the display content switching switch 74, and the monitoring area switching switch 76, etc.

[0257] The monitoring function switch 72 is located, for example, near the cockpit in the cabin 10, and is an operating unit for the operator to turn the monitoring function of the surrounding monitoring device 150 ON / OFF. In other words, the monitoring function switch 72 is an operating unit for enabling or disabling the functions of alarm output and operation restriction of the shovel 100, which are executed based on the detection of the monitored object.

[0258] For example, when the monitoring function is enabled in response to an operator's operation on the monitoring function switch 72, the functions of the display control unit 3011, detection unit 3012, tracking unit 3013, alarm output unit 3014, and operation restriction unit 3015, which will be described later, are enabled.

[0259] On the other hand, if the monitoring function is disabled in response to an operation by an operator on the monitoring function switch 72, for example, all functions of the display control unit 3011 (at least the functional part related to the monitoring function), detection unit 3012, tracking unit 3013, alarm output unit 3014, and operation restriction unit 3015 are disabled. In this case, only the functions of the alarm output unit 3014 and operation restriction unit 3015 are disabled, while the functions related to the monitoring function of the display control unit 3011 and the functions of the detection unit 3012 and tracking unit 3013 remain enabled. This eliminates the need to restart the functions of the display control unit 3011, detection unit 3012, and tracking unit 3013 when the monitoring function is enabled in response to an operation on the monitoring function switch 72, thus shortening the time required to restart the monitoring function.

[0260] Furthermore, for example, if the monitoring function is disabled in response to an operator's operation on the monitoring function switch 72, an indicator showing that the monitoring function is disabled will be displayed on the display device 50 under the control of the controller 30 (display control unit 3011, described later). Alternatively, a dedicated indicator consisting of a red lamp or the like may be provided inside the cabin 10, and this dedicated indicator may light up in response to an operator's operation on the monitoring function switch 72 if the monitoring function is disabled. This allows the peripheral monitoring device 150 to make the operator aware that the monitoring function (i.e., functions such as outputting alarms based on the detection of the monitored object and limiting the operation of the shovel 100) is disabled.

[0261] Information regarding the operating status of the monitoring function switch 72 is received by the controller 30.

[0262] The display content switching switch 74 is located, for example, near the cockpit in the cabin 10 and is an operating unit for the operator to switch the content displayed on the display device 50. For example, the display content switching switch 74 is an operating unit that switches whether the monitoring image displayed on the display device 50 is the image captured by the imaging device 40, the monitoring image including the surrounding image described later, or both displayed simultaneously. Information regarding the operating status of the display content switching switch 74 is received by the controller 30.

[0263] The monitoring area switching switch 76 is located, for example, near the cockpit in the cabin 10 and is an operating unit for setting a monitoring area (an example of a monitoring range) for monitoring the intrusion of a target. For example, by operating the monitoring area switching switch 76, the operator can set some or all of the individual monitoring areas corresponding to cameras 40B, 40L, and 40R as the monitoring area of ​​the peripheral monitoring device 150. Hereinafter, the individual monitoring areas for cameras 40B, 40L, and 40R will be referred to as "individual monitoring areas" and distinguished from the overall monitoring area of ​​the peripheral monitoring device 150 (i.e., the detection unit 3012 described later).

[0264] Furthermore, the monitoring area has a default setting. For example, if the monitoring area switch 76 is not operated after the shovel 100 is started, the surrounding monitoring device 150 will monitor the target within the monitoring area set by default.

[0265] The display control unit 3011 causes the display device 50 to display a monitoring image showing the surrounding conditions of the shovel 100.

[0266] For example, the display control unit 3011 displays an image captured by at least one of the cameras 40B, 40L, and 40R on the display device 50 as a monitoring image in response to an operator's operation of the display content switching switch 74. Hereinafter, the image captured displayed on the display device 50 may be referred to as a "through image".

[0267] Further, for example, the display control unit 3011 generates a peripheral image (an example of a composite image) described later based on the captured image of the imaging device 40 in response to an operation on the display content switching switch 74 by the operator, and causes the display device 50 to display a monitoring image including the peripheral image.

[0268] Specifically, the display control unit 3011 generates a viewpoint conversion image viewed from a virtual viewpoint by performing known viewpoint conversion processing, synthesis processing, etc. based on the captured images of the cameras 40B, 40L, and 40R as the peripheral image, and causes the display device 50 to display it. Further, when causing the display device 50 to display the peripheral image, the display control unit 3011 also causes the display device 50 to display a shovel image schematically representing the shovel 100 in order to clarify the relative positional relationship between the imaging range of the imaging device 40 and the shovel 100. That is, the display control unit 3011 generates a monitoring image including the shovel image and the peripheral image arranged around the shovel image in accordance with the relative positional relationship between the shovel 100 and the imaging range of the imaging device 40, and causes the display device 50 to display it.

[0269] Note that the function of the display control unit 3011 may be built into the display device 50. In this case, information such as the captured images of the imaging device 40 (cameras 40B, 40L, 40R) and the detection results of the detection unit 3012 are respectively taken into the display device 50 from the imaging device 40 and the controller 30.

[0270] For example, the monitoring images shown in FIGS. 5A and 5B described above are displayed on the display device 50.

[0271] As shown in FIGS. 5A and 5B, in this example, the display device 50 includes a display unit 50A which is a display area where various information images are displayed, and an operation unit 50B including hardware such as button switches that receives operations by an operator or the like regarding the various information images displayed on the display unit 50A.

[0272] The display unit 50A is, for example, a horizontally long rectangular display (for example, a display with an aspect ratio of 4:3) in the display device 50.

[0273] As shown in Figure 5A, the display unit 50A displays a pass-through image from one of the cameras 40B, 40L, or 40R as the monitoring image MP1, as described above.

[0274] In this example, the monitoring image MP1 (through image) is overlaid with a guideline GLa. The guideline GLa represents, for example, a position where the horizontal distance D from the shovel 100 is a predetermined distance D1. This allows operators to understand how far away the monitored object is from the shovel 100 when it is visible in the monitoring image MP1 (through image).

[0275] Furthermore, the predetermined distance D1 is set appropriately within a range of less than or equal to the predetermined distance D2 corresponding to the monitoring area described later.

[0276] Furthermore, as described above, cameras 40B, 40L, and 40R are positioned so as to capture a wide area from the ground near the upper rotating body 3 to the distance, with their optical axes directed diagonally downward from the upper end of the upper rotating body 3. Therefore, a portion of the vehicle body (upper rotating body 3) is visible at the lower end of the surveillance image MP1 (through image) in this example.

[0277] Furthermore, as shown in Figure 5B, the display unit 50A displays a monitoring image MP2, which includes the shovel image CG and the surrounding image EP arranged around the shovel image CG, as described above. This allows operators and others to properly understand the positional relationship between the monitoring target (i.e., a person) shown in the surrounding image EP and the shovel 100.

[0278] The peripheral image EP in this example is a viewpoint transformation image that combines an overhead view image BVP of the surrounding area adjacent to the shovel 100 viewed from directly above, and a horizontal view image HVP of the surrounding area viewed horizontally from the shovel 100, which is positioned around the overhead view image BVP. The peripheral image EP, which is a viewpoint transformation image, is obtained by projecting the images captured by cameras 40B, 40L, and 40R onto a spatial model, and then reprojecting the projected images onto that spatial model onto another two-dimensional plane. The spatial model is the projection target of the captured images in virtual space, and consists of one or more planes or curved surfaces, including planes or curved surfaces other than the plane on which the captured images are located.

[0279] Furthermore, the guideline GLb is superimposed on the monitoring image MP2. Similar to the guideline GLa in the monitoring image MP1 (through image) in Figure 5A, the guideline GLb represents a position where the horizontal distance D from the shovel 100 is a predetermined distance D1. In other words, the guideline GLb is a line segment obtained by offsetting (enlarging) the contour shape of the shovel image CG, which corresponds to a plan view from directly above the shovel 100, by an amount equivalent to a predetermined distance D1 outside the shovel image CG. This allows operators, etc., to understand how far away the monitored object (i.e., a person) is from the shovel 100, if it is visible in the surrounding image, similar to the case of guideline GLa in Figure 5A.

[0280] Furthermore, the process by the display control unit 3011 to generate viewpoint-converted images (peripheral images) (peripheral image generation process) and the process by the detection unit 3012 to detect the monitored object (monitoring object detection process), which will be described later, may be asynchronous processes. In this case, the controller 30 may be configured such that the peripheral image generation process starts up and has a faster processing cycle than the monitoring object detection process. This prevents situations where the peripheral image generation process is delayed and the peripheral images in the monitoring image displayed on the display device 50 are not updated.

[0281] Returning to Figures 11 and 12, the detection unit 3012 detects the object to be monitored in the monitoring area around the shovel 100 based on the image captured by the imaging device 40. For example, the detection unit 3012 detects the object to be monitored in the monitoring area defined (extending) in the horizontal direction as seen from the shovel 100 (hereinafter simply referred to as the "horizontal direction"), that is, along the plane on which the shovel 100 is working (where the lower traveling body 1 is in contact with the ground) (hereinafter conveniently referred to as the "working plane"). Specifically, the detection unit 3012 detects the object to be monitored within a monitoring area where the horizontal distance D from the shovel 100 is within a predetermined distance D2 (for example, 5 meters). Hereinafter, the horizontal monitoring area as seen from the shovel 100 may be referred to as the "horizontal monitoring area". Details of the monitoring area will be described later.

[0282] For example, the detection unit 3012 recognizes the target to be monitored within the captured image by arbitrarily applying various known image processing techniques or machine learning-based classifiers, including artificial intelligence (AI).

[0283] Furthermore, the detection unit 3012 can determine (estimate) the location (hereinafter referred to as "actual location," for example, the location at the feet) of the recognized surveillance target (person) included in the image captured by the monocular imaging device 40 by applying various known methods.

[0284] For example, the detection unit 3012 estimates the horizontal position (hereinafter referred to as "horizontal position") of the recognized monitored object based on its size on the captured image (for example, its height on the captured image). This is because the size of the recognized monitored object on the captured image is correlated to decreasing as the monitored object moves further away from the shovel 100. Specifically, a range of assumed size (for example, a range of assumed human height) can be defined for the monitored object, and therefore, a correlation between the horizontal position of the monitored object, which is within the assumed size range and can be viewed from the shovel 100, and its size on the captured image can be defined in advance. For this reason, the detection unit 3012 can estimate the actual location of the recognized monitored object (horizontal position from the shovel 100) based on a map or conversion formula that represents the correlation between the size of the monitored object on the captured image and its horizontal position as viewed from the shovel 100, which is pre-stored in the internal memory of the controller 30, such as an auxiliary storage device.

[0285] Furthermore, for example, the detection unit 3012 can estimate the actual location (e.g., the position at the user's feet) of the monitored object by projecting the captured image onto the same plane (homography), assuming that the monitored object is on the same plane as the shovel 100 (or its lower traveling body 1). In this case, a certain part (a certain point) of the captured image is associated with a certain position on the same plane as the shovel 100.

[0286] Furthermore, the detection unit 3012 may detect a target object within the monitoring area around the shovel 100 based on detection information regarding objects present around the shovel 100 from other sensors that may be mounted on the shovel 100, in addition to or in addition to the image captured by the imaging device 40. For example, the detection unit 3012 may detect a target object within the monitoring area (horizontal monitoring area) around the shovel 100 based on detection information from other sensors such as a stereo camera, millimeter-wave radar, or LIDAR (Light Detecting and Ranging).

[0287] The tracking unit 3013 tracks the monitored object detected by the detection unit 3012. Specifically, the tracking unit 3013 may identify (specify) the monitored object detected by the detection unit 3012 and track it by considering known image recognition technologies and the changes in the position of the monitored object within the monitoring area after it has been detected by the detection unit 3012 (movement status).

[0288] Furthermore, the tracking unit 3013 may perform various determinations by tracking the movement status of the monitored object within the monitoring area detected by the detection unit 3012.

[0289] For example, the tracking unit 3013 (an example of an exit determination unit) determines whether the monitored object, which was detected by the detection unit 3012 within the monitoring area and is no longer detected, has moved to an area farther horizontally from the shovel 100 than the monitoring area (horizontal monitoring area) as viewed from the shovel 100 (hereinafter referred to as the "distant area"). In other words, the tracking unit 3013 determines whether the monitored object, which is no longer detected by the detection unit 3012, has moved to a distant area where the horizontal distance D from the shovel 100 exceeds a predetermined distance D2. Furthermore, if the monitored object, which was detected by the detection unit 3012 within the monitoring area and is no longer detected, has not moved to the distant area, the tracking unit 3013 may determine that the monitored object may be in an area relatively close to the shovel 100, including the monitoring area (hereinafter referred to as the "proximity area"). This is because, as will be described later, the monitoring area cannot cover the entire proximity area of ​​the shovel 100. In other words, the imaging device 40 cannot cover the entire area in close proximity to the shovel 100 due to its imaging range. Furthermore, even if the target being monitored is within the monitoring area, it may not be detected by the detection unit 3012 if, as described later, the target is hidden by an obstacle.

[0290] Furthermore, for example, the tracking unit 3013 (an example of a valid / unvalidation unit) specifically determines whether a monitored object that has been detected by the detection unit 3012 but is no longer detected may be located in a nearby area outside the monitoring area (hereinafter referred to as the "nearby area outside the monitoring area"). In other words, when a monitored object that was detected by the detection unit 3012 is no longer detected, the tracking unit 3013 may determine whether the monitored object has moved from within the monitoring area to the nearby area outside the monitoring area. The nearby area outside the monitoring area may include, as described later, the area in front of the upper rotating body 3, the area (space) between the upper rotating body 3 and the ground below, the upper part of the upper rotating body 3, i.e., the area of ​​the house section, etc.

[0291] Furthermore, for example, the tracking unit 3013 determines whether the detection unit 3012 has lost sight of the monitored object within the monitoring area when the monitored object that was previously detected within the monitoring area by the detection unit 3012 is no longer detected. This is because if the monitored object becomes indistinguishable from the background of the image captured by the imaging device 40, or if the monitored object is hidden by an obstacle, the monitored object may not be detected by the detection unit 3012 even if it is within the monitoring area. Also, when the detection unit 3012 estimates the horizontal distance D from the shovel 100 to the monitored object based on the size of the monitored object included in the image captured by the imaging device 40, it may mistakenly determine that the monitored object is outside the monitoring area (horizontal monitoring area) even if it is within the monitoring area. Specifically, if there is a difference in height (i.e., vertical direction as seen from the shovel 100) between the position of the shovel 100 (i.e., the position on the work plane) and the position of the object being monitored, the size of the object being monitored included in the image captured by the imaging device 40 may change compared to the case where there is no difference (i.e., the object being monitored is also located on the work plane). As a result, the detection unit 3012 may mistakenly determine that an object being monitored that is actually within the monitoring area is outside the monitoring area. In this case, if there is a difference in height between the position of the shovel 100 (i.e., the position on the work plane) and the position of the object being monitored, this may include cases where there is a step between the work plane of the shovel 100 and the plane on which the object being monitored is located. Furthermore, if there is a difference in height between the position of the shovel 100 and the position of the object being monitored, this may include cases where there is a difference in the angle of inclination between the work plane of the shovel 100 and the plane on which the object being monitored is located (for example, one is located on a horizontal plane and the other on an inclined surface).

[0292] Details of the determination process and the area adjacent to the outside area will be described later (see Figures 13A, 13B, 14, and 15A to 15D).

[0293] The alarm output unit 3014 (an example of a control unit) outputs an alarm to the inside or outside of the cabin 10 when the detection unit 3012 detects a monitored object within the monitoring area. This allows the surrounding monitoring device 150 to make the operator, workers around the shovel 100, supervisors, etc., aware that a monitored object (for example, a person such as a worker) has entered the monitoring area around the shovel 100. Therefore, the surrounding monitoring device 150 can prompt the operator to check the safety conditions around the shovel 100 and prompt workers, etc., within the monitoring area to evacuate from the monitoring area.

[0294] For example, the alarm output unit 3014 outputs an alarm using an auditory method, that is, by sound. Specifically, the alarm output unit 3014 outputs a control command to the voice output device 52, causing it to output a warning sound.

[0295] Furthermore, the alarm output unit 3014 may vary the pitch, sound pressure, and tone of the warning sound, as well as the sounding period when the warning sound (e.g., a buzzer sound) is sounded periodically, depending on various conditions.

[0296] Furthermore, for example, the alarm output unit 3014 outputs an alarm in a visual manner, that is, by displaying it on the display device 50. Specifically, the alarm output unit 3014 sends an alarm request to the display control unit 3011. In response to the alarm request, the display control unit 3011 highlights the monitored object included in the monitoring image (through image or surrounding image) displayed on the display device 50, thereby enabling the display device 50 to output an alarm to the operator.

[0297] Furthermore, the alarm output unit 3014 may also output a visual alarm to workers and supervisors around the shovel 100 through an external display device that may be separately provided in the housing section of the upper rotating body 3. Alternatively, the alarm output unit 3014 may output an alarm in a tactile manner, for example, through a vibration generating device that vibrates the cockpit where the operator sits.

[0298] Further, the alarm output unit 3014 may vary the type (alarm level) of the alarm according to the positional relationship between the monitoring target detected within the monitoring area by the detection unit 3012 and the shovel 100.

[0299] For example, when the monitoring target detected within the monitoring area by the detection unit 3012 is present at a relatively distant position from the shovel 100, the alarm output unit 3014 outputs an alarm with a relatively low alarm level (hereinafter, "alarm of attention level") that prompts an operator or the like to pay attention to the monitoring target. Specifically, the alarm output unit 3014 may output an alarm of attention level when the horizontal distance D between the monitoring target detected by the detection unit 3012 and the shovel 100 exceeds a predetermined distance D1 (when D1 < D ≤ D2). Hereinafter, the area within the monitoring area (horizontal monitoring area) where the distance D from the shovel 100 exceeds the predetermined distance D1 is conveniently referred to as the "attention area". On the other hand, when the monitoring target detected within the monitoring area by the detection unit 3012 is present at a relatively close position from the shovel 100, the alarm output unit 3014 outputs an alarm with a relatively high alarm level (hereinafter, "alarm of warning level") that notifies that the monitoring target is approaching the shovel 100 and the risk level is increasing. Specifically, the alarm output unit 3014 may output an alarm of warning level when the horizontal distance D between the monitoring target detected by the detection unit 3012 and the shovel 100 is less than or equal to the predetermined distance D1 (when D ≤ D1). Hereinafter, the area within the monitoring area (horizontal monitoring area) where the distance D from the shovel 100 is less than or equal to the predetermined distance D1 is referred to as the "warning area".

[0300] In this case, the alarm output unit 3014 may differentiate the pitch, sound pressure, timbre, and sounding cycle of the sound output from the voice output device 52 between a caution level alarm and a warning level alarm. Furthermore, the alarm output unit 3014 may differentiate the color, shape, size, presence or absence of flashing, and flashing cycle of markers that highlight the monitored object included in the monitoring image (through image or surrounding image) displayed on the display device 50 between a caution level alarm and a warning level alarm. As a result, the surrounding monitoring device 150 can allow the operator to understand the alarm level, that is, the degree of proximity to the monitored shovel 100, by differences in the alarm sound and markers that highlight the monitored object displayed on the display device 50.

[0301] Furthermore, the alarm output unit 3014 cancels the alarm output if the monitored object detected by the detection unit 3012 is no longer detected after the alarm output has started, or if the release switch 70 is operated. Details of the conditions for canceling the alarm output by the alarm output unit 3014 will be described later (see Figures 15A to 15D and Figure 16).

[0302] The operation limiting unit 3015 (an example of a control unit) restricts the operation of the operating elements of the shovel 100 when the detection unit 3012 detects a target being monitored within the monitoring area. In this case, the operating elements subject to operation restriction include some or all of the lower traveling body 1, upper slewing body 3, boom 4, arm 5, and bucket 6. For example, the operation limiting unit 3015 transmits an ON signal as a command signal to the gate lock valve 54, thereby causing the pilot line to be disconnected from the gate lock valve 54. As a result, hydraulic fluid (pilot pressure) is not supplied from the pilot pump to the operating device that operates the operating elements of the shovel 100, so even if the operator inputs an operation to the operating device, the operating elements of the shovel 100 are restricted from operating. Therefore, even if a target being monitored (e.g., a person) enters the monitoring area around the shovel 100, the operation of the shovel 100 is restricted, and collisions between the shovel 100 and the target being monitored can be suppressed, thereby maintaining safety around the shovel 100.

[0303] For example, instead of the gate lock valve 54, a pressure reducing valve is provided that reduces the secondary pilot pressure output from the operating device that operates the operating element, and the operation limiting unit 3015 controls the pressure reducing valve. In this case, the pressure reducing valve can disable the operator's operation of the operating device by reducing the secondary pilot pressure output from the operating device to a pressure value below zero. Therefore, the operation limiting unit 3015 can, by controlling the pressure reducing valve, thereby limiting the operation of the operating element of the shovel 100. Alternatively, the operation limiting unit 3015 may, by controlling the pressure reducing valve, limit the operation of the shovel 100 in a manner that suppresses the operation of various operating elements in response to operator operation more gently than usual, without stopping the operating element of the shovel 100.

[0304] Furthermore, similar to the case of the alarm output unit 3014, the operation limiting unit 3015 may set different levels of operation limiting depending on the positional relationship between the monitored object detected by the detection unit 3012 and the shovel 100.

[0305] For example, when the monitoring target detected within the monitoring area by the detection unit 3012 is located at a relatively far position from the excavator 100, the operation restriction unit 3015 performs operation restriction on the excavator 100 in a mode with a relatively low restriction level. Specifically, when the horizontal distance D between the monitoring target detected by the detection unit 3012 and the excavator 100 exceeds a predetermined distance D1 (when D1 < D ≤ D2), the operation restriction unit 3015 may perform operation restriction on the excavator 100 in a mode with a relatively low restriction level (for example, making the operation speed of the operation element with respect to the operation amount slower than normal). On the other hand, when the monitoring target detected within the monitoring area by the detection unit 3012 is located at a relatively close position from the excavator 100, the operation restriction unit 3015 performs operation restriction on the excavator 100 in a mode with a relatively high restriction level. Specifically, when the horizontal distance D between the monitoring target detected by the detection unit 3012 and the excavator 100 is less than or equal to the predetermined distance D1 (when D ≤ D1), the operation restriction unit 3015 may perform operation restriction on the excavator 100 in a mode with a relatively high restriction level (for example, stopping the operation of the operation element).

[0306] Further, after the start of operation restriction, the operation restriction unit 3015 releases the operation restriction on the excavator 100 when the monitoring target detected by the detection unit 3012 disappears, or when the release switch 70 is operated. Details of the release conditions for the operation restriction on the excavator 100 by the operation restriction unit 3015 will be described later (see FIGS. 15A to 15D).

[0307] [Details of the Monitoring Area of the Peripheral Monitoring Device] Next, referring to FIGS. 13 (FIGS. 13A and 13B) and FIG. 14, details of the monitoring area of the peripheral monitoring device 150 will be described.

[0308] Figures 13 (Figures 13A and 13B) and 14 illustrate an example of the monitoring area (monitoring area MA) of the peripheral monitoring device 150. Specifically, Figure 13A schematically shows the monitoring area MA (i.e., individual monitoring areas MAB, MAL, and MAR corresponding to cameras 40B, 40L, and 40R) when the shovel 100 is viewed from above. Figure 13B schematically shows the monitoring area MA (individual monitoring area MAB) on the through-image of camera 40B. Figure 14 schematically shows the monitoring area MA (individual monitoring area MAB) when the shovel 100 is viewed from the side.

[0309] In Figure 13A, the imaging ranges AVB, AVL, and AVR of cameras 40B, 40L, and 40R represent the ranges in which cameras 40B, 40L, and 40R can acquire images of the monitored object that can be detected by the detection unit 3012, respectively. The imaging ranges AVB, AVL, and AVR can be defined, for example, by the horizontal field of view of cameras 40B, 40L, and 40R and the horizontal distance from cameras 40B, 40L, and 40R, respectively.

[0310] Furthermore, in Figures 13A and 13B, guidelines L1a and L1b correspond to positions where the horizontal distance D from the shovel 100 is a predetermined distance D1, and guidelines L2a and L2b correspond to positions where the horizontal distance D from the shovel 100 is a predetermined distance D2, respectively.

[0311] As shown in Figures 13A and 13B, the individual monitoring areas MAB, MAL, and MAR are each centered around the area within the imaging range AVB, AVL, and AVR where the horizontal distance D from the shovel 100 is less than or equal to a predetermined distance D2. That is, the boundary lines at the far end of the individual monitoring areas MAB, MAL, and MAR in the near-far direction (radial direction) as seen from the shovel 100 are defined by the line segments of guidelines L2a and L2b within the imaging range AVB, AVL, and AVR, respectively.

[0312] Furthermore, the individual monitoring areas MAB, MAL, and MAR do not include the areas 401B, 401L, and 401R near cameras 40B, 40L, and 40R within the imaging ranges AVB, AVL, and AVR, respectively. That is, the boundary lines MABe1, MALe1, and MARe1 at the near-field end of the individual monitoring areas MAB, MAL, and MAR, as viewed from the shovel 100, are all a certain distance from the shovel 100. As shown in Figure 13B, the images captured by cameras 40B, 40R, and 40L may include the vehicle body (upper rotating body 3). Therefore, if the monitoring target enters an area very close to the shovel 100, part of it may be hidden by the vehicle body, and the detection unit 3012 may not be able to properly recognize (detect) the monitoring target. Furthermore, even if a part of the vehicle body is not captured in the images taken by cameras 40B, 40R, and 40L, cameras 40B, 40R, and 40L are set to have a very wide vertical field of view that includes the ground near the shovel 100 and the area far away. Therefore, the ground in the close-range area very close to the shovel 100 (cameras 40B, 40L, and 40R) will be outside the field of view, and as a result, the entire image of the monitored object may not be captured, and the detection unit 3012 may not be able to properly recognize (detect) the monitored object.

[0313] In other words, the areas 401B, 401L, and 401R, which are adjacent to cameras 40B, 40L, and 40R within the imaging ranges AVB, AVL, and AVR, are outside the monitoring area MA (individual monitoring areas MAB, MAL, and MAR), and fall under the category of areas adjacent to the monitoring area.

[0314] Furthermore, the individual monitoring areas MAB, MAL, and MAR do not include the circumferential (lateral) regions 402B, 402L, and 402R at both ends as seen from the shovel 100, which are defined by the horizontal field of view of the imaging ranges AVB, AVL, and AVR, respectively. In other words, the space between the boundary lines MABe2, MALe2, and MARe2 at both ends of the individual monitoring areas MAB, MAL, and MAR as seen from the shovel 100 is slightly narrower than the horizontal field of view of cameras 40B, 40L, and 40R, respectively. As mentioned above, cameras 40B, 40L, and 40R are wide-angle lenses, and as shown in Figure 13B, the distortion increases as you move away from the center of the captured image to the left and right, resulting in a phenomenon where the image in the captured image is tilted to the left or right (hereinafter referred to as the "image tilt phenomenon"). Therefore, at the edges of the horizontal field of view of cameras 40B, 40L, and 40R, image distortion may occur, causing a portion of the monitored object to not be included in the captured image. As a result, the detection unit 3012 may not be able to properly recognize (detect) the monitored object.

[0315] In other words, the regions 402B, 402L, and 402R at both ends of the imaging range AVB, AVL, and AVR as seen from the shovel 100 in the circumferential (lateral) direction are outside the monitoring area MA (individual monitoring areas MAB, MAL, MAR) and fall under the category of areas adjacent to the monitoring area.

[0316] Furthermore, in the close-range areas to the left rear and right rear of the shovel 100, there are areas 403L and 403R that are not included in the imaging ranges AVB, AVL, and AVR of cameras 40B, 40L, and 40R. This is because there are limitations to the horizontal field of view of cameras 40B, 40L, and 40R.

[0317] In other words, the areas 403L and 403R on the left rear side and right rear side of Shovel 100 are outside the monitoring area MA and fall under the category of areas adjacent to the monitoring area.

[0318] Thus, within the horizontal proximity area viewed from the shovel 100, areas such as 401B, 401L, and 401R, which are adjacent to cameras 40B, 40L, and 40R in a plan view, and areas 403L and 403R, which are blind spots of cameras 40B, 40L, and 40R, located behind and to the sides of the shovel 100, may not be included in the monitoring area MA. Therefore, even if the monitored object enters the monitoring area MA (individual monitoring areas MAB, MAL, MAR) and is detected by the detection unit 3012, it may subsequently move to that area (proximity area outside the area), or the shovel 100 may move and become relatively located in that area (proximity area outside the area). In other words, even though the monitored object is near the shovel 100, it may no longer be detected by the detection unit 3012.

[0319] Furthermore, the forward area AF, which includes the front central area AFM, the left front lateral area AFL, and the right front lateral area AFR within the proximity area of ​​Shovel 100, is not designated as a monitoring area MA for monitoring the intrusion of the target object, as the operator can visually confirm these areas.

[0320] In other words, the forward area AF of Shovel 100 is outside the monitoring area MA and falls into the adjacent area outside the monitoring area.

[0321] Thus, within the horizontal proximity area of ​​the shovel 100, the forward area AF of the shovel 100 may not be included in the monitoring area MA in the first place. Therefore, even if the monitored object enters the monitoring area MA (individual monitoring areas MAB, MAL, MAR) and is detected by the detection unit 3012, it may subsequently move to the forward area AF, or become located in the forward area AF. In other words, even though the monitored object is present near the shovel 100, it may not be detected by the detection unit 3012.

[0322] Furthermore, as shown in Figure 14, when the shovel 100 is viewed from the side, the imaging range AVB of camera 40B can be defined by the vertical field of view from approximately directly below camera 40B to approximately horizontal. The individual monitoring area MAB is composed of the area within the imaging range AVB of camera 40B excluding the portion in which the body of the shovel 100 (upper rotating body 3) is captured. The same applies to the other cameras (cameras 40L, 40R) included in the imaging device 40. Therefore, the monitoring targets in the upper part of the upper rotating body 3, that is, the area of ​​the housing section (hereinafter, "upper area") AU, and the area between the upper rotating body 3 and the ground below it, including the space between the left and right pairs of crawlers of the lower traveling body 1 (hereinafter, "lower area") AD, cannot be detected by the detection unit 3012. In other words, the upper area AU and lower area AD of the upper rotating body 3 are not set as monitoring areas for monitoring the intrusion of monitored targets in the first place. For the monitored object to enter the upper area AU or lower area AD, it must pass through the monitoring area MA (horizontal monitoring area), which extends horizontally from the perspective of the shovel 100, or the forward area AF, which is visible to the operator, and it is thought that the operator will notice the object before it enters.

[0323] In other words, the vertical proximity areas viewed from the shovel 100 (upper rotating body 3), such as the upper region AU and lower region AD of the upper rotating body 3, are outside the monitoring area MA and fall under the category of proximity areas outside the monitoring area.

[0324] Thus, the vertical proximity area as seen from the shovel 100, namely the upper area AU and the lower area AD, may not be included in the monitoring area MA at all. Therefore, even if the monitored object enters the monitoring area MA (individual monitoring areas MAB, MAL, MAR) and is detected by the detection unit 3012, it may subsequently move to the upper area AU, move into the lower area AD, or become located in the lower area AD using the lifting steps of the upper rotating body 3. In other words, even though the monitored object is present near the shovel 100, it may no longer be detected by the detection unit 3012.

[0325] [Specific examples of peripheral monitoring device operation] Next, the specific operation of the peripheral monitoring device 150 will be explained with reference to Figures 15A to 15D.

[0326] First, Figure 15A illustrates the first example of the specific operation of the peripheral monitoring device 150. Specifically, Figure 15A schematically shows an example of the movement status of the worker W being monitored, based on the monitoring area MA (individual monitoring area MAB) of Figure 13B described above. In the figure, the solid and dotted lines of worker W represent the state in which they are detected by the detection unit 3012 and the state in which they are not detected, respectively. The same applies to Figures 15B to 15D below.

[0327] In this example, worker W, located at position Pa0 in the remote area, enters the attention area (position Pa1) within the monitoring area (individual monitoring area MAB), then enters the alert area (position Pa2), and then moves away from the shovel 100, leaving the monitoring area and moving out into the remote area.

[0328] When worker W enters the attention area (location Pa1) within the monitoring area from the distant area (location Pa0), the detection unit 3012 detects worker W. Then, the alarm output unit 3014 and the operation restriction unit 3015 start outputting an alarm and restricting the operation of the shovel 100, respectively, in response to the detection of worker W by the detection unit 3012.

[0329] Subsequently, when worker W moves from the attention area (position Pa1) to the warning area (position Pa2) within the monitoring area, the detection unit 3012 recognizes that worker W, who has entered the monitoring area, has entered the warning area. Then, the alarm output unit 3014 and the operation restriction unit 3015 increase the alarm level and the degree of restriction on the operation of the shovel 100, respectively, in accordance with the change in the relative position of worker W with respect to the shovel 100 as recognized by the detection unit 3012.

[0330] Subsequently, when worker W leaves the monitoring area and moves to the distant area, the detection unit 3012 ceases to detect worker W. Furthermore, since worker W, who was continuously detected by the detection unit 3012 within the monitoring area, is last detected near the boundary line between the monitoring area and the distant area (i.e., guideline L2b) and then ceases to be detected, the tracking unit 3013 can determine that worker W has moved to the distant area. Specifically, the tracking unit 3013 can determine that the monitored object (worker W), who was detected by the detection unit 3012, has moved to the distant area when the monitored object is no longer detected in an area relatively far from the shovel 100 within the monitoring area (horizontal monitoring area). In this case, the area relatively far from the shovel 100 within the monitoring area is, for example, a caution area where the distance D from the shovel 100 within the monitoring area exceeds a predetermined distance D1. Furthermore, the area within the monitoring area that is relatively far from the shovel 100 may be the area furthest from the shovel 100 when the monitoring area (horizontal monitoring area) is divided into three or more areas in the direction away from the shovel 100. In addition, the tracking unit 3013 may further determine whether the monitored object has moved into the distant area by considering the movement status of the monitored object detected by the detection unit 3012, that is, the direction of movement and the speed of movement. For example, the tracking unit 3013 may determine whether the monitored object has moved into the distant area based on whether the direction of movement of the monitored object before it was no longer detected by the detection unit 3012 was toward the distant area, or whether the speed of movement of the monitored object is greater than or equal to a predetermined speed that would indicate it has moved into the distant area.

[0331] In this case, the alarm output unit 3014 and the operation restriction unit 3015 release the alarm output and the operation restriction of the shovel 100, respectively. This is because it is recognized that worker W has moved away from the monitoring area to a distant area away from the shovel 100, and it is considered that safety has been ensured around the shovel 100.

[0332] Thus, the alarm output unit 3014 and the operation restriction unit 3015 release the alarm output and the operation restriction of the shovel 100 when the monitored object that was detected by the detection unit 3012 is no longer detected, and the tracking unit 3013 determines that the monitored object has moved into a remote area.

[0333] Next, Figure 15B illustrates a second example of the specific operation of the peripheral monitoring device 150.

[0334] In this example, worker W, located at position Pb0 in the distant area, enters the attention area (position Pb1) within the monitoring area (individual monitoring area MAB), approaches the shovel 100, and moves out of the monitoring area into area 401B (position Pb2) or area 403L (position Pb3), that is, the adjacent area outside the monitoring area. The operation of the surrounding monitoring device 150 when worker W moves from position Pb0 to position Pb1 is the same as the operation when worker W moves from position Pa0 to position Pa1 in the first example in Figure 15A, so the explanation will be omitted.

[0335] When worker W moves from within the monitoring area (position Pb1) to the adjacent area outside the monitoring area (positions Pb2, Pb3), the detection unit 3012 stops detecting worker W, who was previously detected within the monitoring area. Furthermore, since worker W, who was continuously detected within the monitoring area by the detection unit 3012, is last detected near the boundary line between the monitoring area and the adjacent area outside the monitoring area, and then stops being detected, the tracking unit 3013 can determine that worker W has not moved into the distant area, that is, that there is a possibility that worker W is still in the adjacent area of ​​the shovel 100. Alternatively, the tracking unit 3013 may determine from this situation that worker W has specifically moved into the adjacent area outside the monitoring area. Specifically, the tracking unit 3013 may determine that the monitored object (worker W), which was detected by the detection unit 3012, has not moved into the distant area (i.e., there is a possibility that worker W is still in the adjacent area) when the monitored object is no longer detected in any area within the monitoring area (horizontal monitoring area) other than the area relatively far from the shovel 100. In this case, the area within the monitoring area that is relatively far from the shovel 100 is, for example, a caution area where the distance D from the shovel 100 within the monitoring area exceeds a predetermined distance D1, similar to the first example in Figure 15A. Alternatively, the area within the monitoring area that is relatively far from the shovel 100 may also be, similar to the first example in Figure 15A, the area furthest from the shovel 100 when the monitoring area is divided into three or more areas in the direction away from the shovel 100. Furthermore, the tracking unit 3013 may determine that the monitored object has moved into the area adjacent to the monitoring area when it is no longer detected in the area within the monitoring area (horizontal monitoring area) that is relatively close to the shovel 100. In this case, the area within the monitoring area that is relatively close to the shovel 100 is, for example, a warning area where the distance D from the shovel 100 within the monitoring area is less than or equal to a predetermined distance D1. Furthermore, the area within the monitoring area that is relatively close to the shovel 100 may be any area other than the area furthest from the shovel 100 when the monitoring area is divided into three or more areas in the direction away from the shovel 100. In addition, the tracking unit 3013 may further determine whether the monitored object has moved out of the monitoring area or into the adjacent area, taking into account the movement status of the monitored object detected by the detection unit 3012, that is, the direction and speed of movement.For example, the tracking unit 3013 may determine whether the monitored object has moved into the area adjacent to the monitoring area based on whether the direction of movement of the monitored object before it was no longer detected by the detection unit 3012 was toward the area adjacent to the monitoring area, or whether the speed of movement of the monitored object is greater than or equal to a predetermined speed that would indicate it has moved into the area adjacent to the monitoring area. Alternatively, the tracking unit 3013 may determine that the monitored object has moved into the area adjacent to the monitoring area when the monitored object, which was detected by the detection unit 3012, is no longer detected at the end of the monitoring area facing the area adjacent to the monitoring area (i.e., near the boundary line facing the area adjacent to the monitoring area). In this example, the tracking unit 3013 may determine that the monitored object has moved into the adjacent area when the monitored object, which was detected by the detection unit 3012, is no longer detected at both ends of the monitoring area in the lateral direction as seen from the shovel 100 (i.e., near the boundary line MABe2 within the monitoring area). Furthermore, the tracking unit 3013 may determine that the monitored object has moved into a nearby area if the monitored object, which was detected by the detection unit 3012, is no longer detected at the end of the monitored area that faces the shovel 100 (camera 40B) in a plan view (i.e., near the boundary line MABe1 within the monitored area).

[0336] In this case, the alarm output unit 3014 and the operation restriction unit 3015 continue to output an alarm and restrict the operation of the shovel 100, respectively. This is because, although the detection unit 3012 no longer detects the worker W, there is still a possibility that the worker W is still in the proximity area close to the shovel 100, specifically, the proximity area outside the monitoring area, and therefore it cannot be determined that the safety of the shovel 100 is ensured. Furthermore, when the alarm output unit 3014 and the operation restriction unit 3015 continue to output an alarm and restrict the operation of the shovel 100, they may change the manner of the alarm output and the manner of the operation restriction of the shovel 100. This allows the operator to understand that although the monitoring target is not detected by the detection unit 3012, there is still a possibility that the monitoring target is still in the proximity area including the monitoring area of ​​the shovel 100 (or the proximity area outside the monitoring area if the tracking unit 3013 has determined that the monitoring target has gone out into the proximity area outside the monitoring area). In other words, the alarm output unit 3014 and the operation restriction unit 3015 can provide the operator with information indicating that the monitored object may still be present in the adjacent area including the monitoring area, or in the adjacent area outside the monitoring area, by changing the alarm output mode or the operation restriction mode. For example, the alarm output unit 3014 may use different types of alarm sounds (sound pressure, tone, sounding cycle, etc.).

[0337] Thus, even if the monitored object that was detected by the detection unit 3012 is no longer detected, the alarm output unit 3014 and the operation restriction unit 3015 will continue to output an alarm and restrict the operation of the shovel 100 if the tracking unit 3013 determines that the monitored object has not moved into the distant area (i.e., it may still be in the nearby area), or that it has moved into the nearby area outside the monitoring area. As a result, the surrounding monitoring device 150 can continue to output an alarm and restrict the operation of the shovel 100 in situations where the monitored object may be in the nearby area close to the shovel 100.

[0338] For example, while the excavator 100 is idling, the operator may leave the excavator 100, and another operator may approach the excavator 100 for inspection or other purposes, placing them in the vicinity outside the monitoring area. In this case, even when the operator returns to the excavator 100, the surrounding monitoring device 150 will continue to output an alarm and restrict the operation of the excavator 100 due to the presence of a worker in the vicinity outside the monitoring area of ​​the excavator 100, thereby preventing the operator from starting work in such a situation.

[0339] Next, Figure 15C illustrates a third example of the specific operation of the peripheral monitoring device 150.

[0340] In this example, worker W, located at position Pc0 in the remote area, enters the attention area (position Pc1) within the monitoring area (individual monitoring area MAB) from the remote area (position Pc0) outside the monitoring area. Subsequently, despite being within the monitoring area (position Pc2), worker W is no longer detected by the detection unit 3012 and leaves the monitoring area to the remote area (position Pc3) without being detected again by the detection unit 3012. The operation of the surrounding monitoring device 150 when worker W moves from position Pc0 to position Pc1 is the same as the operation when worker W moves from position Pa0 to position Pa1 in the first example in Figure 14A, so the explanation is omitted.

[0341] Even when worker W is within the monitoring area (position Pc2), as described above, if worker W blends in with the background of the image captured by the imaging device 40, or if worker W is hidden by an obstacle, worker W will no longer be detected by the detection unit 3012. Also, as described above, if there is a difference in height between the position of worker W and the position of the shovel 100, worker W may no longer be detected by the detection unit 3012. Furthermore, if worker W, who had been continuously detected by the detection unit 3012, is no longer detected without any indication that he or she has left the monitoring area, the tracking unit 3013 can determine that the detection unit 3012 has lost sight of worker W within the monitoring area. Specifically, the tracking unit 3013 may determine that the detection unit 3012 has lost sight of the monitored object (worker W) within the monitoring area if it is no longer detected in an area relatively far from the boundary of the monitoring area (i.e., a predetermined central area of ​​the horizontal monitoring area). Furthermore, the tracking unit 3013 may determine whether or not it has lost sight of the worker W within the monitoring area by considering the movement of the monitored object within the monitoring area, that is, its direction of movement and speed of movement. For example, the tracking unit 3013 may determine whether or not the detection unit 3012 has lost sight of the monitored object based on whether or not the speed of the monitored object's movement is below a predetermined speed that would suggest it has crossed the boundary of the monitoring area.

[0342] In this case, the alarm output unit 3014 and the operation restriction unit 3015 continue to output an alarm and restrict the operation of the shovel 100, respectively. This is because, although the detection unit 3012 no longer detects the worker W, there is a possibility that the worker W, which the detection unit 3012 has lost sight of, is still in the nearby area including the monitoring area, specifically within the monitoring area, and therefore it cannot be determined that the safety of the shovel 100 is ensured. Furthermore, the alarm output unit 3014 and the operation restriction unit 3015 may change the manner of alarm output and the manner of operation restriction of the shovel 100 when continuing to output an alarm and restrict the operation of the shovel 100, similar to the second example in Figure 15B. This allows the operator to understand that although the monitoring target is not detected by the detection unit 3012, there is a possibility that the monitoring target is still present in the nearby area including the monitoring area of ​​the shovel 100. In other words, the alarm output unit 3014 and the operation restriction unit 3015 can provide the operator with information indicating that the monitored object may still be present in the adjacent area, including the monitoring area, by changing the alarm output mode or the operation restriction mode. Furthermore, the alarm output unit 3014 and the operation restriction unit 3015 may change the alarm output mode or the operation restriction mode of the shovel 100 to a mode different from that in the second example in Figure 15B, that is, when the monitored object has moved out of the monitoring area into the adjacent area. This allows the operator to understand that although the monitored object has not been detected by the detection unit 3012, there is a possibility that the monitored object may still be present within the monitoring area of ​​the shovel 100. In other words, the alarm output unit 3014 and the operation restriction unit 3015 can provide the operator with information that indicates the possibility that the monitored object may still be present within the monitoring area and information that indicates the possibility that the monitored object may still be present in the adjacent area outside the monitoring area, by changing the alarm output mode or the operation restriction mode. For example, the alarm output unit 3014 may have different types of alarm sounds (sound pressure, tone, sounding period, etc.), similar to the second example in Figure 15B.

[0343] Furthermore, it is possible that the detection unit 3012 may detect worker W again within the monitoring area before worker W leaves the monitoring area (position Pc3). For example, this may occur when worker W or the shovel 100 moves, causing the assimilation of worker W with the background in the image captured by the imaging device 40 to be resolved, or when worker W is no longer hidden by an obstacle from the perspective of the imaging device 40. In this case, the tracking unit 3013 can determine that worker W has moved to a distant area from the monitoring area, and in accordance with this determination by the tracking unit 3013, the alarm output and the restriction on the operation of the shovel 100 are released.

[0344] Thus, even if the monitored object that was detected by the detection unit 3012 is no longer detected, the alarm output unit 3014 and the operation restriction unit 3015 will continue to output an alarm and restrict the operation of the shovel 100 if the tracking unit 3013 determines that the detection unit 3012 has lost sight of the monitored object within the monitoring area. As a result, the surrounding monitoring device 150 can continue to output an alarm and restrict the operation of the shovel 100 in situations where the monitored object that has been lost sight of within the monitoring area may be in a nearby area close to the shovel 100.

[0345] Next, Figure 15D illustrates a fourth example of the specific operation of the peripheral monitoring device 150.

[0346] In this example, worker W1, located at position Pb0 in the distant region, enters the monitoring area (individual monitoring area MAB) (position Pb1), approaches the shovel 100, and exits the monitoring area to region 401B (position Pb2) or region 403L (position Pb3), that is, the area adjacent to the shovel 100 outside its monitoring area. At a similar time, worker W2, located at position Pd0 in the distant region, enters the monitoring area (position Pd1), approaches the shovel 100, and exits the monitoring area to region 401B (position Pd2), that is, the area adjacent to the shovel 100 outside its monitoring area. Subsequently, worker W2 moves away from the shovel 100, returns from the area adjacent to the monitoring area to the monitoring area (position Pd3), and then exits from the monitoring area to the distant region (position Pd4). The operation of the surrounding monitoring device 150 when workers W1 and W2 move from positions Pb0 and Pd0 to positions Pb1 and Pd1 is the same as the operation when worker W moves from position Pa0 to position Pa1 in the first example in Figure 14A, so the explanation will be omitted.

[0347] When workers W1 and W2 move from within the monitoring area (positions Pb1 and Pd1) to the adjacent area outside the monitoring area (positions Pb2, Pb3, and Pd2), the detection unit 3012 stops detecting workers W1 and W2, who were previously detected within the monitoring area. Furthermore, since workers W1 and W2, who were continuously detected within the monitoring area by the detection unit 3012, are last detected near the boundary line between the monitoring area and the adjacent area outside the monitoring area before being no longer detected, the tracking unit 3013 can determine, as in the second example in Figure 15B, that worker W has not moved into the distant area, meaning that there is a possibility that he is still present in the adjacent area including the monitoring area. In addition, the tracking unit 3013 counts the number of monitored objects that were detected by the detection unit 3012 but are no longer detected, and that have not moved into the distant area, meaning there is a possibility that they are still present in the adjacent area (hereinafter, for convenience, referred to as "undetected monitored objects"). Undetected monitored targets include not only monitored targets that have moved into the area adjacent to the monitoring area, as in this example, but also monitored targets that the detection unit 3012 has lost sight of within the monitoring area, as in the third example in Figure 15C. In this example, when workers W1 and W2 move from the monitoring area into the area adjacent to the monitoring area and are no longer detected by the detection unit 3012, the tracking unit 3013 counts the number of undetected monitored targets as "2".

[0348] In this case, the alarm output unit 3014 and the operation restriction unit 3015 continue to output an alarm and restrict the operation of the shovel 100, respectively, as in the second example in Figure 15B. Furthermore, when the alarm output unit 3014 and the operation restriction unit 3015 continue to output an alarm and restrict the operation of the shovel 100, as in the second example in Figure 15B, they may change the manner of the alarm output and the manner of the operation restriction of the shovel 100 to allow the system to recognize that the monitored object may still be present in the nearby area.

[0349] Subsequently, when worker W2 returns to the monitoring area (position Pd3) from the area adjacent to the monitoring area outside the monitoring area (position Pd2), the detection unit 3012 detects worker W2 again. At this time, the detection unit 3012 applies known image recognition techniques and, for example, compares the features of the monitoring target that is no longer detected with the features of the monitoring target that has been detected, thereby identifying that the monitoring target that has been detected again is the same target as the monitoring target that was previously detected (worker W2). Furthermore, the tracking unit 3013 changes the number of undetected monitoring targets to "1" because worker W1 is the only undetected monitoring target remaining after worker W2 is detected again by the detection unit 3012.

[0350] In this case, the alarm output unit 3014 and the operation restriction unit 3015 continue to output an alarm and restrict the operation of the shovel 100, respectively. This is because the detection unit 3012 has detected worker W2 in the monitoring area, and there is still a possibility that the monitored target (worker W1) is still present in the adjacent area including the monitoring area. Furthermore, the alarm output unit 3014 and the operation restriction unit 3015 may continue to output an alarm and restrict the operation of the shovel 100 in a manner that makes it clear that the monitored target may still be present in the adjacent area including the monitoring area. This is because it is considered more difficult for the operator to identify a monitored target (worker W1) that has not been detected by the detection unit 3012 than a monitored target (worker W2) that has been detected by the detection unit 3012 within the monitoring area.

[0351] Subsequently, when worker W2 leaves the monitoring area (location Pd3) and moves to the distant area (location Pd4), the detection unit 3012 stops detecting worker W2. Also, since worker W2, who was continuously detected by the detection unit 3012 within the monitoring area, is last detected near the boundary between the monitoring area and the distant area and then goes undetected, the tracking unit 3013 can determine that worker W2 has moved to the distant area. On the other hand, the tracking unit 3013 keeps the number of undetected monitored targets at "1" because, although worker W2 has moved to the distant area, there is still a possibility that the monitored target (worker W1) is still present in the nearby area including the monitoring area.

[0352] In this case, the alarm output unit 3014 and the operation restriction unit 3015 continue to output an alarm and restrict the operation of the shovel 100, respectively. This is because, although the detection unit 3012 does not detect the monitored object, the number of undetected monitored objects is "1", and there is still a possibility that the monitored object (worker W1) is still present in the nearby area, including the monitoring area.

[0353] Thus, even if the detection unit 3012 no longer detects the monitored object, the alarm output unit 3014 and the operation restriction unit 3015 continue to output an alarm and restrict operations as long as the number of undetected monitored objects counted by the tracking unit 3013 is not zero. On the other hand, if the detection unit 3012 no longer detects the monitored object and the tracking unit 3013 counts zero, the alarm output unit 3014 and the operation restriction unit 3015 release the alarm output and the operation restriction on the monitored object. As a result, the peripheral monitoring device 150 can determine whether there are any monitored objects that have not moved to a distant area (undetected monitored objects) among the monitored objects that have not been detected by the detection unit 3012, even when multiple monitored objects are entering and leaving the monitoring area. The peripheral monitoring device 150 can then continue to output an alarm and restrict operations if there are undetected monitored objects, depending on the presence or absence of undetected monitored objects that it has determined.

[0354] [Details of peripheral monitoring processing by peripheral monitoring devices] Next, referring to Figure 15, we will describe in detail the process related to the output of an alarm and the restriction of the operation of the shovel 100 based on the detection of a monitored object by the surrounding monitoring device 150 according to this embodiment (hereinafter referred to as "surrounding monitoring process"), specifically the processing flow.

[0355] Figure 16 is a flowchart illustrating an example of the peripheral monitoring process performed by the peripheral monitoring device 150 (controller 30). The process shown in this flowchart is repeatedly executed at predetermined processing cycles, for example, when the alarm output unit 3014 and the operation limiting unit 3015 are not outputting alarms or limiting the operation of the shovel 100 during the operation of the shovel 100.

[0356] Furthermore, since the processing in steps S302, S306 to S316 only needs to be performed by at least one of the alarm output unit 3014 and the operation limiting unit 3015, the main operators of these processes will be conveniently referred to as "alarm output unit 3014, etc."

[0357] In step S302, the alarm output unit 3014, etc., determines whether the detection unit 3012 has detected a monitored object within the monitoring area, specifically whether the monitored object within the monitoring area is recognized in the image captured by the imaging device 40. If the detection unit 3012 has detected a monitored object within the monitoring area, the alarm output unit 3014, etc., proceeds to step S304; otherwise, the process ends.

[0358] In step S304, the alarm output unit 3014 starts outputting an alarm through the display device 50 and the audio output device 52, and the operation limiting unit 3015 controls the gate lock valve 54 to start limiting the operation of the shovel 100.

[0359] In step S306, the alarm output unit 3014, etc., determines whether the release switch 70 has been operated by an operator or the like. If the release switch 70 has not been operated by an operator or the like, the alarm output unit 3014, etc., proceeds to step S308; if the release switch 70 has been operated, proceeds to step S314.

[0360] In step S308, the alarm output unit 3014, etc., determines whether the state in which the monitored object is detected within the monitoring area by the detection unit 3012 continues to occur. If the state in which the monitored object is detected within the monitoring area by the detection unit 3012 does not continue to occur, that is, if the monitored object is no longer detected by the detection unit 3012, the alarm output unit 3014, etc. proceeds to step S310. Otherwise, it returns to step S306 and repeats the processing from step S306 onward. As a result, the surrounding monitoring device 150 can continue to output alarms and restrict the operation of the shovel 100 when the monitored object continues to exist within the monitoring area, thereby improving the safety of the shovel 100.

[0361] In step S310, the alarm output unit 3014, etc., determines whether there are any monitored objects that may still exist in the nearby area including the monitoring area (i.e., undetected monitored objects) among the monitored objects that were detected by the detection unit 3012 but are no longer detected. Specifically, the alarm output unit 3014, etc., determines whether the number of undetected monitored objects counted by the tracking unit 3013 is 1 or more. If there are no undetected monitored objects, that is, if the number of undetected monitored objects counted by the tracking unit 3013 is zero, the alarm output unit 3014, etc., proceeds to step S112. On the other hand, if there are undetected monitored objects, that is, if the number of undetected monitored objects counted by the tracking unit 3013 is not zero, the alarm output unit 3014, etc., returns to step S106 and repeats the processing from step S106 onward. As a result, the surrounding monitoring device 150 can continue to output alarms and restrict the operation of the shovel 100 in situations where there is still a possibility that the target to be monitored is present in the nearby area, including the monitoring area of ​​the shovel 100, thereby further improving the safety of the shovel 100.

[0362] In step S112, the alarm output unit 3014 cancels the alarm output through the display device 50 and the audio output device 52, and the operation restriction unit 3015 controls the gate lock valve 54 to release the operation restriction of the shovel 100, thus ending this process.

[0363] Meanwhile, in step S114, the alarm output unit 3014, etc., determines whether or not there are undetected monitored targets, that is, whether or not the number of undetected monitored targets counted by the tracking unit 3013 is 1 or more, similar to step S110. If there are no undetected monitored targets, that is, if the number of undetected monitored targets counted by the tracking unit 3013 is zero, the alarm output unit 3014, etc. proceeds to step S112. On the other hand, if there are undetected monitored targets, that is, if the number of undetected monitored targets counted by the tracking unit 3013 is not zero, the alarm output unit 3014, etc. proceeds to step S116.

[0364] In step S116, the alarm output unit 3014, etc., determines whether or not additional conditions (hereinafter referred to as "safety confirmation conditions") indicating that the safety of the area around the shovel 100 has been confirmed have been met.

[0365] For example, the safety confirmation condition is "the operator has confirmed the safety of the area around the shovel 100." The controller 30 may determine whether the safety confirmation condition has been met based on the output of the alarm and the elapsed time from the start of the operation restriction of the shovel 100 until the release switch 70 is operated. If the elapsed time is relatively long, it may be determined that there is a high probability that the operator has confirmed the safety of the area around the shovel 100. Alternatively, the controller 30 may determine whether the safety confirmation condition has been met based on detection information from sensors that detect the operator's actions (e.g., gaze sensors, cameras, etc.).

[0366] Furthermore, for example, a safety confirmation condition is "permission has been granted to output an alarm and to release the operational restrictions on the shovel 100 from outside the shovel 100." The controller 30 may determine whether the safety confirmation condition has been met based on whether workers, supervisors, etc., around the shovel 100, as captured in the image captured by the imaging device 40, have performed a predetermined pose or gesture indicating permission to release the restrictions. Alternatively, the controller 30 may determine whether the safety confirmation condition has been met based on whether a signal corresponding to permission to release the restrictions (hereinafter referred to as the "release permission signal") has been received via a predetermined communication device (not shown) from a mobile terminal (e.g., a general-purpose smartphone, tablet, or dedicated mobile device) held by a supervisor at the work site, a management terminal in the management office of the work site, or a management server installed outside the work site. In this case, the mobile terminal, management terminal, or management server may acquire images of the area around the shovel 100 from a drone or other shovels at the work site. As a result, the mobile terminal, the management terminal, or the management server can analyze the captured image to confirm the safety of the area around the shovel 100, and if safety is confirmed, it can automatically send a release permission signal to the shovel 100. Alternatively, the release permission signal may be sent directly to the shovel 100 from the drone or the other shovel. The shovel 100 may also send a signal corresponding to the release request (release request signal) to the mobile terminal, the management terminal, the management server, the drone, or the other shovel, etc., in response to a request from an operator, etc. (for example, a predetermined operation). After receiving the release request signal, the mobile terminal, the management terminal, the management server, the drone, or the other shovel, etc., may send a release permission signal to the shovel 100 if it receives an operation from the supervisor or manager of the work site or the administrator of the management server, etc., to authorize the output of an alarm or the release of the restrictions on the operation of the shovel 100.

[0367] The alarm output unit 3014, etc., proceeds to step S112 if the safety confirmation conditions are met, and returns to step S114 if the safety conditions are not met, and repeats the process. As a result, even if the release switch 70 is operated in a situation where there is still a possibility that the monitored object is present in the vicinity area including the monitoring area of ​​the shovel 100, if the safety confirmation is not sufficiently performed, i.e., if the safety confirmation conditions are not met, the alarm output and the operation restriction of the shovel 100 will continue, further improving the safety of the shovel 100.

[0368] [Effect] Thus, in this embodiment, even if the monitored object that was detected by the detection unit 3012 is no longer detected after the alarm output unit 3014 and the operation restriction unit 3015 have started to output an alarm and restrict the operation of the shovel 100, the detection unit 3012 will continue to output an alarm and restrict the operation of the shovel 100, depending on the circumstances when the monitored object transitions from being detected to not being detected. As described above, depending on the circumstances when the monitored object transitions from being detected to not being detected by the detection unit 3012, there is a high possibility that the monitored object is still present in the vicinity area including the monitoring area of ​​the shovel 100. In response to this, in such a situation, the peripheral monitoring device 150 can provide the operator with information regarding the possibility that the monitored object is still present in the vicinity area of ​​the shovel 100 including the monitoring area by continuing to output an alarm and restrict the operation of the shovel 100. Therefore, the peripheral monitoring device 150 according to this embodiment can improve the safety of the shovel 100.

[0369] In this embodiment, information regarding the possibility that a monitored object may still be present in a proximity area relatively close to the excavator 100, including the monitoring area, is provided to the operator, etc., by continuing to output an alarm or restrict the operation of the excavator 100. However, such information may be provided by other means. For example, if the surrounding monitoring device 150 finds that a monitored object that is no longer detected by the detection unit 3012 may still be present in a proximity area, it may display this fact on the display device 50 or output it as an audible message through the audio output device 52. Alternatively, if the surrounding monitoring device 150 finds that a monitored object that is no longer detected by the detection unit 3012 may still be present in a proximity area, it may display the number of undetected monitored objects on the display device 50 or output it as an audible message through the audio output device 52. In other words, the surrounding monitoring device 150 can provide the operator with information in any way that indicates the possibility that a monitored object may still be present in the proximity area relatively close to the shovel 100, including the monitoring area of ​​the detection unit 3012. Specifically, it can provide information that a monitored object not detected by the detection unit 3012 may still be present in the proximity area of ​​the shovel 100, including the monitoring area. This allows the surrounding monitoring device 150 to inform the operator and workers around the shovel 100 that there may be an undetected monitored object in the proximity area of ​​the shovel 100, even if the monitored object has not been detected by the detection unit 3012.

[0370] [Transformation / Modification] Although the second embodiment has been described in detail above, various modifications and changes are possible within the scope of the gist of this embodiment.

[0371] For example, in this embodiment, the monitoring area of ​​the peripheral monitoring device 150 consists only of an area defined (extending) horizontally as viewed from the shovel 100, that is, a horizontal monitoring area. However, it may also include a monitoring area in the vertical direction as viewed from the shovel 100, that is, an up-and-down monitoring area (hereinafter, for convenience, referred to as the "vertical monitoring area"). The peripheral monitoring device 150 in which a monitoring area including a vertical monitoring area in addition to the horizontal monitoring area is set will be described below with reference to Figures 17 and 18.

[0372] Figures 17 and 18 show other and yet another example of the configuration of the peripheral monitoring device 150, respectively.

[0373] As shown in Figure 17, in this example, the imaging device 40 includes cameras 40B, 40L, and 40R, as well as camera 40U, which images the upper region AU of the shovel 100 (upper rotating body 3).

[0374] The camera 40U can be positioned at any location as long as it can capture images of the entire upper part (house section) of the upper rotating body 3 extending from the rear to the left side of the cabin 10. For example, it can be attached to the upper rear end of the cabin 10. This allows the camera 40U to output captured images to the controller 30 that include objects to be monitored (e.g., people such as workers) that may be present on the upper part of the upper rotating body 3. As a result, the detection unit 3012 can detect objects to be monitored in the upper region AU of the shovel 100 (upper rotating body 3) within the proximity area of ​​the shovel 100.

[0375] Furthermore, as shown in Figure 18, in this example, the imaging device 40 includes cameras 40B, 40L, and 40R, as well as camera 40D, which images the area AD below the shovel 100 (upper rotating body 3).

[0376] The camera 40D can be positioned in any location as long as it can capture images of the entire space between the upper rotating body 3 and the ground below it, for example, by being mounted on the lower rear end of the upper rotating body 3. This allows the camera 40D to output images to the controller 30 that include objects to be monitored (e.g., people such as workers) that may be present in the space between the upper rotating body 3 and the ground below it, including the space between the left and right pairs of crawlers of the lower traveling body 1. As a result, the detection unit 3012 can detect objects to be monitored in the area AD below the shovel 100 (upper rotating body 3) within the proximity area of ​​the shovel 100.

[0377] Furthermore, naturally, the imaging device 40 of the surrounding monitoring device 150 may include both camera 40U and camera 40D. This allows the detection unit 3012 to detect monitoring targets in both the upper region AU and the lower region AD of the shovel 100 within the proximity area of ​​the shovel 100.

[0378] Furthermore, in place of or in addition to cameras 40U and 40D, other sensors (for example, the stereo camera, millimeter-wave radar, LIDAR, etc.) capable of outputting detection information regarding objects located in the upper region AU and lower region AD of the shovel 100 may be provided. This allows the detection unit 3012 to detect objects to be monitored in the upper region AU and lower region AD of the shovel 100 based on detection information from the other sensors.

[0379] Thus, the monitoring area of ​​the peripheral monitoring device 150 may include not only the horizontal monitoring area but also a vertical monitoring area that includes the upper area AU and lower area AD of the shovel 100. This allows the peripheral monitoring device 150 to detect the monitored object again within the vertical monitoring area even if the monitored object moves out of the horizontal monitoring area into the area adjacent to the monitoring area and can no longer be detected. In other words, the blind spot of the detection unit 3012 in the area adjacent to the shovel 100, which is the area adjacent to the monitoring area, can be reduced. Therefore, the safety of the shovel 100 can be further improved.

[0380] Furthermore, in this embodiment, the monitoring area (horizontal monitoring area) of the surrounding monitoring device 150 does not include the forward region AF of the shovel 100, but it may include the forward region AF. This reduces the blind spot of the detection unit 3012 in the area close to the shovel 100 that is outside the monitoring area. Thus, the safety of the shovel 100 can be further improved. In this case, the imaging device 40 of the surrounding monitoring device 150 may include a camera that images the area in front of the shovel 100, in addition to cameras 40B, 40L, and 40R. Alternatively, in place of, or in addition to, the camera that images the area in front of the shovel 100, other sensors capable of outputting detection information regarding objects present in front of the shovel 100 (e.g., the stereo camera, millimeter-wave radar, LIDAR, etc.) may be provided. This allows the detection unit 3012 to detect the object being monitored within the monitoring area that includes the forward region AF of the shovel 100.

[0381] Furthermore, in this embodiment, the surrounding monitoring device 150 includes both the alarm output unit 3014 and the operation limiting unit 3015, but it may also include only one of them. In other words, the surrounding monitoring device 150 may be configured to output an alarm and limit the operation of the shovel 100 when it detects a monitored object within the monitoring area. In this case, the surrounding monitoring device 150 will naturally continue to output an alarm and limit the operation of the shovel 100 in situations where the monitored object, which is no longer detected, may still be present in the vicinity of the shovel 100, including the monitoring area.

[0382] Furthermore, in this embodiment, the shovel 100 was configured to have all of its various operating elements, such as the lower traveling body 1, upper rotating body 3, boom 4, arm 5, and bucket 6, hydraulically driven. However, some or all of these elements may be electrically driven. In other words, the configurations disclosed in the above-described embodiment may be applied to hybrid shovels, electric shovels, and the like.

[0383] Furthermore, the following is disclosed regarding this embodiment.

[0384] (1) A detection unit that detects a predetermined monitoring target within a monitoring range set around the work machine, A peripheral monitoring device comprising: a control unit that outputs an alarm or restricts the operation of the work machine when the monitoring target is detected by the detection unit, When the monitoring target that was detected by the detection unit is no longer detected, the operator is provided with information regarding the possibility that the monitoring target may still be present in a proximity area relatively close to the work machine, including the monitoring range. Peripheral monitoring device.

[0385] (2) When the monitoring target that was detected by the detection unit is no longer detected, the operator is provided with information indicating that the monitoring target may still be present in the nearby area outside the monitoring range. (1) The peripheral monitoring device described above.

[0386] (3) When the monitoring target that was detected by the detection unit is no longer detected, the operator is provided with information indicating that the monitoring target may still be present within the monitoring range. (1) or (2) the peripheral monitoring device described above.

[0387] (4) Even if the monitored object that was detected by the detection unit is no longer detected after the output of the alarm or the start of the operation restriction, the control unit may continue the output of the alarm or the operation restriction depending on the circumstances under which the monitored object transitions from being detected to not being detected by the detection unit. A peripheral monitoring device as described in any one of items (1) to (3).

[0388] (5) If, after the output of the alarm or the start of the operation restriction, the control unit continues to output the alarm or the operation restriction if the monitored object that was detected by the detection unit is no longer detected at a position relatively close to the work machine within the monitoring range, (4) Peripheral monitoring device as described above.

[0389] (6) The system further includes a determination unit that determines whether or not the monitored object may still be present in the adjacent area outside the monitoring range, based on the position of the monitored object within the monitoring range when it was detected by the detection unit, when the monitored object is no longer detected. The matching unit determines that if the monitored object, which was detected by the detection unit, is no longer detected at a position relatively close to the work machine in the horizontal direction as seen from the work machine within the monitoring range, there is a possibility that the monitored object is still present in the nearby area outside the monitoring range. Even if the monitored object that was detected by the detection unit is no longer detected after the output of the alarm or the start of the operation restriction, if the validity determination unit determines that the monitored object may still be present in the nearby area outside the monitoring range, the control unit will continue to output the alarm or the operation restriction. (5) Peripheral monitoring device as described above.

[0390] (7) The monitoring range includes, in the horizontal direction as viewed from the work machine, a first range closest to the work machine, a second range furthest from the work machine, and a third range between the first range and the second range. The control unit, after the output of the alarm or the start of the operation restriction, releases the output of the alarm or the operation restriction if the monitored object previously detected by the detection unit is no longer detected within the second range, and continues the output of the alarm or the operation restriction if the monitored object previously detected by the detection unit is no longer detected within the first or third range. (5) or (6) Peripheral monitoring device as described above.

[0391] (8) If, after the alarm is output or the operation restriction is initiated, the control unit releases the alarm output or the operation restriction is released if the monitored object, which was detected by the detection unit, moves out of the monitoring range to a distant area that is further away from the monitoring range in the horizontal direction from the perspective of the work machine. A peripheral monitoring device as described in any one of items (4) through (7).

[0392] (9) The system further includes an exit determination unit that determines whether the monitored object has moved out of the monitoring range into the distant region, based on the position of the monitored object within the monitoring range when it was detected by the detection unit, when the monitored object is no longer detected by the detection unit. If the control unit determines, after the output of the alarm or the start of the operation restriction, that the monitored object detected by the detection unit has moved out of the monitoring range into the distant area, it releases the output of the alarm or the operation restriction. Peripheral monitoring device as described in 8.

[0393] (10) The aforementioned monitoring range includes a horizontal monitoring range that extends horizontally when viewed from the work machine, If, after the output of the alarm or the start of the operation restriction, the control unit continues the output of the alarm or the operation restriction if the monitored object, which was detected by the detection unit, is no longer detected at the lateral end of the horizontal monitoring range as seen from the work machine, or at the end of the horizontal monitoring range facing the work machine. A peripheral monitoring device as described in any one of items (1) through (9).

[0394] (11) The system further includes a counting unit that counts the number of monitored objects that are no longer detected by the detection unit, even though the exit determination unit has not determined that they have moved from the monitoring range to the distant area. The control unit releases the alarm output or the operation restriction if, after the alarm output or the operation restriction begins, the detection unit no longer detects the monitored object and the number of monitored objects counted by the count unit is zero. (9) Peripheral monitoring device as described above.

[0395] [Third Embodiment] Next, a third embodiment will be described.

[0396] Conventionally, techniques for preventing contact between excavators and surrounding workers are known (for example, Japanese Patent Publication No. 2011-163835).

[0397] However, at construction sites, changes in conditions can occur not only due to internal factors such as the movement of excavators and workers, but also due to external factors such as the entry of trucks for material delivery and waste removal. Therefore, even if an excavator operator is aware of their relationship with the workers inside the construction site, they may be slow to notice, or even unaware of, an approaching truck from outside. Thus, it is desirable to ensure the safety of excavators in a way that responds to the overall changes in the construction site conditions, including both internal and external factors.

[0398] Therefore, this embodiment provides a shovel and the like that can respond to changes in the overall conditions of a construction site and improve safety.

[0399] [Overview of the On-Site Safety Support System] First, with reference to Figure 19, the SYS2 field safety support system according to this embodiment will be described.

[0400] Figure 19 is a schematic diagram showing an example of the configuration of the SYS2 field safety support system.

[0401] The SYS2 site safety support system includes an excavator 100 and a support device 200. In response to user operation, the support device 200 sends a command (hereinafter referred to as a "warning command") to the excavator 100, which triggers a warning to the operator of the excavator 100 about the surrounding area. This allows the user of the support device 200 to warn the operator of the excavator 100 about the surrounding area from outside the excavator 100. The SYS2 site safety support system may include one or multiple excavators 100. Similarly, the SYS2 site safety support system may include one or multiple support devices 200. In other words, a single support device 200 can, for example, send warning commands to multiple excavators 100 within the same construction site. Furthermore, a single shovel 100 can, for example, alert its operator to the surrounding area in response to warning commands from multiple support devices 200 held by multiple workers working at the same construction site.

[0402] In Figures 19 and 20, for convenience, one excavator 100 and one support device 200 are shown. Furthermore, the SYS2 site safety support system may include other types of work machinery in place of, or in addition to, the excavator 100. For example, the SYS2 site safety support system may include a lifting magnet machine with a lifting magnet attached as an end attachment, a bulldozer, a wheel loader, an asphalt finisher, forestry machinery, a crawler crane, etc.

[0403] <Overview of an excavator> The shovel 100 comprises a lower traveling body 1, an upper rotating body 3 mounted on the lower traveling body 1 so as to be rotatable via a slewing mechanism 2, a boom 4, an arm 5, and a bucket 6 as attachments (working equipment), a cabin 10, and an engine 11.

[0404] The lower traveling body 1 includes, for example, a pair of left and right crawlers, and each crawler is hydraulically driven by a travel hydraulic motor, thereby enabling it to move under its own power.

[0405] The upper rotating body 3 rotates relative to the lower traveling body 1 by being hydraulically driven by a rotating hydraulic motor or electrically driven by an electric motor.

[0406] The boom 4 is pivotally attached to the front center of the upper slewing body 3 so as to be able to move up and down. The arm 5 is pivotally attached to the tip of the boom 4 so as to be able to rotate up and down. The bucket 6 is pivotally attached to the tip of the arm 5 so as to be able to rotate up and down. The boom 4, arm 5, and bucket 6 are hydraulically driven by the boom cylinder 7, arm cylinder 8, and bucket cylinder 9, respectively.

[0407] Cabin 10 is the cockpit where the operator sits and is mounted on the front left side of the upper rotating body 3.

[0408] The engine 11 is the power source for the shovel 100 and is mounted, for example, at the rear of the upper slewing body 3. The engine 11 is, for example, a diesel engine that uses light oil as fuel. The engine 11 operates, for example, to maintain a predetermined rotational speed. The rotation shaft of the engine 11 is connected to a main pump that supplies hydraulic fluid to hydraulic actuators, including a travel hydraulic motor, a slewing hydraulic motor, a boom cylinder 7, an arm cylinder 8, and a bucket cylinder 9, as well as a pilot pump that generates the source pressure for pilot pressure to operate the hydraulic actuators.

[0409] Shovel 100 can communicate with support device 200 via P2P (Peer to Peer) connection using a predetermined short-range communication method such as Bluetooth® communication, WiFi® communication, or RF (Radio Frequency) communication. This allows shovel 100 to acquire (receive) various information from support device 200, including the aforementioned warning commands and other command information. Further details will be described later.

[0410] Furthermore, the shovel 100 may be configured to communicate with the support device 200 via a wide-area communication network, such as a mobile communication network or the Internet, with a base station as the endpoint, using a P2P (Peer to Peer) connection. Alternatively, the shovel 100 and the support device 200 may each be configured to communicate with a management server installed outside the construction site, a management terminal installed in the building of the construction site's management office, etc., via a communication network, and communicate with each other via the management server, management terminal, etc.

[0411] <Overview of support device> The support device 200 (an example of an external device) is, for example, a portable terminal carried by a user (hereinafter referred to as "worker, etc.") who may be present around the excavator 100 at a construction site, such as a worker, supervisor, or maintenance person at the construction site. The support device 200 is, for example, a general-purpose notebook PC, tablet terminal, or smartphone carried by the user. Alternatively, the support device 200 may be a dedicated terminal for transmitting command information such as warning commands to the excavator 100 (for example, a remote control terminal including an operating part such as a button for transmitting the aforementioned specific command information such as warning commands).

[0412] As described above, the support device 200 can establish a P2P connection with the shovel 100 via a predetermined short-range communication method such as Bluetooth or Wi-Fi, and communicate with each other. This allows the support device 200 to transmit command information, such as the aforementioned warning commands, to the shovel 100 in response to user operations. Further details will be described later.

[0413] [Configuration of the on-site safety support system] Next, with reference to Figure 19 and Figure 20, the specific configuration of the SYS2 field safety support system according to this embodiment will be described.

[0414] Figure 20 is a diagram showing an example of the configuration of the SYS2 field safety support system.

[0415] <Shovel configuration> The shovel 100 includes a controller 30, an imaging device 40, a display device 50, an audio output device 52, a gate lock valve 54, and a communication device 60.

[0416] The controller 30 controls the drive of the shovel 100. The functions of the controller 30 may be realized by any hardware, software, or a combination thereof. The controller 30 is mainly composed of a microcomputer including a CPU (Central Processing Unit), RAM (Random Access Memory), ROM (Read Only Memory), auxiliary storage device, RTC (Real Time Clock), and various communication interfaces. The same applies to the control device 210 of the support device 200, which will be described later.

[0417] For example, the controller 30 controls the monitoring of the area around the shovel 100. Specifically, the controller 30 controls the imaging device 40 to acquire information about the surrounding conditions of the shovel 100, that is, captured images showing the conditions (appearance) around the shovel 100. The controller 30 also detects predetermined monitoring targets within a predetermined monitoring range around the shovel 100 based on the information (captured images) acquired from the imaging device 40. At this time, the monitoring targets may include not only people such as workers and construction site supervisors working around the shovel 100, but also any objects such as stationary obstacles such as construction site materials and soil, and mobile obstacles (i.e., moving objects) such as other construction machinery and trucks at the construction site.

[0418] Furthermore, for example, the controller 30 controls the operation of the shovel 100 to alert the operator of the shovel 100 to the surrounding area. Specifically, the controller 30 alerts the operator of the shovel 100 to the shovel 100 in response to an alert command received from the support device 200.

[0419] The controller 30 includes, for example, a monitoring image display processing unit 3021, a detection unit 3022, a communication processing unit 3023, an alarm output unit 3024, and an operation limiting unit 3025, as functional units realized by executing one or more programs stored in ROM or a non-volatile auxiliary storage device on the CPU.

[0420] The imaging device 40 is mounted on the upper part of the upper rotating body 3 and images the area around the shovel 100. The imaging device 40 includes cameras 40B, 40L, and 40R.

[0421] Cameras 40B, 40L, and 40R are mounted on the upper rear end, upper left end, and upper right end of the upper rotating body 3, respectively, and capture images of the rear, left, and right sides of the upper rotating body 3. For example, cameras 40B, 40L, and 40R are each monocular wide-angle cameras with a very wide field of view. Specifically, cameras 40B, 40L, and 40R are mounted on the upper part of the upper rotating body 3 with their optical axes pointing diagonally downwards, and capture an image range in the vertical direction, including from the ground near the shovel 100 to far away from the shovel 100. Cameras 40B, 40L, and 40R each output captured images at predetermined intervals (e.g., 1 / 30 second) while the shovel 100 is in operation, and the output captured images are taken into the controller 30.

[0422] The display device 50 is installed in the cabin 10 around the cockpit, specifically in a position easily visible to the operator seated in the cockpit, and displays various image information to notify the operator. The display device 50 may be, for example, a liquid crystal display or an organic EL (electroluminescence) display, and may also be a touch panel type with a touch panel that also serves as an operation unit. Specifically, as described later, the display device 50 displays images captured by the imaging device 40 (hereinafter sometimes referred to as "through images") as monitoring images showing the surroundings of the shovel 100, or composite images generated (combined) based on the images captured by the imaging device 40 (for example, viewpoint transformation images described later).

[0423] The audio output device 52 is installed around the cockpit in the cabin 10 and outputs various audio information to notify the operator. The audio output device 52 is, for example, a speaker or a buzzer. Specifically, the audio output device 52 outputs an alarm sound.

[0424] The gate lock valve 54 is located at the uppermost part of the pilot line that supplies pilot pressure from the aforementioned pilot pump to the operating devices used by the operator to control various operating elements of the shovel 100 (i.e., the lower traveling body 1, the upper slewing body 3, the boom 4, the arm 5, and the bucket 6, etc.), and switches the connection / disconnection of the pilot line.

[0425] For example, the gate lock valve 54 switches the connection / disconnection of the pilot line in accordance with the output signal (ON / OFF) of a gate lock switch, which is usually linked to the operating state of a gate lock lever located in the area corresponding to the entrance to the cockpit inside the cabin 10. Specifically, the gate lock valve 54 connects the pilot line when the output signal of the gate lock switch corresponds to the state in which the gate lock lever is raised (i.e., the state in which an operator is seated in the cockpit) (hereinafter referred to as the "OFF signal" for convenience). On the other hand, the gate lock valve 54 disconnects the pilot line when the output signal of the gate lock switch corresponds to the state in which the gate lock lever is lowered (i.e., the state in which an operator has left the cockpit) (hereinafter referred to as the "ON signal" for convenience).

[0426] Furthermore, for example, the gate lock valve 54 is configured to also accept command signals input from the controller 30. Specifically, the signal input section of the gate lock valve 54 may be connected to a logic circuit that can interrupt with command signals from the controller 30. This allows the gate lock valve 54 to switch the pilot line between connected and disconnected in response to command signals (OFF signals / ON signals) from the controller 30. In other words, under the control of the controller 30, the gate lock valve 54 can disconnect the pilot line even when the gate lock lever is raised.

[0427] The communication device 60 is a device that conforms to a predetermined short-range communication standard, such as Bluetooth communication, WiFi communication, or RF communication, which enables mutual communication in a relatively small area, i.e., within a construction site. Various information received from the outside by the communication device 60 is taken into the controller 30.

[0428] The monitoring image display processing unit 3021 displays the monitoring image on the display device 50 based on the image captured by the imaging device 40.

[0429] For example, the monitoring image display processing unit 3021 displays the image captured by at least one of the cameras 40B, 40L, and 40R on the display device 50 as a monitoring image.

[0430] Furthermore, for example, the monitoring image display processing unit 3021 generates a separate image (hereinafter referred to as the "surrounding image") representing the surroundings of the shovel 100 as a monitoring image, based on the image captured by the imaging device 40. Specifically, the monitoring image display processing unit 3021 may generate the surrounding image by combining the images captured by cameras 40B, 40L, and 40R, or by performing a process to transform the viewpoint. The monitoring image display processing unit 3021 then displays the monitoring image, including the generated surrounding image, on the display device 50.

[0431] More specifically, the monitoring image display processing unit 3021 generates a viewpoint-transformed image as seen from a virtual viewpoint by performing a known viewpoint transformation process based on the images captured by cameras 40B, 40L, and 40R as surrounding images, and displays it on the display device 50. In addition, when the monitoring image display processing unit 3021 displays the surrounding images on the display device 50, it also displays a shovel image schematically representing the shovel 100 on the display device 50 in order to clearly show the relative positional relationship between the imaging range of the imaging device 40 and the shovel 100. That is, the monitoring image display processing unit 3021 generates a monitoring image that includes the shovel image and surrounding images arranged around the shovel image in accordance with the relative positional relationship between the shovel 100 and the imaging range of the imaging device 40, and displays it on the display device 50.

[0432] For example, the display device 50 displays the monitoring images shown in Figures 5A and 5B described above.

[0433] As shown in Figures 3A and 3B, in this example, the display device 50 includes a display unit 50A which is a display area on which various information images are displayed, and an operation unit 50B which is a hardware unit such as a button switch that can accept operations by an operator or the like regarding the various information images displayed on the display unit 50A.

[0434] The display unit 50A is, for example, a horizontally elongated rectangular display in the display device 50 (for example, a display with an aspect ratio of 4:3).

[0435] As shown in Figure 3A, in this example, the display unit 50A displays a pass-through image from one of the cameras 40B, 40L, or 40R as the monitoring image MP1, as described above.

[0436] In this example, the monitoring image MP1 (through image) is overlaid with a guideline GLa. The guideline GLa represents, for example, a position where the horizontal distance D from the shovel 100 is a predetermined distance D1. This allows operators, etc., to understand how far away the monitored object is from the shovel 100 if it is visible (included) in the through image.

[0437] Furthermore, the predetermined distance D1 can be set as appropriate within a range of less than or equal to the predetermined distance D2 corresponding to the monitoring area described later.

[0438] Furthermore, as shown in Figure 3B, in this example, the display unit 50A displays a monitoring image MP2, which includes the shovel image CG and the surrounding image EP arranged around the shovel image CG, as described above. This allows the operator to appropriately understand the positional relationship between the monitoring target shown in the surrounding image EP and the shovel 100.

[0439] The peripheral image EP in this example is a viewpoint transformation image that combines an overhead view image BVP of the surrounding area adjacent to the shovel 100 viewed from directly above, and a horizontal view image HVP of the surrounding area viewed horizontally from the shovel 100, which is positioned around the overhead view image BVP. The peripheral image EP, which is a viewpoint transformation image, is obtained by projecting the images captured by cameras 40B, 40L, and 40R onto a spatial model, and then reprojecting the projected images onto that spatial model onto another two-dimensional plane. The spatial model is the projection target of the captured images in virtual space, and consists of one or more planes or curved surfaces, including planes or curved surfaces other than the plane on which the captured images are located.

[0440] Furthermore, the guideline GLb is superimposed on the monitoring image MP2. Similar to the guideline GLa in the monitoring image MP1 (through image) in Figure 3A, the guideline GLb represents a position where the horizontal distance D from the shovel 100 is a predetermined distance D1. In other words, the guideline GLb is a line obtained by offsetting (enlarging) the contour shape of the shovel image CG, which corresponds to a plan view from directly above the shovel 100, by an amount equivalent to a predetermined distance D1 outside the shovel image CG. This allows operators, etc., to understand how far away the monitoring target etc. is located from the shovel 100, if it is visible (included) in the surrounding image EP, similar to the case of guideline GLa in Figure 3A.

[0441] Returning to Figures 19 and 20, the detection unit 3022 detects the object to be monitored within the monitoring area around the shovel 100, specifically, within the monitoring area in the horizontal direction as seen from the shovel 100, that is, along the plane on which the shovel 100 is working (where the lower traveling body 1 is in contact with the ground), based on the image captured by the imaging device 40. Specifically, the detection unit 3022 detects the object to be monitored within the monitoring area where the horizontal distance D from the shovel 100 is within a predetermined distance D2 (for example, 5 meters).

[0442] For example, the monitoring area is shown as in Figure 6 above.

[0443] In the diagram, the dotted line L1 and the dashed line L2 correspond to positions where the horizontal distance D from the shovel 100 is a predetermined distance D1 and a predetermined distance D2, respectively.

[0444] As shown in Figure 6, the monitoring area MA (shaded area in the figure) is defined as the range within which the horizontal distance D from the shovel 100 is less than or equal to a predetermined distance D2, which is determined by the imaging ranges AVB, AVL, and AVR defined by the horizontal field of view of cameras 40B, 40L, and 40R, respectively.

[0445] Returning to Figures 19 and 20, for example, the detection unit 3022 recognizes the target to be monitored in the captured image by arbitrarily applying various known image processing techniques or machine learning-based classifiers, including artificial intelligence (AI).

[0446] Furthermore, the detection unit 3022 can determine (estimate) the location of the recognized surveillance target (hereinafter referred to as "actual location," for example, the foot position of a person being monitored) included in the image captured by the monocular imaging device 40 by applying various known methods.

[0447] For example, the detection unit 3022 estimates the horizontal position (hereinafter referred to as "horizontal position") of the recognized monitored object based on its size on the captured image (for example, its height on the captured image). This is because the size of the recognized monitored object on the captured image is correlated to decreasing as the monitored object moves further away from the shovel 100. Specifically, a range of assumed size (for example, a range of assumed human height) can be defined for the monitored object, and therefore, a correlation between the horizontal position of the monitored object, which is within the assumed size range and can be viewed from the shovel 100, and its size on the captured image can be defined in advance. For this reason, the detection unit 3022 can estimate the actual location of the recognized monitored object (horizontal position from the shovel 100) based on a map or conversion formula that represents the correlation between the size of the monitored object on the captured image and its horizontal position as viewed from the shovel 100, which is pre-stored in the internal memory of the controller 30, such as an auxiliary storage device.

[0448] Furthermore, for example, the detection unit 3022 can estimate the actual location (e.g., the position at the user's feet) of the monitored object by projecting the captured image onto the same plane as the shovel 100 (or its lower traveling body 1) (homography). In this case, a certain part (a certain point) of the captured image is associated with a certain position on the same plane as the shovel 100.

[0449] Furthermore, the detection unit 3022 may detect the target to be monitored within the monitoring area around the shovel 100 based on detection information from other sensors that may be mounted on the shovel 100, in addition to or in addition to the image captured by the imaging device 40. For example, the detection unit 3022 may detect the target to be monitored within the monitoring area around the shovel 100 based on detection information from other sensors such as a stereo camera, millimeter-wave radar, or LIDAR (Light Detecting and Ranging).

[0450] The communication processing unit 3023 controls the communication equipment 60 and establishes a P2P connection between the shovel 100 and the support devices 200 held by workers and others in the vicinity of the shovel 100 using the predetermined short-range communication method described above.

[0451] Specifically, first, the communication processing unit 3023 detects devices capable of P2P connection via a predetermined short-range communication method (hereinafter referred to as "connectable devices") in the vicinity of the shovel 100.

[0452] For example, the communication processing unit 3023 intermittently transmits a signal (hereinafter referred to as the "advertisement signal") containing identification information of the excavator 100 or the device corresponding to the communication device 60 within a predetermined range around the excavator 100, i.e., within the communication range (for example, several meters to several tens of meters), via the communication device 60. As a result, connectable devices such as the support device 200 can receive the advertisement signal when they are located within the communication range of the excavator 100 (i.e., when they enter the communication range of the excavator 100). The communication processing unit 3023 can then detect the connectable device that sent the connection request signal by receiving a signal requesting a P2P connection (hereinafter referred to as the "connection request signal") from the connectable device that received the advertisement signal via the communication device 60.

[0453] Furthermore, for example, the communication processing unit 3023 can detect the connectable device that is the source of the advertisement signal by receiving the advertisement signal through the communication device 60.

[0454] Next, the communication processing unit 3023 determines whether the connected device detected around the excavator 100 is a device to be connected, that is, a support device 200. For example, the communication processing unit 3023 determines whether it is a device to be connected based on various information for identifying the source of the signal contained in the advertisement signal or connection request signal.

[0455] Finally, when the communication processing unit 3023 determines that the detected connectable device is the device to be connected to, it establishes a P2P connection with the detected connectable device.

[0456] For example, when the communication processing unit 3023 detects a connectable device at the source by receiving a connection request signal through the communication device 60, it performs processing for P2P connection on the shovel 100 side. Then, the communication processing unit 3023 establishes a P2P connection with the detected connectable device, i.e., the support device 200, by sending a signal (hereinafter referred to as a "connection response signal") to the connectable device via the communication device 60 to notify it that the connection is complete.

[0457] Furthermore, for example, if the communication processing unit 3023 detects a connectable device at the source by receiving an advertised signal through the communication device 60, it sends a connection request signal to that connectable device. The communication processing unit 3023 then receives a connection response signal from the connectable device through the communication device 60, and in response to this reception, performs processing for P2P connection on the shovel 100 side to establish a P2P connection with the detected connectable device, i.e., the support device 200.

[0458] In this way, the communication processing unit 3023 can establish a P2P connection when the shovel 100 and the connected equipment (support device 200) are within each other's communication range.

[0459] Furthermore, the communication processing unit 3023 may establish a P2P connection between the shovel 100 and the support device 200 only when requested by the support device 200 (for example, when a connection request signal is received from the support device 200).

[0460] Furthermore, the communication processing unit 3023 receives various information transmitted from the support device 200, with which a P2P connection has been established between the shovel 100 and the support device 60, via the communication device 60.

[0461] For example, when the communication processing unit 3023 (an example of the first information acquisition unit and the second information acquisition unit) receives command information such as a warning command or a release command from the support device 200 via the communication device 60, it notifies the alarm output unit 3024 and the operation restriction unit 3025 accordingly.

[0462] The alarm output unit 3024 outputs an alarm to both the inside and outside of the cabin 10 of the shovel 100.

[0463] For example, the alarm output unit 3024 outputs an alarm to the inside or outside of the cabin 10 when the detection unit 3022 detects a monitored object within the monitoring area. This allows the controller 30 to make the operator, workers around the shovel 100, supervisors, etc., aware that a monitored object has entered the monitoring area around the shovel 100. As a result, the controller 30 can prompt the operator to check the safety conditions around the shovel 100 and prompt workers, etc., within the monitoring area to evacuate from the monitoring area. Hereinafter, this alarm will be conveniently referred to as the "monitored object detection alarm".

[0464] Furthermore, for example, when the alarm output unit 3024 (an example of a warning unit) receives a warning command from the support device 200 via the communication processing unit 3023, it outputs an alarm to the operator in the cabin 10 to warn them about the surrounding conditions of the shovel 100. This allows the controller 30 to prompt the operator to check the surrounding conditions of the shovel 100. Hereinafter, this alarm will be conveniently referred to as a "warning alarm".

[0465] The alarm output unit 3024 may output an alarm (monitoring target detection alarm or warning alarm) in any manner.

[0466] For example, the alarm output unit 3024 outputs an alarm in an auditory way, that is, by sound.

[0467] Specifically, the alarm output unit 3024 may output a control command to the audio output device 52, causing it to output a warning sound. In this case, the alarm output unit 3024 may vary the pitch, sound pressure, timbre, etc. of the warning sound, as well as the sounding period when the warning sound (e.g., a buzzer sound) is sounded periodically, depending on various conditions (for example, the type of alarm, i.e., whether it is a monitoring target detection alarm or a warning alarm).

[0468] Furthermore, the alarm output unit 3024 may output voice messages through an audio output device 52 such as a speaker indicating that a monitored object has been detected, or voice messages instructing (urging) the user to check the surrounding conditions of the shovel 100.

[0469] Furthermore, for example, the alarm output unit 3024 outputs an alarm in a visual manner, that is, by displaying an informational image on the display device 50.

[0470] Specifically, the alarm output unit 3024 transmits an alarm request to the monitoring image display processing unit 3021. In response to the alarm request, the monitoring image display processing unit 3021 highlights the monitored object included in the monitoring image (through image or surrounding image) displayed on the display device 50, thereby outputting a monitored object detection alarm to the operator via the display device 50.

[0471] Furthermore, the alarm output unit 3024 can output a warning alert to the operator by displaying text information on the display device 50 prompting the operator to check the area around the shovel 100.

[0472] Furthermore, the alarm output unit 3024 may output alarms by tactile means. Specifically, for example, the alarm may be output through a vibration generator that vibrates the cockpit where the operator is seated. In this case, the alarm output unit 3024 may vary the magnitude and frequency of the vibrations output from the vibration generator depending on the type of alarm, that is, whether it is a monitoring target detection alarm or a warning alarm.

[0473] Further, the alarm output unit 3024 may vary the type (alarm level) of the monitored object detection alarm according to the positional relationship between the monitored object detected within the monitoring area by the detection unit 3022 and the excavator 100.

[0474] For example, when the monitored object detected within the monitoring area by the detection unit 3022 is located at a relatively distant position from the excavator 100, the alarm output unit 3024 outputs a monitored object detection alarm with a relatively low alarm level (hereinafter, "monitored object detection alarm of the caution level") that prompts an operator or the like to pay attention to the monitored object. Specifically, the alarm output unit 3024 may output a monitored object detection alarm of the caution level when the horizontal distance D between the monitored object detected by the detection unit 3022 and the excavator 100 exceeds a predetermined distance D1 (when D1 < D ≤ D2). Hereinafter, the area within the monitoring area where the distance D from the excavator 100 exceeds the predetermined distance D1 is referred to as the "caution area" for convenience. On the other hand, when the monitored object detected within the monitoring area by the detection unit 3022 is located at a relatively close position to the excavator 100, the alarm output unit 3024 outputs an alarm with a relatively high alarm level (hereinafter, "monitored object detection alarm of the warning level") that notifies that the monitored object is approaching the excavator 100 and the risk level is increasing. Specifically, the alarm output unit 3024 may output an alarm of the warning level when the horizontal distance D between the monitored object detected by the detection unit 3022 and the excavator 100 is less than or equal to the predetermined distance D1 (when D ≤ D1). Hereinafter, the area within the monitoring area where the distance D from the excavator 100 is less than or equal to the predetermined distance D1 is referred to as the "warning area".

[0475] In this case, the alarm output unit 3024 may differentiate the pitch, sound pressure, timbre, and alarm cycle of the sound output from the voice output device 52 between the warning level alarm and the alert level alarm. The alarm output unit 3024 may also differentiate the color, shape, size, presence or absence of flashing, and flashing cycle of markers that highlight the monitored object included in the monitoring image (through image or surrounding image) displayed on the display device 50 between the warning level alarm and the alert level alarm. This allows the controller 30 to allow the operator to understand the alarm level, that is, the degree of proximity to the monitored shovel 100, based on the differences in the alarm sound and the markers that highlight the monitored object displayed on the display device 50.

[0476] Furthermore, the alarm output unit 3024 cancels the alarm output after the start of outputting the monitored target detection alarm if the monitored target that was detected by the detection unit 3022 is no longer detected, or if a predetermined cancellation operation is performed.

[0477] Furthermore, after the alarm output unit 3024 starts outputting a warning alarm, if the communication processing unit 3023 receives a release command from the support device 200, it will stop the warning alarm.

[0478] Furthermore, the alarm output unit 3024 may stop the warning alarm without receiving a release command if the operator has checked the surrounding conditions of the shovel 100. In this case, for example, the alarm output unit 3024 may stop the warning alarm after a predetermined time (for example, 1 minute) has elapsed since the start of the warning alarm output, during which it can be determined that the operator has checked the surrounding conditions of the shovel 100. Alternatively, for example, the alarm output unit 3024 may check the operator's line of sight using a line-of-sight sensor installed in the cabin 10, and stop the warning alarm if it determines that the operator has checked the surrounding conditions of the shovel 100. Alternatively, for example, the alarm output unit 3024 may display text information on the display device 50 asking whether the surrounding conditions of the shovel 100 have been checked, and then stop the warning alarm if the operator performs a confirmation operation indicating that the surrounding conditions of the shovel 100 have been checked. Alternatively, for example, when an operator performs a predetermined operation, a signal requesting the cancellation of the warning alert (hereinafter referred to as the "cancellation request signal") may be transmitted from the shovel 100 to the support device 200, the source of the warning alert, via the communication processing unit 3023. In this case, when the support device 200 receives the cancellation request signal from the shovel 100, it may display an operation screen on the display device 240 (described later) to authorize the cancellation of the warning alert, and upon receiving authorization from the user, it may send a signal to the shovel 100 indicating that cancellation is permitted (hereinafter referred to as the "cancellation permission signal"). The alarm output unit 3024 then stops the warning alert when it receives the cancellation permission signal from the support device 200 via the communication processing unit 3023. Furthermore, even if the alarm output unit 3024 does not receive a cancellation permission signal from the support device 200, it may stop the warning alert after a predetermined time has elapsed since the transmission of the cancellation request signal.

[0479] The operation limiting unit 3025 restricts the operation of the operating elements of the shovel 100 when predetermined conditions for ensuring safety around the shovel 100 are met. At this time, the operating elements subject to operation restriction include some or all of the lower traveling body 1, upper slewing body 3, boom 4, arm 5, and bucket 6. For example, the operation limiting unit 3025 transmits a command signal (ON signal) to the gate lock valve 54, thereby causing the pilot line to be disconnected from the gate lock valve 54. As a result, hydraulic fluid (pilot pressure) is not supplied from the pilot pump to the operating device that operates the operating elements of the shovel 100, so even if the operator inputs an operation to the operating device, the operating elements of the shovel 100 are restricted from operating. Therefore, collisions with objects in a relatively close area around the shovel 100 can be suppressed.

[0480] For example, the operation limiting unit 3025 restricts the operation of the operating elements of the shovel 100 when the detection unit 3022 detects a monitored object within the monitoring area. As a result, even if a monitored object (e.g., a person) enters the monitoring area around the shovel 100, the operation of the shovel 100 is restricted, and collisions between the shovel 100 and the monitored object can be suppressed. Therefore, safety around the shovel 100 can be maintained.

[0481] Furthermore, for example, the operation limiting unit 3025 restricts the operation of the excavator 100's operating elements when it receives a warning command from the support device 200 via the communication processing unit 3023. This helps to suppress collisions between the excavator 100 and objects that trigger a warning (for example, a truck entering the construction site) when a warning command is sent from the support device 200, that is, when the operator should pay attention to the surrounding conditions of the excavator 100. As a result, safety around the excavator 100 can be maintained.

[0482] In this case, the operation limiting unit 3025 may start limiting the operation of the shovel 100 after the operator has been alerted to the surrounding conditions of the shovel 100, that is, after the alarm output unit 3024 has started outputting an alert warning. Specifically, the operation limiting unit 3025 may start limiting the operation of the shovel 100 after a predetermined time (for example, a few seconds) has elapsed after the alarm output unit 3024 has started outputting an alert warning. This ensures that the operation of the shovel 100 is limited only after the operator has confirmed the alert warning from the alarm output unit 3024. Therefore, it is possible to prevent situations where the sudden start of limiting the operation of the shovel 100 may cause the operator to feel uneasy, or where the shovel 100 becomes unstable due to the sudden stopping of operating elements during operation.

[0483] Furthermore, the operation restriction unit 3025 (an example of a notification unit) may, after the alarm output unit 3024 has started outputting a warning alarm, and before the operation restriction of the shovel 100 begins, notify the operator that the operation restriction will begin (hereinafter referred to as "operation restriction warning notification") via the display device 50 or the voice output device 52. This allows the user to recognize that the operation restriction of the shovel 100 will begin before it actually starts. Therefore, it is possible to further suppress situations such as the operator feeling uneasy due to the sudden start of the operation restriction of the shovel 100, or the shovel 100 becoming unstable due to the sudden stopping of operating elements during operation. In this case, the operation restriction warning notification displayed on the display device 50 may include the remaining time until the operation restriction of the shovel 100 begins, shown in countdown format. This allows the user to understand when the operation restriction of the shovel 100 will actually begin.

[0484] Alternatively, instead of the gate lock valve 54, a pressure reducing valve may be provided to reduce the secondary pilot pressure output from the operating device that operates the operating elements, and the operation limiting unit 3025 may control the pressure reducing valve. In this case, the pressure reducing valve can disable the operator's operation of the operating device by reducing the secondary pilot pressure output from the operating device to a pressure value below zero. Therefore, the operation limiting unit 3025 can, by controlling the pressure reducing valve, thereby limiting the operation of the operating elements of the shovel 100. Furthermore, the operation limiting unit 3025 may, by controlling the pressure reducing valve, limit the operation of the shovel 100 in a manner that suppresses the operation of various operating elements in response to operator operation more gently than usual, without stopping the operating elements of the shovel 100.

[0485] Furthermore, similar to the case of the alarm output unit 3024, the operation limiting unit 3025 may vary the level of operation restriction of the shovel 100 caused by the detection of the monitored object, depending on the positional relationship between the monitored object detected by the detection unit 3022 and the shovel 100.

[0486] For example, when the monitored object detected within the monitoring area by the detection unit 3022 is located at a relatively far position from the excavator 100, the operation restriction unit 3025 restricts the operation of the excavator 100 in a relatively low-restriction manner. Specifically, when the horizontal distance D between the monitored object detected by the detection unit 3022 and the excavator 100 exceeds a predetermined distance D1 (when D1 < D ≤ D2), the operation restriction unit 3025 may restrict the operation of the excavator 100 in a relatively low-restriction manner (for example, making the operating speed of the operation element with respect to the operation amount slower than normal). On the other hand, when the monitored object detected within the monitoring area by the detection unit 3022 is located at a relatively close position to the excavator 100, the operation restriction unit 3025 restricts the operation of the excavator 100 in a relatively high-restriction manner. Specifically, when the horizontal distance D between the monitored object detected by the detection unit 3022 and the excavator 100 is less than or equal to the predetermined distance D1 (when D ≤ D1), the operation restriction unit 3025 may restrict the operation of the excavator 100 in a relatively high-restriction manner (for example, stopping the operation of the operation element).

[0487] Further, after the start of the operation restriction of the excavator 100 due to the detection of the monitored object by the detection unit 3022, when the monitored object detected by the detection unit 3022 is no longer detected, or when the above-described release operation is performed, the operation restriction unit 3025 releases the operation restriction of the excavator 100.

[0488] Also, after the start of the operation restriction of the excavator 100 due to the reception of the warning instruction, when the operation restriction unit 3025 receives a release instruction from the support device 200, the operation restriction unit 3025 releases the operation restriction of the excavator 100.

[0489] Furthermore, if both the conditions for implementing the operation restriction related to the detection of a monitored object by the detection unit 3022 and the conditions for implementing the operation restriction related to the receipt of a warning command are met, the operation restriction unit 3025 will not release the operation restriction of the shovel 100 unless the conditions for releasing both are met. Specifically, the operation restriction unit 3025 may not release the operation restriction of the shovel 100 unless the conditions for releasing both are met, regardless of whether the trigger for the operation restriction of the shovel 100 is the detection of a monitored object by the detection unit 3022 or the receipt of a release command from the support device 200, if both conditions are met. This further improves the safety of the shovel 100.

[0490] <Configuration of support device> The support device 200 includes a control device 210, a communication device 220, an operating device 230, and a display device 240.

[0491] The control device 210 controls various operations of the support device 200. The control device 210 includes a communication processing unit 2104, a warning command unit 2105, and a release command unit 2106, which are functional units realized by executing one or more programs stored in ROM or a non-volatile auxiliary storage device on the CPU.

[0492] The communication device 220 is a device that conforms to the same short-range communication standards as the communication device 60 of the shovel 100, such as Bluetooth communication, WiFi communication, or RF communication as described above. As a result, the communication device 220 can receive signals transmitted from the communication device 60 of the shovel 100 and transmit signals that can be received by the communication device 60 of the shovel 100.

[0493] The operating device 230 receives various operations from the user on the support device 200. The operating device 230 includes, for example, hardware-based operating parts such as buttons, keyboards, mice, touchpads, and touch panels implemented on the display device 50. Alternatively, the operating device 230 may be a combination of hardware-based operating parts such as touch panels implemented on the display device 240 and software-based operating parts such as button icons on the operation screen displayed on the display device 240.

[0494] The display device 240 displays various information images. The display device 240 is, for example, a liquid crystal display or an organic EL display.

[0495] The communication processing unit 2104 controls the communication equipment 220 to establish a P2P connection between the support device 200 and the shovel 100 in the vicinity of the support device 200 using the predetermined short-range communication method described above. The method for establishing the P2P connection between the communication processing unit 2104 and the shovel 100 is the same as the method for establishing the P2P connection between the shovel 100 and the support device 200 by the communication processing unit 3023 of the shovel 100, except that the connection destination is the shovel 100 instead of the support device 200, so a detailed explanation is omitted.

[0496] Furthermore, the communication processing unit 2104 controls the communication equipment 220 and transmits various command information to the shovel 100 in response to requests from the warning command unit 2105, the release command unit 2106, etc.

[0497] When a user, such as a worker at a construction site, performs an operation via the operating device 230 to instruct the warning command unit 2105 to send a warning command (hereinafter referred to as "warning command transmission operation"), the warning command unit 2105 transmits the warning command to the shovel 100 via the communication processing unit 2104. This allows users, such as workers at a construction site, to have the shovel 100 alert the operator to the surrounding conditions when they want the operator to check the surrounding conditions (for example, when a truck entering the construction site approaches the shovel 100).

[0498] For example, Figure 21 shows an example of an operation screen (hereinafter referred to as the "warning operation screen") (warning operation screen 500) on which an operation to send a warning command displayed on the display device 240 is performed.

[0499] As shown in Figure 21, the warning operation screen 500 includes button icons 501 and 502 that can be operated by an operating device 230 such as a touch panel mounted on the display device 240.

[0500] The button icon 501 is labeled with the text "Send," and the user can send a warning command by operating the button icon 501 via the operating device 230. In other words, when the button icon 501 is operated via the operating device 230, the warning command unit 2105 sends a warning command to the shovel 100 via the communication processing unit 2104. At this time, if the support device 200 is connected to multiple shovels 100 via P2P, the warning command unit 2105 broadcasts the warning command to multiple shovels 100. This allows the user to send a warning command to multiple shovels 100, each issuing a warning about the surrounding conditions to its respective operator.

[0501] Furthermore, if the support device 200 is connected via P2P to multiple excavators 100, an operation screen for selecting the destination excavator 100 may be displayed on the display device 240. This allows the user to send a warning command only to the excavator 100 corresponding to the operator who needs to be alerted.

[0502] The button icon 502 is labeled with the text "Back," and by operating the button icon 502 via the operating device 230, the user can return the display content of the display device 50 to the screen that was displayed immediately before the warning operation screen 500 was shown.

[0503] Returning to Figures 19 and 20, the release command unit 2106 transmits a release command to the shovel 100 via the communication processing unit 2104 when an operation to instruct a user, such as a worker at the construction site, to transmit a release command (hereinafter referred to as the "release command transmission operation") is performed via the operating device 230. This allows the user, such as a worker at the construction site, to release the warning alarm or the operation restriction of the shovel 100, for example, when a truck or the like that has entered the construction site moves far away from the shovel 100.

[0504] For example, Figure 22 shows an example of an operation screen (hereinafter referred to as the "release operation screen") (release operation screen 600) on which a release command transmission operation is performed, which is displayed on the display device 240.

[0505] As shown in Figure 22, the release operation screen 600 includes button icons 601 and 602 that can be operated by an operating device 230 such as a touch panel mounted on the display device 240.

[0506] The button icon 601 is labeled with the text "Send," and the user can send a release command by operating the button icon 601 via the operating device 230. In other words, when the button icon 601 is operated via the operating device 230, the release command unit 2106 sends a release command to the shovel 100 via the communication processing unit 2104. At this time, if the support device 200 is P2P connected to multiple shovels 100, the release command unit 2106 broadcasts the release command to multiple shovels 100. Alternatively, the release command unit 2106 may select only the shovels 100 to which a warning command has been previously sent from among the multiple shovels 100 that are P2P connected to the support device 200, and send the release command to the selected shovels 100 by multicast or unicast. This allows the user to have multiple shovels 100 alert the operator to the surrounding conditions and release the operation restrictions with a single release command transmission operation.

[0507] Furthermore, if the support device 200 is connected via P2P between multiple excavators 100, an operation screen for selecting the destination excavator 100 may be displayed on the display device 240. This allows the user to select the excavator 100 to which the release command will be issued, and to have the operator be alerted to the surrounding conditions and the operation restrictions lifted.

[0508] [Overview of the operation of the on-site safety support system] Next, with reference to Figure 23, we will explain the overview of the operation of the SYS2 field safety support system.

[0509] Figure 23 is a diagram illustrating the overview of the operation of the SYS2 site safety support system. Specifically, Figure 23 shows the situation at a construction site where worker W, carrying a shovel 100 and a support device 200, is working, and the flow of operation of the shovel 100 and the support device 200. In this example, the explanation will be based on the premise that worker W is within the communication range of the communication device 60 of the shovel 100, and that the shovel 100 is within the communication range of the communication device 220 of the support device 200.

[0510] In step S10, the shovel 100 and the support device 200 establish a P2P connection between them.

[0511] At step S12, truck TK enters the construction site and approaches excavator 100.

[0512] In step S14, worker W notices that truck TK has entered the construction site and sends a warning command to the support device 200. In response to worker W's command, the support device 200 sends a warning command to the shovel 100.

[0513] In step S16, the shovel 100 receives a warning command from the support device 200 and, in response to the warning command, starts outputting a warning alert.

[0514] For example, as shown in Figure 23, the controller 30 (alarm output unit 3024) of the shovel 100 displays a warning alarm ALM on the monitoring image MP1 (through image) displayed on the display device 50, prompting the operator to check the area around the shovel 100. This allows the operator to easily understand that a warning alarm has been issued while working and checking the monitoring image displayed on the display device 50. Furthermore, the operator can simultaneously check the start of the warning alarm output and the surrounding situation of the shovel 100 displayed on the monitoring image MP1 (for example, the situation of the worker W or a truck entering the construction site).

[0515] Furthermore, if the controller 30 of the shovel 100 is displaying an information image other than the monitoring image on the display device 50, it may automatically switch the display content of the display device 50 to the monitoring image (for example, a monitoring image including the through image and surrounding image from camera 40B). This improves operator convenience by eliminating the need for operators to perform operations to check the surrounding conditions in places that are difficult to directly check, such as behind the shovel 100.

[0516] Then, after the warning command is issued, the shovel 100 begins to restrict its operation. This allows the shovel 100 (controller 30) to prevent accidents such as contact with an approaching truck TK caused by the operation of the shovel 100.

[0517] In step S18, when worker W confirms that truck TK has moved away from shovel 100, he sends a release command to support device 200, and in response to worker W's transmission, support device 200 sends a release command to shovel 100.

[0518] In step S20, the shovel 100 receives a release command from the support device 200 and, in response to the release command, releases the warning alarm and operational restrictions. This allows the shovel 100 to return to normal operation.

[0519] In this way, the support device 200 transmits a command (warning command) to the shovel 100 to alert the operator to the surrounding conditions of the shovel 100, in response to the user's (worker W's) operation. The shovel 100 then receives the warning command from the support device 200 and alerts the operator to the surrounding conditions of the shovel 100, specifically by outputting a warning alarm. Normally, the operator focuses on the operation of the shovel 100's attachments, making them less likely to notice changes in the construction site conditions due to external factors than an external worker W. Therefore, the site safety support system SYS2 can improve the safety of the shovel 100 by prompting the operator to check the surrounding conditions of the shovel 100 in response to changes in the construction site conditions due to external factors such as the entry of a truck TK into the construction site. Thus, the site safety support system SYS2 can respond to changes in the overall conditions of the construction site and improve safety.

[0520] Furthermore, the shovel 100 restricts its operation in response to the receipt of a warning command. In other words, the support device 200 alerts the operator of the shovel 100 to the surrounding conditions and transmits a warning command to the shovel 100 to restrict its operation. As a result, the site safety support system SYS2 can restrict the operation of the shovel 100 in response to changes in the construction site conditions due to external factors such as the entry of truck TK into the construction site, thereby further improving the safety of the shovel 100.

[0521] Furthermore, worker W can also send a warning command to the support device 200 if, for example, another worker working around the shovel 100 makes a sudden movement or collapses due to a sudden change in their physical condition, or if worker W themselves collapses due to a sudden change in their physical condition. If something happens to a worker working around the shovel 100, there is a possibility that the worker may enter the area that the shovel 100 passes through due to its path or the rotation of the upper rotating body 3. Therefore, the site safety support system SYS2 can further improve the safety of the shovel 100 by prompting the operator to check the surrounding conditions of the shovel 100 or by restricting the operation of the shovel 100 in response to changes in the construction site conditions due to internal factors such as something happening to a worker.

[0522] Furthermore, the shovel 100 receives a warning command from the support device 200 as information regarding changes in the surrounding conditions of the shovel 100 (hereinafter referred to as "warning information" for convenience), but warning information may be obtained by means other than a warning command. In other words, the worker W may communicate changes in the surrounding conditions of the shovel 100 to the shovel 100 by other means. For example, the worker W may perform a predetermined gesture or pose toward the shovel 100 to notify it of changes in the surrounding conditions of the shovel 100, such as the approach of a truck TK. In this case, the controller 30 of the shovel 100 can recognize the gesture or pose by the worker W by performing image recognition processing on the image captured by the imaging device 40. In other words, the shovel 100 may obtain an image captured by the imaging device 40 that includes a predetermined person, such as a worker, performing a predetermined pose or gesture in the vicinity of the shovel 100, as warning information. As a result, the shovel 100 can recognize a worker W performing a predetermined gesture or pose from the image captured by the imaging device 40, and can alert the operator to the surrounding conditions of the shovel 100 or restrict the operation of the shovel 100. Similarly, the shovel 100 may acquire information regarding the release of warning alerts or operation restrictions for the shovel 100 (hereinafter referred to as "release information" for convenience) from an external source by means other than a release command. Specifically, the shovel 100 may acquire release information through the image captured by the imaging device 40, which includes a predetermined person, such as a worker, performing a predetermined gesture or pose different from that in the case of warning information.

[0523] Furthermore, worker W may not carry the support device 200 individually, but may use the support device 200 installed near the work area at the construction site to send a warning command to the shovel 100. In other words, one or more support devices 200 may be installed in the work area where the shovel 100 is working, and may be shared by workers working in the work area.

[0524] [Details of the operation of the on-site safety support system] Next, with reference to Figure 24, we will explain the details of the operation of the SYS2 field safety support system.

[0525] Figure 8 is a flowchart illustrating an example of the process by which the controller 30 of the shovel 100 alerts the operator to the surrounding conditions. The process shown in this flowchart is executed by the communication processing unit 3023 when it receives an alert command from the support device 200.

[0526] In step S402, the alarm output unit 3024, upon receiving a warning command from the support device 200 via the communication processing unit 3023, outputs a warning alarm to the operator in the cabin 10 via the display device 50, the voice output device 52, etc.

[0527] In step S404, the operation restriction unit 3025 issues an operation restriction warning via the display device 50 and the audio output device 52 after the alarm output unit 3024 starts issuing a warning alarm. This allows the operator to understand that the operation restriction of the shovel 100 will begin and to stop operating the shovel 100 in advance.

[0528] In step S406, the operation limiting unit 3025 determines whether a predetermined time has elapsed since the alarm output unit 3024 started outputting a warning alarm. If the predetermined time has elapsed since the start of the warning alarm output, the operation limiting unit 3025 proceeds to step S408. If the predetermined time has not elapsed, it waits until the predetermined time has elapsed (the process of this step is repeated).

[0529] In step S408, the operation limiting unit 3025 controls the gate lock valve 54 and starts limiting the operation of the shovel 100.

[0530] In step S410, the communication processing unit 3023 determines whether or not a release command has been received from the support device 200. If a release command is received from the support device 200, the communication processing unit 3023 proceeds to step S412; otherwise, it waits until a command is received (repeating the processing in this step).

[0531] Furthermore, if the controller 30 has not received a release command after a certain amount of time (for example, several tens of minutes) has elapsed from the receipt of the warning command, which is deemed to be sufficiently longer than the time required to ensure safety around the shovel 100, it may consider that safety has been ensured around the shovel 100 and proceed to step S412. This is because there may be cases where the shovel 100 cannot receive a release command due to communication failures or the user of the support device 200 forgetting to send a release command.

[0532] In step S412, the alarm output unit 3024 and the operation restriction unit 3025 release the warning alarm and the operation restriction of the shovel 100, respectively, and the process ends.

[0533] [Transformation / Modification]

[0534] Although the second embodiment has been described in detail above, various modifications and changes are possible within the scope of the gist of this embodiment.

[0535] For example, in this embodiment, the shovel 100 (communication processing unit 3023) establishes a P2P connection with the support device 200, but it may also establish a P2P connection with other shovels 100 in the same site. In this case, one shovel 100 may transmit various command information, such as warning commands and release commands, to other shovels 100 in response to operations by an operator, similar to the case of the support device 200. This allows, for example, the operator of one shovel 100 to have the other shovels 100 warn the operator about the surrounding situation or temporarily restrict their operation if they detect a truck or the like approaching another shovel 100.

[0536] Furthermore, in this embodiment, the shovel 100 acquires warning information and release information from objects within the same construction site (other shovels 100, support devices 200, workers, etc.), but is not limited to this embodiment. Specifically, the shovel 100 may acquire warning information (e.g., warning commands) from objects entering the site from the outside, that is, objects that are external factors causing changes in the conditions of the construction site (e.g., trucks, etc.), and in response to such warning information, may issue a warning to the operator regarding the surrounding conditions of the shovel 100. In this case, the truck (an example of an external device) may be equipped with a communication device that performs short-range communication using the same predetermined method as the communication device 60 of the shovel 100. As a result, the driver of a truck entering the construction site from the outside can perform a predetermined operation when approaching the shovel 100 in operation, thereby causing the truck to send a warning command to the shovel 100. Therefore, the truck driver can have the shovel 100 alert the operator to the surrounding conditions of the shovel 100, ensuring that the operator notices the truck being driven. When the truck driver determines that the operator of the shovel 100 has noticed the truck, the truck sends a release command to the shovel 100. This allows the truck driver to have the shovel 100 release the alert warning and the restrictions on its operation. Alternatively, the shovel 100 may receive alert commands from a device that can monitor the conditions of the construction site from outside the construction site, such as an external server installed outside the construction site that manages the work status of the construction site and the work machinery such as the shovel 100 (an example of an external device), and based on this alert information, it may alert the operator to the surrounding conditions of the shovel 100. In this case, the shovel 100 is equipped with communication equipment (e.g., a mobile communication module) for communicating with the management device via a communication network such as a mobile communication network or the ...

Claims

1. a sensor that acquires information about the situation around the work machine; a transmission unit that transmits information about the situation around the work machine, which is acquired by the sensor or generated based on information acquired by the sensor, to an external device; an activation unit that activates a monitoring unit including at least the sensor and the transmission unit in response to a command received from outside while the work machine is stopped; Equipped with Work machinery.

2. the sensor includes an imaging device that images the periphery of the work machine, the transmission unit transmits to the external device an image of the surroundings of the work machine captured by the imaging device, 2. The work machine according to claim 1.

3. When the monitoring unit is activated by the activation unit, activation of at least some of a plurality of devices that are activated in response to activation of the work machine is restricted. Claim 1 or 2 The work machine described in

4. a detection unit that detects a predetermined monitoring target within a predetermined range around the work machine based on information acquired by the sensor, the activation unit activates the detection unit in addition to the sensor and the transmission unit as the monitoring unit while the work machine is stopped, the transmission unit transmits information about the monitoring target in the vicinity of the work machine detected by the detection unit to the external device or records the information in a predetermined storage unit. Claim 1 to 3 Any one of the following The work machine described in

5. a power source that supplies power to equipment of the work machine including the monitoring unit, the monitoring unit is configured to be operable when the amount of power that can be supplied from the power source is relatively high while the work machine is stopped. Claim Any one of items 1 to 4 The work machine described in

6. the activation unit activates the monitoring unit when the amount of power that can be supplied from the power source exceeds a predetermined first threshold. Claim 5 The work machine described in

7. and further comprising a stop unit that stops the monitoring unit when the available power supply amount of the power source falls below a predetermined second threshold value in a case where the monitoring unit is started by the start unit while the work machine is stopped. Claim 5 or 6. The work machine according to claim 6.

8. and a first external notification unit that, when the available power supply amount of the power source is relatively low while the work machine is stopped, transmits a notification to the external device indicating that the monitoring unit cannot operate. Claim Any one of items 5 to 7 The work machine described in

9. the monitoring unit has an operation mode in which it operates with some functions restricted so that the amount of power consumed by the power source is relatively reduced while the work machine is stopped. Claim Any one of items 5 to 8 The work machine described in

10. a second external notification unit that notifies the external device of information regarding the amount of power that can be supplied from the power source or information regarding the time that the monitoring unit can operate with power supplied from the power source while the work machine is stopped, Claim Any one of items 5 to 9 The work machine described in

11. the power source includes at least one of a primary battery, a secondary battery, a capacitor, and a power generation means; Claim Any one of items 5 to 10 The work machine described in

12. an internal notification unit that notifies an operator that information relating to the situation around the work machine is being transmitted to the external device by the transmission unit while the work machine is in operation; Claim 1 Any one of items 1 to 11 The work machine described in

13. further comprising a permission operation unit that accepts a predetermined operation for permitting remote operation of the work machine, Claim 1 Any one of items 1 to 12 The work machine described in

14. An information processing device capable of communicating with a work machine, the information processing device comprising a monitoring unit that acquires information about a situation around the work machine and transmits the information to an outside of the work machine, a transmission unit that transmits to the work machine a command requesting information about the situation around the work machine in response to a predetermined operation by a user; a first acquisition unit that acquires information about the situation around the work machine, which information is transmitted from the work machine in response to the command; a first notification unit that notifies a user of information relating to the situation around the work machine acquired by the first acquisition unit, Information processing device.

15. a second acquisition unit that acquires, in response to the command, from the work machine, information regarding whether the monitoring unit in the work machine that is stopped can be started, or whether the monitoring unit that is operating can continue to operate after the monitoring unit has been started. Claim 1 4 The information processing device described in

16. a second acquisition unit that is mounted on the work machine and acquires information regarding a state of a power source that supplies power to the monitoring unit, Claim 1 4 or 15 The information processing device described in

17. An information processing method executed by an information processing device capable of communicating with a work machine, the information processing device having a monitoring unit that acquires information about a situation around the work machine and transmits the information to an outside of the work machine, a transmission step of transmitting to the work machine a command requesting information about a situation around the work machine in response to a predetermined operation by a user; a first acquisition step of acquiring information about a situation around the work machine, the information being transmitted from the work machine in response to the command; a first notification step of notifying a user of information relating to the situation around the work machine acquired in the first acquisition step, Information processing methods.

18. an information processing device capable of communicating with the work machine, the information processing device having a monitoring unit that acquires information about the situation around the work machine and transmits the information to an outside of the work machine; a transmission step of transmitting to the work machine a command requesting information about a situation around the work machine in response to a predetermined operation by a user; a first acquisition step of acquiring information about a situation around the work machine, the information being transmitted from the work machine in response to the command; a first notification step of notifying a user of information relating to the situation around the work machine, which information is acquired in the first acquisition step; program.

Images