Information presentation system

Abstract

To allow a worker to grasp information on a relative distance of a specific range, which is a range related to a specific part of a work machine, with respect to work-related equipment.SOLUTION: The information presenting system 1 includes an information presenting device 57b. The information presenting device 57b presents information corresponding to a relative distance L of a specific range R, which is a range relating to the specific location 16 of the work machine 10, with respect to a work-related facility 90, which is a facility arranged in the vicinity of the work position of the work machine 10.SELECTED DRAWING: Figure 4

Description

[Technical Field]

[0001] The present invention relates to an information presentation system that presents information about work being performed by a work machine. [Background technology]

[0002] For example, a conventional information presentation system is described in Patent Document 1. In the technology described in this document, information is presented in a manner according to the distance from the working body (specific range) of the work machine to the object of the work (earth and sand, etc.) (see claim 1, abstract, etc. of this document). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent Publication No. 2021-130973 Summary of the Invention [Problem to be solved by the invention]

[0004] With the technology described in this document, it may be difficult for a worker to grasp the relative distance of a specific range to the work-related equipment, for example, when the work-related equipment is hidden by an object of work.

[0005] Therefore, an object of the present invention is to provide an information presentation system that allows a worker to grasp information about the relative distance of a specific range of a work machine to work-related equipment. [Means for solving the problem]

[0006] The information presentation system includes an information presentation device that presents information according to a relative distance of a specific range relating to a specific location of a work machine relative to work-related equipment that is equipment located near the work machine. [Effects of the Invention]

[0007] The above-described information presentation system allows the worker to grasp information on the relative distance of a specific range of the work machine relative to the work-related equipment. [Brief explanation of the drawings]

[0008] [Figure 1] 1 is a diagram showing an information presentation system 1, in which a work machine 10 and work-related equipment 90 are seen from the side. [Figure 2] FIG. 2 is a block diagram of the information presentation system 1 shown in FIG. [Figure 3] 1. FIG. 4 is a diagram showing a manual setting screen G10 for setting the position information of the work-related facility 90 shown in FIG. [Figure 4] 3 is a diagram showing the display of the information presentation device 57b shown in FIG. 2, showing an image of the vehicle 90a and the specific location 16 as viewed from the lateral direction Y. FIG. [Figure 5] 4, showing an image of the vehicle 90a and the specific location 16 as viewed from the upper side Z1. [Figure 6] 4, showing an image of the vehicle 90a and the specific location 16 as viewed from the near side X2. [Figure 7] 4, showing an image of the retaining wall 90b, the pavement 90c, and the specific location 16 as viewed from the lateral direction Y. FIG. [Figure 8] 3 is a diagram showing the relationship between information presented by an information presentation device 57b shown in FIG. 2 and a relative distance L. FIG. [Figure 9] 3 is a flowchart of a process of the information presentation system 1 shown in FIG. 2. DETAILED DESCRIPTION OF THE INVENTION

[0009] The information presentation system 1 will be described with reference to FIGS.

[0010] The information presentation system 1 is a system that presents information according to the relative position between a specific range R of a work machine 10 shown in Fig. 1 and work-related equipment 90. The information presentation system 1 includes a work machine 10, a remote operation system 50, an information processing system 70, and work-related equipment 90.

[0011] The work machine 10 is a machine that performs work. The work machine 10 may be a construction machine that performs construction work, or a loading and unloading machine that performs loading and unloading work. The work machine 10 may be, for example, a shovel, a crane, or a bulldozer. The following describes a case where the work machine 10 is a shovel. The work machine 10 operates in response to operation by a worker (operator). For example, the work machine 10 may be operated (on-board operation) by a worker in a cab 13c (described later), or may be remotely operated from a remote operation system 50 external to the work machine 10. The work machine 10 may also operate under automatic control. The automatic control of the work machine 10 may be automatic operation or semi-automatic operation. The work machine 10 comprises a machine main body 10a, an attachment 15, a specific location 16, a drive control device 17 (see FIG. 2), and an actual machine system 30 (see FIG. 2).

[0012] The machine body 10a is the main body portion of the work machine 10. The machine body 10a includes a lower body 11 and an upper rotating body 13.

[0013] The lower body 11 rotatably supports the upper rotating body 13. The lower body 11 may be a lower traveling body that can travel on a traveling surface (such as the ground). When the lower body 11 is capable of traveling, the lower body 11 may be provided with crawlers or wheels.

[0014] The upper rotating body 13 is mounted rotatably on the lower main body 11. A boom 15a and the like are attached to the upper rotating body 13. The upper rotating body 13 is provided with a cab 13c. The cab 13c is a section where an operator can operate the work machine 10 (operate the actual machine from inside). As mentioned above, the work machine 10 may be remotely controlled.

[0015] (direction) The direction in which the rotation axis of the upper rotating body 13 relative to the lower body 11 extends is defined as the up-down direction Z. In the up-down direction Z, the side (facing) from the lower body 11 toward the upper rotating body 13 is defined as the upper side Z1, and the side opposite to the upper side Z1 is defined as the lower side Z2. The direction in which the rotation axis of the boom 15a relative to the upper rotating body 13 extends is defined as the lateral direction Y. The direction intersecting (e.g., perpendicular to) each of the up-down direction Z and the lateral direction Y is defined as the front-to-rear direction X. In the front-to-rear direction X, the side from which the attachment 15 protrudes relative to the upper rotating body 13 is defined as the rear side X1, and the side opposite to the rear side X1 is defined as the front side X2. Note that the up-to-down direction Z does not have to be the direction in which the rotation axis of the upper rotating body 13 relative to the lower body 11 extends, and may be a vertical direction. In this case, both the front-to-rear direction X and the lateral direction Y may be horizontal.

[0016] The attachment 15 is a part that performs work (work mechanism). The attachment 15 is attached to the machine body 10a. For example, the attachment 15 includes a boom 15a, an arm 15b, and a tip attachment 15c. The boom 15a is attached to the upper rotating body 13 so as to be rotatable (rotatable in the front-to-back direction X and the up-to-down direction Z). The arm 15b is attached to the boom 15a so as to be rotatable (rotatable in the front-to-back direction X and the up-to-down direction Z).

[0017] The tip attachment 15c is provided at the tip of the attachment 15. The tip attachment 15c is rotatably attached to the arm 15b (rotatable in the forward / backward direction X and the upward / downward direction Z). The tip attachment 15c may be a bucket capable of scooping and digging the work object W. The tip attachment 15c may be equipped with a device for clamping the work object W (a grapple, nibbler, rotary fork, etc.), a device for crushing the work object W (a breaker, etc.), or a magnet for attracting a metal work object W. The work object W is an object that is the target of work by the work machine 10. The work object W may be soil-like, granular, chip-like, powder-like, etc. The work object W may be soil or sand, rock, a magnetic material (metal, etc.), resin, waste, wood (logs, etc.), or a structure (block, etc.).

[0018] The specific location 16 is a specific location of the work machine 10. The specific location 16 is a location that serves as a reference for the specific range R. The specific location 16 may include a portion where work is performed on the work object W. For example, the specific location 16 may be all or part of the attachment 15, or all or part of the tip attachment 15c. The specific location 16 may include all or part of the machine body 10a.

[0019] The specific range R is a range relating to the specific location 16. The specific range R is a range (range, area, position in three-dimensional space) for which the relative distance L to the work-related equipment 90 is calculated by a relative distance calculation device 75b (see FIG. 2) described later. The specific range R is set in the controller 1c (described later). The specific range R may be the specific location existence range Ra or the specific location operation range Rb.

[0020] The specific location existence range Ra is a range in which the entity (object) of the specific location 16 exists. For example, if the specific location 16 is the tip attachment 15c, the specific location existence range Ra is a range in which the entity of the tip attachment 15c exists.

[0021] The specific location operating range Rb is all or part of the range (movable range, range of motion) in which the specific location 16 can operate. When only a portion of the range in which the specific location 16 can operate is set as the specific location operating range Rb, which part of the range in which the specific location 16 can operate is the specific location operating range Rb is set in advance in the controller 1c (before calculating the relative distance L). For example, the specific location operating range Rb is all or part of the range of the trajectory (movable range, range of motion) of the specific location 16 when it is assumed that the "reference location of the work machine 10" does not move and the specific location 16 moves. The "reference location of the work machine 10" and the specific location 16 can each be set to various locations on the work machine 10.

[0022] [Specific Example 1 of Specific Location Operation Range Rb] For example, the above-mentioned "reference location of work machine 10" may be the entire portion of work machine 10 excluding specific location 16. Specifically, as shown in the example in FIG. 1, specific location 16 may be tip attachment 15c, and the "reference location of work machine 10" may be machine body 10a, boom 15a, and arm 15b. In this case, specific location operation range Rb is all or part of the range of the trajectory of tip attachment 15c when it is assumed that tip attachment 15c rotates relative to arm 15b while machine body 10a, boom 15a, and arm 15b remain stationary.

[0023] [Specific Example 2 of Specific Location Operation Range Rb] For example, the above-mentioned "reference location of the work machine 10" may be only a portion of the work machine 10 excluding the specific location 16. [Specific Example 2a] Specifically, for example, the specific location 16 may be the tip attachment 15c, and the "reference location of the work machine 10" may be the machine body 10a. In this case, the specific location operation range Rb is all or part of the range of the trajectory of the tip attachment 15c, assuming that the machine body 10a does not move and the attachment 15 (boom 15a, arm 15b, and tip attachment 15c) moves within its movable range. [Specific Example 2b] For example, the specific location 16 may be the attachment 15, and the "reference location of the work machine 10" may be the lower body 11. In this case, the specific location operating range Rb is all or part of the range of the trajectory of the attachment 15 when it is assumed that the lower body 11 does not move (run), the upper rotating body 13 rotates at various angles, and the attachment 15 moves within its movable range.

[0024] The drive control device 17 (see FIG. 2) controls the actuators that move the work machine 10. The drive control device 17 may include a hydraulic circuit that controls a hydraulic actuator that is operated by hydraulic pressure. The drive control device 17 may include an electric circuit that controls an electric actuator that is operated by electricity. Specifically, the drive control device 17 controls the travel of the lower body 11. The drive control device 17 controls a swing motor that swings the upper rotating body 13 relative to the lower body 11. The drive control device 17 controls a boom cylinder that rotates (raises and lowers) the boom 15a relative to the upper rotating body 13. The drive control device 17 controls an arm cylinder that rotates the arm 15b relative to the boom 15a. The drive control device 17 controls a tip attachment cylinder that rotates the tip attachment 15c relative to the arm 15b. Below, the lower body 11, the upper rotating body 13, each part of the attachment 15 (boom 15a, arm 15b, tip attachment 15c), the specific location 16, and the specific range R will be described with reference to FIG. 1.

[0025] The actual machine system 30 (see FIG. 2) is equipment provided (mounted) on the work machine 10. As shown in FIG. 2, the actual machine system 30 includes an actual machine input interface 33, an actual machine control device 35, an actual machine output interface 37, and an actual machine communication device 39.

[0026] The actual machine input interface 33 is an input interface provided on the work machine 10. In FIG. 2, the actual machine input interface 33 is written as the "actual machine input I / F." The "input interface" is a device for inputting information to the controller 1c from outside the controller 1c (see the explanation of the actual machine control device 35 described below). The input interface can be arranged in various positions, and may be arranged on the work machine 10, the remote operation system 50, the information processing system 70, or elsewhere. The same can be said for the "operation device," "detection device," controller 1c, and "image output device," which will be described later, and the like, which can be arranged in various positions. The input interface arranged on (mounted on) the work machine 10 is the actual machine input interface 33.

[0027] This actual machine input interface 33 is a device for inputting information to the actual machine control device 35 from outside the actual machine control device 35. This "outside of the actual machine control device 35" may include, for example, the worker, or the environment around the work machine 10. The actual machine input interface 33 includes an actual machine operation device 33a and an actual machine sensor group 33b.

[0028] The actual machine operating device 33a is an operating device provided (mounted) on the work machine 10. The above-mentioned "operating device" is a device (input device) for inputting information. The operating device is operated by the worker and outputs a signal according to the operation. The operating device outputs information to the controller 1c. The operating device may be equipped with a touch panel, a mouse, a keyboard, or a switch. The operating device may be equipped on a tablet, a smartphone, or a personal computer. The operating device can be arranged in various positions. The operating device provided on the work machine 10 is the actual machine operating device 33a.

[0029] This actual machine operation device 33a is provided, for example, in the operator's cab 13c (see FIG. 1). The actual machine operation device 33a is a mechanism for operating the work machine 10. Specifically, the actual machine operation device 33a may be provided with an operation lever or an operation pedal. The actual machine operation device 33a outputs a command according to the amount of operation. The actual machine operation device 33a may be provided with a switch or the like provided on the operation lever, or may be provided with a switch or the like provided separately from the operation lever. The actual machine operation device 33a may be provided on a display (for example, a cluster gauge). The actual machine operation device 33a may be capable of performing operations related to information presentation by an information presentation device 57b (described later) (the actual machine operation device 33a may be an information presentation operation device 53a1 described later).

[0030] The actual machine sensor group 33b is a detection device provided (mounted) on the work machine 10. The above-mentioned "detection device" is a device (sensor) that detects various conditions. The detection device can be arranged in various positions. Some or all of the "detection device" does not have to be provided on the work machine 10, and may be arranged outside the work machine 10 (for example, at the work site). The following mainly describes the case where the detection device is the actual machine sensor group 33b (when provided on the work machine 10). The actual machine sensor group 33b includes an imaging device 33b1, a position detection device 33b2, an attitude detection device 33b3, and an equipment detection device 33b4.

[0031] The imaging device 33b1 captures an image of an object to be imaged. The object to be imaged by the imaging device 33b1 may include the surroundings (work environment) of the work machine 10. When the imaging device 33b1 is disposed outside the work machine 10, the object to be imaged may include the work machine 10. The imaging device 33b1 may detect a two-dimensional image, or may detect a three-dimensional image (distance image) having depth information. The imaging device 33b1 may be a passive type or an active type. Specifically, the imaging device 33b1 may be equipped with a camera (monocular camera) that detects two-dimensional information. The imaging device 33b1 may be equipped with a stereo camera that detects three-dimensional information. The imaging device 33b1 may detect three-dimensional information of the object to be imaged by irradiating the object with waves such as electromagnetic waves and detecting the reflected waves. The imaging device 33b1 may be equipped with a time-of-flight (TOF) sensor that detects distance based on the time from when the waves are emitted until the reflected waves return, or may be equipped with a sensor that detects distance based on the frequency of the reflected waves. The imaging device 33b1 may include a device that detects three-dimensional information using light (e.g., laser light), and may include, for example, LiDAR (Light Detection and Ranging). The imaging device 33b1 may include a device that detects three-dimensional information using radio waves (e.g., millimeter-wave radar). Only one imaging device 33b1 may be provided, or multiple imaging devices 33b1 may be provided. Only one type (e.g., one method) of imaging device 33b1 may be used, or multiple types of imaging devices 33b1 may be combined. The imaging device 33b1 may detect three-dimensional information of the imaging target based on a three-dimensional image (distance image) and a two-dimensional image.

[0032] The position detection device 33b2 detects the position of the object to be measured. For example, the position detection device 33b2 detects the position of a specific part (reference part) of the work machine 10. Specifically, the position detection device 33b2 may detect the position of a specific part of the upper rotating body 13, or may detect the position of a specific part of the attachment 15. The position detection device 33b2 may detect the position using electromagnetic waves (light, radio waves, etc.). The position detection device 33b2 may use a satellite positioning system, for example, a global navigation satellite system (GNSS). The position detection device 33b2 may use a (terrestrial) transmitter and receiver without using a satellite, or may use reflection of light (e.g., laser light) (e.g., a total station, etc.). The position detection device 33b2 may calculate the position of the object to be measured based on position information detected by multiple types of devices. In Figure 1, the position of the GNSS antenna is marked with the symbol for position detection device 33b2 when the position detection device 33b2 performs detection using a positioning system using GNSS (the same applies to the equipment position detection device 95 described later).

[0033] This position detection device 33b2 (see FIG. 2) detects the direction (orientation, posture) of the object to be measured. For example, the position detection device 33b2 shown in FIG. 2 detects the direction of a specific part of the work machine 10. Specifically, the position detection device 33b2 may detect the direction of a specific part of the upper rotating body 13, or may detect the direction of a specific part of the attachment 15. The position detection device 33b2 may detect the orientation of the object to be measured using geomagnetism. The position detection device 33b2 may detect the direction of the object to be measured based on the positions of multiple parts of the object to be measured relative to the work site. Note that an equipment position detection device 95 similar to the position detection device 33b2 may be provided in the work-related equipment 90.

[0034] The attitude detection device 33b3 detects the attitude of the work machine 10. The attitude detection device 33b3 may detect the inclination (tilt) of the work machine 10 with respect to a horizontal plane. The attitude detection device 33b3 may detect information (angle, angular velocity, angular acceleration, etc.) about the rotation of the upper rotating body 13 with respect to the lower main body 11. The attitude detection device 33b3 may detect information (angle, angular velocity, angular acceleration, etc.) about the rotation of the boom 15a with respect to the upper rotating body 13. The attitude detection device 33b3 may detect information about the rotation of the arm 15b with respect to the boom 15a. The attitude detection device 33b3 may be equipped with one or more types of detection devices. The attitude detection device 33b3 may be equipped with a detection device (e.g., a rotary encoder) that detects information about the angle of a certain element of the work machine 10 with respect to another element. The attitude detection device 33b3 may include a stroke sensor that detects the stroke of a cylinder that moves the attachment 15. The attitude detection device 33b3 may include an inclination sensor that detects the angle (inclination) with respect to the horizontal direction. The attitude detection device 33b3 may include a sensor (such as a gyro sensor) that detects angular velocity with respect to the work site, or a sensor that detects acceleration with respect to the work site. The attitude detection device 33b3 may include an inertial measurement unit or the like.

[0035] This attitude detection device 33b3 may be equipped with the position detection device 33b2 described above. In this case, the attitude detection device 33b3 may detect the attitude of a specific part of the work machine 10 based on position information detected by the position detection device 33b2. The attitude detection device 33b3 may detect the attitude of a specific part of the work machine 10 based on direction information detected by the position detection device 33b2. The attitude detection device 33b3 may be equipped with the imaging device 33b1 described above. The attitude detection device 33b3 may detect the attitude of the work machine 10 based on image recognition of two-dimensional images. The attitude detection device 33b3 may detect the attitude of the work machine 10 based on three-dimensional images. The attitude detection device 33b3 may detect the attitude of the work machine 10 based on three-dimensional images and two-dimensional images.

[0036] The equipment detection device 33b4 is a device (work-related equipment detection device) that detects information about the work-related equipment 90. The information that the equipment detection device 33b4 needs to detect varies depending on the method for acquiring the position information of the work-related equipment 90 (described later). For example, the equipment detection device 33b4 may detect the presence or absence of the work-related equipment 90. The equipment detection device 33b4 may detect the position of the work-related equipment 90 or may detect the shape of the work-related equipment 90, and may be equipped with an imaging device 33b1, for example. In the example shown in FIG. 2, the equipment detection device 33b4 is part of the actual machine sensor group 33b (a detection device provided on the work machine 10). On the other hand, the equipment detection device 33b4 may be provided outside the work machine 10, and may be arranged, for example, at the work site or on the work-related equipment 90 (e.g., vehicle 90a). An equipment position detection device 95, described later, may be included in the equipment detection device 33b4.

[0037] The actual machine control device 35 is a controller 1c provided (mounted) on the work machine 10. The controller 1c is a computer that inputs and outputs signals, performs calculations (processing), stores information, etc. The functions of the controller 1c are realized when a program stored in the memory unit of the controller 1c is executed by the calculation unit. Information is input to the controller 1c from an input interface (e.g., an actual machine input interface 33). The controller 1c outputs information to an output interface (e.g., an actual machine output interface 37). The controller 1c inputs and outputs information to a communication device (e.g., an actual machine communication device 39). The controller 1c may be distributed and disposed in multiple parts (may constitute a distributed system). The components of the controller 1c may be connected to each other via a network N (described later). The controller 1c can be disposed in various locations. The controller 1c provided (mounted) on the work machine 10 is the actual machine control device 35.

[0038] This actual machine control device 35 outputs commands (operation commands, operation signals) to the drive control device 17 for moving the work machine 10. The actual machine control device 35 outputs commands to the drive control device 17 for moving the work machine 10 in response to the operation of the actual machine operation device 33a or the remote operation device 53a.

[0039] The actual machine output interface 37 is an output interface provided on the work machine 10. In FIG. 2, the actual machine output interface 37 is written as the "actual machine output I / F." The "output interface" is something (a device) external to the controller 1c, to which information is output from the controller 1c. The "output interface" can be arranged in various positions. The output interface arranged (mounted) on the work machine 10 is the actual machine output interface 37. The actual machine output interface 37 is something (a device) external to the actual machine control device 35, to which information (signals, commands, etc.) is output from the actual machine control device 35. The drive control device 17 may be included in the actual machine output interface 37.

[0040] The actual machine communication device 39 is a communication device provided (mounted) on the work machine 10. The above-mentioned "communication device" is a device (communication equipment) that exchanges information (communicates) via the network N. The "communication device" is a device that allows multiple controllers 1c to communicate with each other. Specifically, the "communication device" is a device that allows the work machine 10 (actual machine system 30), the remote operation system 50, and the information processing system 70 to communicate with each other. The communication performed by the "communication device" (communication performed over the network N) may be wireless communication or wired communication. The "communication device" may be a wireless communication device or a wired communication device. The communication means of the "communication device" (network N) is specifically communication means such as a mobile phone line, optical fiber line, wireless LAN (Local Area Network), or wired LAN.

[0041] The remote operation system 50 is a system (for example, a remote cockpit) for remotely operating the work machine 10. The remote operation system 50 is used when the work machine 10 is remotely operated. Note that the work machine 10 does not have to be remotely operated, and the information presentation system 1 does not have to be equipped with the remote operation system 50. The remote operation system 50 comprises a remote input interface 53, a remote control device 55, a remote output interface 57, and a remote communication device 59.

[0042] The remote input interface 53 is an input interface provided in the remote operation system 50 (details of the input interface are described above). In FIG. 2, the remote input interface 53 is described as a "remote input I / F." The remote input interface 53 is a device for inputting information to the remote control device 55 from outside the remote control device 55 (for example, from an operator). The remote input interface 53 includes a remote operation device 53a.

[0043] The remote control device 53a is an operation device provided in the remote operation system 50 (details of the operation device are described above). The remote control device 53a is equipped with a device similar to the actual machine operation device 33a provided in the driver's cab 13c. The remote control device 53a is a mechanism for operating the work machine 10. Specifically, the remote control device 53a may be equipped with an operation lever, an operation pedal, or a switch provided on the operation lever or the like. The remote control device 53a may be equipped with a touch panel, a mouse, a keyboard, a switch, etc. (see the explanation of "operation device" above). The remote control device 53a is equipped with an information presentation operation device 53a1.

[0044] The information presentation operation device 53a1 is an operation device for performing operations related to information presentation by the information presentation device 57b. For example, the information presentation operation device 53a1 may be configured to be capable of performing an operation to determine whether or not information is presented by the information presentation device 57b. The information presentation operation device 53a1 may be configured to be capable of performing an operation to set a threshold value for information presentation (described later). The information presentation operation device 53a1 may be configured to be capable of performing an operation to set the mode of information presentation (display, sound, etc. (described later)).

[0045] This information presentation operation device 53a1 may be configured to enable operation of setting the work-related facility 90 (see FIG. 1). For example, the information presentation operation device 53a1 may be configured to enable operation (described later) of manually setting the position information (coordinates, shape) of the work-related facility 90. The information presentation operation device 53a1 may be configured to enable selection of the work-related facility 90 (selection by manual operation).

[0046] The information presentation operation device 53a1 may be configured to allow an operation to set the specific range R. For example, the information presentation operation device 53a1 may be configured to allow an operation to set (select) which part of the work machine 10 will be the specific location 16. The information presentation operation device 53a1 may be configured to allow an operation to select the type of specific location 16 (for example, to select the type of end attachment 15c). The information presentation operation device 53a1 may be configured to allow an operation to set (select) the relationship between the specific location 16 and the specific range R (specifically, whether it is a specific location existence range Ra or a specific location operation range Rb). For example, the information presentation operation device 53a1 may be configured to allow an operation to set (select) which part of the work machine 10 will be the above-mentioned "reference location of the work machine 10".

[0047] As described above, this information presentation operation device 53a1 can be arranged in various positions. The information presentation operation device 53a1 may be arranged in the remote operation system 50, or may be arranged in a position different from the remote operation system 50. The information presentation operation device 53a1 may be arranged in the work machine 10, for example, in the operator's cab 13c. The information presentation operation device 53a1 may be arranged in a position different from the work machine 10 and a position different from the remote operation system 50.

[0048] The remote control device 55 is a controller 1c provided in the remote operation system 50 (details of the controller 1c are described above). For example, when the work machine 10 is remotely operated, an image of the surroundings of the work machine 10 (work environment image) is input to the remote control device 55. The remote control device 55 outputs this work environment image to a remote image output device 57a (described below). The remote control device 55 transmits a command (remote operation command) corresponding to the operation of the remote operation device 53a to the actual machine control device 35.

[0049] The remote output interface 57 is an output interface provided in the remote operation system 50 (the output interface is described above). In FIG. 2, the remote output interface 57 is described as a "remote output I / F." The remote output interface 57 is an external device to the remote control device 55, to which information (signals, commands, etc.) is output from the remote control device 55. The remote output interface 57 includes a remote image output device 57a and an information presentation device 57b.

[0050] The remote image output device 57a is an image output device (monitor) provided in the remote operation system 50. The "image output device" is a device that outputs an image, and outputs an image in response to a signal output by the controller 1c. The "image output device" can be arranged in various positions. The "image output device" may be arranged inside the operator's cab 13c. The image output device provided in the remote operation system 50 is the remote image output device 57a. The remote image output device 57a displays an image of the work environment around the work machine 10 for remote operation. For example, the remote image output device 57a displays an image captured by the imaging device 33b1 mounted on the work machine 10. The remote image output device 57a may display various images other than work environment images.

[0051] The information presentation device 57b is a device that presents information. As shown in FIG. 4, the information presentation device 57b presents information (for example, proximity information, which will be described later) according to a relative distance L of a specific range R with respect to the work-related equipment 90. The information presentation device 57b may present information other than information according to the relative distance L. The information presentation device 57b shown in FIG. 2 may include a device (monitor) that displays an image, and may be, for example, the remote image output device 57a. The information presentation device 57b may include a device (speaker) that outputs sound. The information presentation device 57b may include a device that outputs vibration.

[0052] This information presentation device 57b presents information to a worker (person). For example, the information presentation device 57b may present information to a worker (operator) who operates the work machine 10. For example, the information presentation device 57b is provided in the remote control device 53a and presents information to an operator who remotely operates the work machine 10. For example, the information presentation device 57b may be provided in the cab 13c of the work machine 10, and may present information to an operator who operates the work machine 10 while actually on board. The information presentation device 57b may be arranged in a position other than the work machine 10 and the remote operation system 50. The information presentation device 57b may present information to a person other than the operator of the work machine 10 (for example, a supervisor, manager, etc.).

[0053] The remote communication device 59 is a communication device provided in the remote operation system 50 (the communication device is described above).

[0054] The information processing system 70 is a controller 1c provided separately from the actual machine control device 35 and the remote control device 55 (the controller 1c is described above). For example, the information processing system 70 may function as a server device, and the actual machine control device 35 and the remote control device 55 may function as client devices. The information processing system 70 includes a storage device 71 and a calculation device 75. As described above, each of the actual machine control device 35 and the remote control device 55 is also a controller 1c, like the information processing system 70, and is a device including a storage unit and a calculation unit.

[0055] The storage device 71 is a device that stores (preserves) information. The storage device 71 is a device that stores information to be processed by the arithmetic device 75. The storage device 71 stores a database. The database stored in the storage device 71 includes machine information 71a and equipment information 71b.

[0056] The machine information 71a is information about the work machine 10. The machine information 71a includes information for calculating position information of the specific range R. The machine information 71a includes information about the work machine 10 for calculating the relative distance L (see FIG. 1) of the specific range R with respect to the work-related equipment 90 (see FIG. 1). The machine information 71a may include information for calculating the relative distance L of the specific location presence range Ra with respect to the work-related equipment 90, or may include information for calculating the relative distance L of the specific location operation range Rb (see FIG. 1) with respect to the work-related equipment 90. For example, the machine information 71a includes information for calculating the position of the specific location 16 from the attitude of the work machine 10. Specifically, the machine information 71a includes specification information of each element of the work machine 10. The machine information 71a may include specification information for all or part of the lower body 11, upper revolving body 13, and each part of the attachment 15 (boom 15a, arm 15b, and tip attachment 15c). The "specification information" may include dimensional information, shape information, or movable range information. The machine information 71a may include three-dimensional shape information (e.g., a three-dimensional model). The machine information 71a may include numerical information representing dimensions. The machine information 71a may include information on the distance (dimension) between the rotation axes of each element. The machine information 71a may include information on the range of angles within which each element can rotate (movable range).

[0057] Specifically, the machine information 71a may include information on a position (e.g., the position of a GNSS antenna) that serves as a reference for position detection by the position detection device 33b2. The machine information 71a may include information on the position of the rotation axis (boom foot) of the boom 15a relative to the upper rotating body 13. The machine information 71a may include information on the length of the boom 15a, or may include information on the dimension from the boom foot to the boom top (the rotation axis of the arm 15b relative to the boom 15a). The machine information 71a may include information on the length of the arm 15b, or may include information on the dimension from the boom top to the arm top (the rotation axis of the tip attachment 15c relative to the arm 15b). The machine information 71a may include information on the tip attachment 15c. For example, the machine information 71a may include information on the dimension from the arm top to a specific portion of the tip attachment 15c. The "particular portion of the tip attachment 15c" may be, for example, the tip of the tip attachment 15c or the bottom of the tip attachment 15c, which is a bucket.

[0058] Equipment information 71b (work-related equipment information) is information about work-related equipment 90 (see FIG. 1). Equipment information 71b includes information for calculating the position information of the work-related equipment 90. Equipment information 71b includes information about the work-related equipment 90 for calculating the relative distance L (see FIG. 1) of the specific range R with respect to the work-related equipment 90. For example, equipment information 71b may include specification information about the work-related equipment 90 (e.g., dimensional information, shape information). Equipment information 71b may include information about three-dimensional shapes (e.g., a three-dimensional model). In cases where the work-related equipment 90 does not move, or where the position where the work-related equipment 90 stops is fixed, equipment information 71b may include information about the position (coordinates) of the work-related equipment 90.

[0059] The arithmetic device 75 is a device that performs various calculations (processing such as calculation and determination), and includes a relative position acquisition device 75a, a relative distance calculation device 75b, and an image processing device 75c.

[0060] The relative position acquisition device 75a acquires the position of the work-related facility 90 (see FIG. 4) and the position of the specific range R (details will be described later). The relative distance calculation device 75b calculates the relative distance L of the specific range R with respect to the work-related facility 90 (details will be described later).

[0061] The image processing device 75c performs image processing. The image processing device 75c may process images displayed by an image output device (e.g., the remote image output device 57a, the information presentation device 57b, etc.). The image processing device 75c may generate images. For example, the image processing device 75c may generate a manual setting screen G10 (see FIG. 3) described later. The image processing device 75c may generate a relative position display G21 (see FIG. 4) described later, or a relative distance display G23 (see FIG. 4). The image processing device 75c may output a signal (image signal) to the image output device to output an image. The image processing device 75c may perform processing to recognize an object from an image (image recognition), for example, processing to recognize one or both of the coordinates and shape of an object from an image.

[0062] The information processing system communication device 79 is a communication device provided in the information processing system 70 (the communication device is described above).

[0063] The work-related equipment 90 is an object (object to be calculated) for which the relative distance L to the specific range R shown in Figure 1 is calculated by the relative distance calculation device 75b (see Figure 2). The work-related equipment 90 is equipment that is placed near the work machine 10. The "near the work machine 10" above is, for example, a position that the work machine 10 may come into contact with when the work machine 10 is working. The "near the work machine 10" above may also be, for example, a position that the attachment 15 may come into contact with when the attachment 15 moves. The "near the work machine 10" above may also be, for example, within the range of detection of the work-related equipment 90 by a detection device (e.g., equipment detection device 33b4).

[0064] This work-related equipment 90 is not the object (work object W) on which the work machine 10 performs work. For example, the work-related equipment 90 is placed near the work object W. The work-related equipment 90 may come into contact with the work object W. The work-related equipment 90 may also be something on which the work object W is placed. The work-related equipment 90 does not have to be placed near the work object W.

[0065] Specific examples of this work-related equipment 90 are as follows. The work-related equipment 90 may include a vehicle 90a. The vehicle 90a may be a transport vehicle (described later) that transports the work object W, or may be other machinery. The work-related equipment 90 may include a container 91 (described later). As shown in FIG. 7, the work-related equipment 90 may include a wall, for example, a retaining wall 90b. The retaining wall 90b is a wall that receives soil and sand (e.g., the work object W). A container enclosure 91b (see FIG. 1), described later, may be included in the wall. The work-related equipment 90 may include pavement 90c. The pavement 90c may be covered with soil and sand (e.g., the work object W), for example. The material of the pavement 90c may include concrete, asphalt, or stone. The work-related equipment 90 may include a buried object. This buried object is an object buried with soil and sand (e.g., the work object W), for example. The pavement 90c may be included in the buried object. The work-related equipment 90 may include a ceiling. The work-related equipment 90 may include an obstacle that obstructs the work of the work machine 10.

[0066] 1, the vehicle 90a is, for example, a machine (transport vehicle) that transports a work object W stored in a container 91 (specifically, a loading platform). For example, the vehicle 90a is a dump truck or the like.

[0067] The container 91 contains the work object W. The container 91 is, for example, a box-shaped container without a lid. The container 91 is, for example, the bed of a vehicle 90a. The container 91 does not have to be a bed, and may be placed on the ground at the work site, or may be provided in a hole dug in the ground at the work site (such as a sand pit). For example, the container 91 includes a container bottom 91a and a container enclosure 91b.

[0068] The container bottom 91a is the bottom portion (lower surface) of the container 91. The container bottom 91a is, for example, plate-shaped (the same applies to the container rear portion 91b1, the container front portion 91b3, and the container side portion 91b2). The container surrounding portion 91b is provided so as to extend upward from the outer periphery of the container bottom 91a. The container surrounding portion 91b is arranged so as to surround the space directly above the container bottom 91a. The container surrounding portion 91b comprises a container rear portion 91b1, a container front portion 91b3, and a container side portion 91b2. The container side portions 91b2 are arranged so as to face each other (see FIG. 5). The container rear portion 91b1 and the container front portion 91b3 are arranged so as to face each other. In the example shown in Fig. 1, the container front portion 91b3 protrudes above the container rear portion 91b1 and the container side portion 91b2.

[0069] 1, the longitudinal direction of the container 91 is the fore-and-aft direction X of the work machine 10, but the longitudinal direction of the container 91 may be inclined relative to the fore-and-aft direction X of the work machine 10, or may be the lateral direction Y of the work machine 10. Also, the container 91 does not have to have a longitudinal direction, and may be, for example, square or circular when viewed from above.

[0070] The work-related facility 90 may be equipped with a facility position detection device 95. The facility position detection device 95 detects the position of the work-related facility 90. The facility position detection device 95 is similar to the location detection device 33b2, and may use GNSS, for example. Information from the facility position detection device 95 is input to the controller 1c via a network N (see FIG. 2).

[0071] The work machine 10 and work-related equipment 90 will be described below mainly with reference to Figure 1. The controller 1c will be described with reference to Figure 2.

[0072] (Calculation of relative position) The controller 1c (specifically, the relative position acquisition device 75a) acquires the position (position information) of the work-related facility 90 shown in FIG. 1 and the position (position information) of the specific range R. That is, the controller 1c acquires the relative position of the specific range R with respect to the work-related facility 90. "Acquisition" may mean reading pre-stored information, calculating (including estimating), or deciding on one of a plurality of options.

[0073] (Location information of work-related equipment 90) The controller 1c acquires position information of the work-related equipment 90. The position information of the work-related equipment 90 is, for example, three-dimensional position information. For example, the position information of the work-related equipment 90 is information indicating the range in three-dimensional space in which the work-related equipment 90 is located. The position information of the work-related equipment 90 includes coordinate information. The coordinate axes of the coordinate information may be coordinate axes based on the work site (for example, global coordinates), coordinate axes based on a specific part of the work machine 10 (for example, a specific part of the upper rotating body 13) (for example, machine coordinates), or other coordinate axes. The position information of the work-related equipment 90 may include shape information. If it is possible to calculate the relative distance L, which will be described later, the position information of the work-related equipment 90 does not need to include shape information.

[0074] (Method for acquiring location information of work-related equipment 90) The controller 1c may acquire the position information of the work-related facility 90 by various methods. The controller 1c may acquire the position information of the work-related facility 90 by one method or by a combination of multiple methods. For example, the controller 1c may acquire the position information (position information) of the work-related facility 90 by one method or a combination of multiple methods from among the following [Acquisition Method A1] to [Acquisition Method A5].

[0075] [Acquisition Method A1] The controller 1c may acquire the position information (one or both of coordinates and shape) of the work-related facility 90 stored in advance (before acquiring the position information) in the storage device 71 (see FIG. 2).

[0076] For example, if the coordinates of the work-related equipment 90 are predetermined, the controller 1c may acquire the coordinates of the work-related equipment 90 that are pre-stored in the storage device 71 (see FIG. 2). For example, as shown in FIG. 7, if the work-related equipment 90 is a retaining wall 90b or a pavement 90c, and the work-related equipment 90 does not move (the coordinates do not change), it is assumed that the coordinates of the work-related equipment 90 are pre-stored. Furthermore, as shown in FIG. 1, even if the work-related equipment 90 moves, there are cases where the stopping position of the work-related equipment 90 is pre-stored, such as when the work-related equipment 90 is a vehicle 90a. In these cases, it is assumed that the coordinates of the work-related equipment 90 are pre-stored. In this case, the controller 1c may acquire the coordinates of the work-related equipment 90 that are pre-stored in the storage device 71 (see FIG. 2).

[0077] For example, if the shape of the work-related facility 90 is predetermined, the controller 1c may acquire the shape of the work-related facility 90 that is pre-stored in the storage device 71 (see FIG. 2). In this case, the storage device 71 pre-stores information about the shape of the work-related facility 90. This information about the shape of the work-related facility 90 is an example of facility information 71b (see FIG. 2), and is, for example, a three-dimensional model.

[0078] [Acquisition method A2] The controller 1c may acquire the shape of the work-related facility 90 from information (type information) about the type of the work-related facility 90. In particular, the controller 1c may acquire the type (type information) of the work-related facility 90, and acquire pre-stored information about the shape of the work-related facility 90 that corresponds to the acquired type of work-related facility 90, thereby acquiring the position information of the work-related facility 90.

[0079] A specific example of this method is as follows. The storage device 71 (see FIG. 2) stores information on the shape of the work-related facility 90 in advance (see [Acquisition method A1] above). If there are multiple types of work-related facility 90, the storage device 71 stores in advance the shape of the work-related facility 90 for each type of work-related facility 90. The controller 1c then acquires the type (type information) of the work-related facility 90 (described later). The controller 1c may then read (acquire) from the storage device 71 the shape of the work-related facility 90 corresponding to the acquired type.

[0080] (How to obtain type information) The controller 1c may acquire type information of the work-related equipment 90 in various ways. For example, the type information of the work-related equipment 90 may be set by the worker operating (manually operating) an operating device (for example, an information presentation operating device 53a1 (see FIG. 2)). Specifically, if the work-related equipment 90 is a vehicle 90a, information indicating the vehicle type may be set by the worker manually operating it.

[0081] For example, the type information of the work-related equipment 90 may be detected (recognized, identified) by a detection device (e.g., the actual equipment sensor group 33b (see FIG. 2)). The type information of the work-related equipment 90 may be acquired by the detection device detecting an identifier. The identifier is used to identify (distinguish, identify) the type of work-related equipment 90. The identifier is, for example, provided on the work-related equipment 90, for example, attached to the work-related equipment 90. The identifier may be a mark 90a1 (see FIG. 3) having a specific form (shape, pattern, color, etc.). The mark 90a1 may be, for example, a two-dimensional barcode or a three-dimensional barcode. If the work-related equipment 90 is a vehicle 90a, the identifier may be, for example, a license plate 90a2 (see FIG. 3) of the vehicle 90a. The identifier may be one that can read electronic information via radio waves, for example, an RFID (Radio Frequency Identification) tag. The type information of the work-related equipment 90 may be acquired by the detection device detecting the work-related equipment 90 itself. In this case, the shape, dimensions, etc. of the work-related facility 90 itself can be said to be an identifier for identifying the type of this work-related facility 90.

[0082] A specific example of when type information of work-related equipment 90 is detected by a detection device is as follows: The detection device detects an identifier (including the work-related equipment 90 itself (the same applies below)). For example, imaging device 33b1 captures an image of the identifier. Also, for example, a detection device capable of reading electronic information reads electronic information stored in the identifier (e.g., an RFID tag). In controller 1c, a relationship between the detected identifier information (identifier form, electronic information, etc.) and the type of work-related equipment 90 is set in advance (before the type is determined). Controller 1c determines (acquires) the type of work-related equipment 90 from this relationship and the detected identifier information. Then, as described above, controller 1c reads (acquires) the shape of the work-related equipment 90 corresponding to the determined type from storage device 71 (see FIG. 2).

[0083] [Acquisition method A3] The controller 1c may acquire position information of the work-related facility 90 from an image including the work-related facility 90. Specifically, the imaging device 33b1 detects one or both of a two-dimensional image and a three-dimensional image of the work-related facility 90. The controller 1c may then calculate (acquire) position information (one or both of coordinates and shape) of the work-related facility 90 from the detected image.

[0084] A specific example of this method is as follows. For example, the imaging device 33b1 detects a two-dimensional image of the work-related facility 90. The controller 1c (e.g., the image processing device 75c (see FIG. 2)) calculates (including estimation) position information (one or both of coordinates and shape) of the work-related facility 90 by image recognition of the detected two-dimensional image. At this time, the controller 1c may perform estimation using AI (artificial intelligence). For example, the controller 1c estimates (calculates) one or both of the coordinates and shape of the work-related facility 90 from information such as the size, position, and angle of the work-related facility 90 in the two-dimensional image. It is assumed that a part of the work-related facility 90 is not shown in this two-dimensional image. For example, it is assumed that a part of the work-related facility 90 is hidden by the work target object W in the two-dimensional image. Furthermore, if the two-dimensional image is an image captured from inside the cab 13c (see FIG. 3), it is assumed that a part of the work-related facility 90 is hidden by the frame of the cab 13c. In such cases where a portion of the work-related equipment 90 is not shown in the two-dimensional image, the controller 1c estimates the coordinates and / or shape of the portion of the work-related equipment 90 that is not shown (the hidden portion) from the portion of the work-related equipment 90 that is shown in the two-dimensional image.

[0085] Furthermore, for example, the imaging device 33b1 detects a three-dimensional image (distance image, e.g., point cloud information) of the work-related facility 90. Then, the controller 1c (e.g., image processing device 75c (see FIG. 2)) identifies which part of the three-dimensional image is the work-related facility 90 by clustering the detected three-dimensional image or the like. At this time, the controller 1c may identify the work-related facility 90 by combining (matching) the two-dimensional image and the three-dimensional image of the work-related facility 90. Then, the controller 1c determines the coordinates of the work-related facility 90 from information (e.g., point cloud information) of the part of the three-dimensional image that has been identified as the work-related facility 90. Furthermore, the controller 1c may estimate the shape of the work-related facility 90 from information of the part of the three-dimensional image that has been identified as the work-related facility 90.

[0086] As described above, the controller 1c may estimate the shape of the work-related facility 90 from a two-dimensional image or a three-dimensional image. In this case, the controller 1c may acquire (estimate, determine) the type of the work-related facility 90 from information (e.g., the shape) of the estimated work-related facility 90, and may acquire shape information of the work-related facility 90 from the acquired type (see [Acquisition method A2] above).

[0087] For example, the controller 1c may calculate the coordinates of the work-related facility 90 from a two-dimensional image of a mark 90a1 (see FIG. 3) (e.g., an AR marker). Specifically, for example, as shown in FIG. 3, the mark 90a1 is provided (e.g., affixed) to the work-related facility 90. The storage device 71 (see FIG. 2) stores the relative positions of the work-related facility 90 and the mark 90a1 in advance (before calculating the coordinates of the work-related facility 90). For example, the storage device 71 stores in advance the position of the mark 90a1 on the work-related facility 90. The imaging device 33b1 (see FIG. 1) captures a two-dimensional image including the mark 90a1. Then, the controller 1c (e.g., an image processing device 75c (see FIG. 2)) recognizes the two-dimensional image and calculates the coordinates of the mark 90a1 from, for example, the size, position, angle, etc. of the mark 90a1 in the two-dimensional image. The controller 1c calculates the coordinates of the portion of the work-related facility 90 where the mark 90a1 is provided. Then, the controller 1c calculates the coordinates of the portion of the work-related facility 90 other than the portion where the mark 90a1 is provided, from shape information of the work-related facility 90 pre-stored in the storage device 71 (see FIG. 2) (see [Acquisition method A1] and [Acquisition method A2]). Note that if the relative position between the work-related facility 90 and the mark 90a1 is constant, the mark 90a1 does not need to be provided on the work-related facility 90.

[0088] [Acquisition method A4] The controller 1c may acquire (detect) the coordinates of the work-related facility 90 using an equipment position detection device 95 provided in the work-related facility 90 shown in Fig. 1. For example, the equipment position detection device 95 detects the coordinates (e.g., global coordinates) of a specific part of the work-related facility 90 (e.g., the position of a GNSS antenna). The controller 1c may use the coordinates detected by the equipment position detection device 95, or coordinates obtained by converting these coordinates, as the coordinates of the work-related facility 90.

[0089] In the above-described [Acquisition method A2], [Acquisition method A3], and [Acquisition method A4], detection is performed by a detection device to acquire the position information of the work-related equipment 90. Specifically, detection of an identifier, image capture of the work-related equipment 90, and detection by the equipment position detection device 95 are performed. This detection by the detection device may be performed continuously (in real time). For example, detection by the detection device may be performed continuously while the work machine 10 is moving. When detection by the detection device is performed continuously, even if the relative position of the equipment with respect to the detection device changes, the detection device can continue to detect information about the work-related equipment 90, and the controller 1c can continue to acquire the position information of the work-related equipment 90. Note that detection by the detection device does not have to be performed continuously, and may be performed only once (at the beginning), for example. For example, when the work machine 10 is not traveling and the work-related equipment 90 is not moving, detection by the detection device does not need to be performed continuously.

[0090] [Acquisition Method A5] The position information of the work-related facility 90 (coordinates and / or shape) may be set manually by the worker. The position information of the work-related facility 90 may be set only by manual setting. Furthermore, the position information of the work-related facility 90 acquired by the controller 1c (see, for example, the above [Acquisition Methods A1] to [Acquisition Methods A4]) may be adjusted (corrected, amended) by manual setting. When the position information of the work-related facility 90 is set manually by the worker, the image output device (for example, the remote image output device 57a) may output a manual setting screen G10, as shown in FIG. 3.

[0091] The manual setting screen G10 is a screen for the worker to manually set the position information of the work-related equipment 90. The manual setting screen G10 includes, for example, a main display section G11, a sub-display section G13, an equipment image G15a, a work machine image G15b, a position information image G15c, and an operation display section G17.

[0092] The main display unit G11, the sub-display unit G13, and the operation display unit G17 may be displayed on a common monitor or on different monitors. Some or all of the main display unit G11, the sub-display unit G13, and the operation display unit G17 may be displayed on a common monitor (for example, as a window). Specifically, for example, the main display unit G11 and the sub-display unit G13 may be displayed on a monitor in front of the operator, and the operation display unit G17 may be displayed on a monitor closer to the operator.

[0093] The main display unit G11 is a screen (display unit) that serves as the main screen for the manual setting screen G10. In the example shown in Fig. 3, the main display unit G11 displays an image viewed from the operator's cab 13c toward the rear side X1 shown in Fig. 1.

[0094] As shown in Fig. 3, the sub-display unit G13 is a screen (display unit) that displays an image from a viewpoint different from that of the main display unit G11 (details will be described later). The "viewpoint" includes the position, direction, and angle of view of the image (camera).

[0095] The equipment image G15a is an image of the work-related equipment 90. The equipment image G15a may be an image (actual image) captured by the imaging device 33b1 (see FIG. 1) or an image (computer graphics) generated by the controller 1c. The equipment image G15a may be an image of the entire work-related equipment 90, an image of part of the work-related equipment 90, or may not be displayed at all. In the example shown in FIG. 3, the equipment image G15a is displayed on the main display unit G11 and is not displayed on the sub-display unit G13. The main display unit G11 may display an image of the work target W (work target image G15w).

[0096] The work machine image G15b is an image of the work machine 10 (of the current work machine 10 (at the time of manual setting)). The work machine image G15b may be an image (actual image) captured by the imaging device 33b1 (see FIG. 1), or may be an image (computer graphics) generated by the controller 1c. The work machine image G15b may be an image of the entire work machine 10, or an image of part of the work machine 10, or it may not be displayed at all. The main display unit G11 displays an image of the far side X1 captured from inside the operator's cab 13c shown in FIG. 1. In the example shown in FIG. 3, the main display unit G11 displays an image of part of the operator's cab 13c (an example of the work machine image G15b). The sub-display unit G13 displays an image (an example of the work machine image G15b) that schematically shows the attachment 15. The sub-display unit G13 displays an image including the specific location 16 (specific location image G15b1). In the example shown in Fig. 3, the specific location image G15b1 is also an image showing the specific location existence range Ra. Note that an image showing the specific location operation range Rb (see Fig. 1) may be displayed on the manual setting screen G10 (sub-display unit G13, etc.). In this case, the image showing the specific location operation range Rb is an image showing all or part of the operation range of the specific location 16 in the current state (position, posture, etc.) of the work machine 10 (at the time of manual setting), for example.

[0097] The position information image G15c is an image that represents the position information of the work-related equipment 90. The position information image G15c may be displayed, for example, on the main display unit G11, on the sub-display unit G13, or on both the main display unit G11 and the sub-display unit G13. The display mode of the position information image G15c (for example, the position on the screen, the shape, etc.) corresponds to the position information of the work-related equipment 90. For example, the display mode of the position information image G15c changes in response to the worker's manual operation (for example, operation of the operation display unit G17) (described later). This allows the worker to understand how the position information of the work-related equipment 90 is changing in response to the manual operation.

[0098] The position information image G15c in the example shown in FIG. 3 is an image obtained when position information in the forward / backward direction X (see FIG. 1) (hereinafter, for directions, see FIG. 1) of the work-related equipment 90 is manually set. Specifically, in the example shown in FIG. 3, the position information image G15c is an image obtained when position information of the container front 91b3 (the shrine gate of the loading platform) and the container rear 91b1 (the tailgate at the rear of the loading platform) is manually set. In the example shown in FIG. 3, the position information image G15c in the main display unit G11 is displayed superimposed on the equipment image G15a. For example, the position information image G15c located on the farther side X1 than the position of the work-related equipment 90 (e.g., the container rear 91b1) may be displayed in a manner such that it is hidden by the work-related equipment 90 (e.g., the container rear 91b1). For example, the position information image G15c located on the nearer side X2 than the position of the work-related equipment 90 (e.g., the container front 91b3) may be displayed in a manner such that it covers the work-related equipment 90 (e.g., the container front 91b3). In this example, the position information image G15c is displayed as a surface (for example, a rectangle) extending in the vertical direction Z and the horizontal direction Y.

[0099] In the example shown in Fig. 3, the position information image G15c of the sub-display unit G13 is displayed as an image (for example, a linear figure) of a plane extending in the vertical direction Z and the horizontal direction Y viewed from the horizontal direction Y. Note that Fig. 3 shows the position information image G15c when the position information of the work-related equipment 90 in the forward / backward direction X is manually set. On the other hand, the position information image G15c when the position information of the work-related equipment 90 in the horizontal direction Y or the vertical direction Z is manually set may be displayed on the manual setting screen G10.

[0100] This position information image G15c moves in the manual setting screen G10 in accordance with the manual setting. At this time, it is preferable that the manual setting screen G10 is displayed so that the worker can easily grasp the movement of the position information image G15c (changes in the parameters of the work-related equipment 90). Specifically, it is preferable that the manual setting screen G10 (the sub-display unit G13 in the example shown in FIG. 3) displays an image of a viewpoint seen from a direction perpendicular to the direction in which the position information image G15c moves (for example, the horizontal direction Y). Note that the image of a viewpoint seen from a direction perpendicular to the direction in which the position information image G15c moves (for example, the horizontal direction Y) may be displayed on the main display unit G11.

[0101] The operation display unit G17 is a part that is operated by the worker to manually set the position information of the work-related equipment 90. The operation display unit G17 is a GUI (Graphical User Interface). The operation display unit G17 may be included in the information presentation operation device 53a1 (see FIG. 2). The position information of the work-related equipment 90 may also be manually set by operating a physical operation device. Here, a case will be described in which the position information of the work-related equipment 90 is manually set using the operation display unit G17. The operation display unit G17 includes a setting target selection unit G17a and a setting value change unit G17b.

[0102] The setting target selection section G17a is a section for selecting a target to be manually set. The setting target selection section G17a is operated (e.g., selected) to select a target to be manually set. Specifically, the setting target selection section G17a for "rear position setting" shown in FIG. 3 is a section for the operator to select to manually set the position in the front-to-rear direction X of the rear side X1 portion of the container 91 (the container front portion 91b3, for example, the torii gate). The setting target selection section G17a for "front side position setting" shown in FIG. 3 is a section for the operator to select to manually set the position in the front-to-rear direction X of the front side X2 portion of the container 91 (the container rear portion 91b1, for example, the rear gate).

[0103] The set value change unit G17b is a unit for changing set values ​​(parameters) of the position information of the work-related equipment 90. Operating the set value change unit G17b changes the set values ​​(parameters) of the manually set target. The set value change unit G17b may be able to change the set values ​​of the coordinates of the work-related equipment 90, or may be able to change the set values ​​of the shape of the work-related equipment 90. The set value change unit G17b may be able to change the set value of the position of the work-related equipment 90 in a certain direction (for example, the front-to-back direction X, the lateral direction Y, or the up-to-down direction Z). The set value change unit G17b may be able to change the set value of the dimension of the work-related equipment 90 in a certain direction (width, height, etc.). The set value change unit G17b may be able to change the set value of the rotation angle (tilt angle, posture, etc.) of the work-related equipment 90.

[0104] The "+" and "-" shown in FIG. 3 are set value change sections G17b for increasing or decreasing the set value, and specifically, for example, are set value change sections G17b for changing the set value in the forward / backward direction X. Each time this set value change section G17b ("+" or "-") is selected (pressed), the coordinates of the work-related facility 90 move a predetermined distance. Specifically, for example, each time the set value change section G17b is selected, the coordinates of the work-related facility 90 move by 250 mm, and when the set value change section G17b is selected four times, the coordinates of the work-related facility 90 move a total of 1 meter. The "manual height adjustment" shown in FIG. 3 is a set value change section G17b for changing (increasing or decreasing) the set value of the height (e.g., the position of the upper end) of the position information of the work-related facility 90.

[0105] The setting value of the position information of the work-related facility 90 may be set manually in various ways. For example, the setting value of the position information of the work-related facility 90 may be changed by dragging and dropping or tapping and dropping a part of the manual setting screen G10 (for example, the position information image G15c, etc.). In this case, the worker can intuitively change the setting value of the position information of the work-related facility 90. The position information of the work-related facility 90 may also be input numerically. For example, if the position information of the work-related facility 90 (coordinates or shape, or both) is predetermined, the position information can be accurately set by inputting numerical values.

[0106] (Location information for specific range R) The controller 1c acquires position information of the specific range R shown in FIG. 1. The position information of the specific range R is three-dimensional position information of the specific range R. The controller 1c may acquire position information of the specific location existence range Ra, may acquire position information of the specific location operation range Rb, or may acquire both of these position information. For example, the position information of the specific location existence range Ra is information indicating the range in three-dimensional space in which the specific location 16 is located. The position information of the specific location operation range Rb is information indicating the range in three-dimensional space in which all or part of the trajectory (movable range, range of motion) of the specific location 16 is located when the specific location 16 is assumed to move (see the description of the specific location operation range Rb above for details). The position information of the specific range R includes coordinate information. The coordinate axes (e.g., global coordinates, machine coordinates, etc.) of the position information of the specific range R are the same as the coordinate axes of the position information of the work-related equipment 90.

[0107] (How to obtain location information for specific range R) The controller 1c may acquire position information of the specific location 16 to acquire position information of the specific location existence range Ra. The position information of the specific location 16 includes coordinate information. The position information of the specific location 16 may also include shape information. If it is possible to calculate the relative distance L, which will be described later, the position information of the specific location 16 does not need to include shape information.

[0108] The controller 1c may acquire position information of the specific location 16, and acquire position information of the specific location operating range Rb based on the acquired information and machine information 71a (see FIG. 2) (e.g., information on the operating range of the specific location 16). The controller 1c may acquire position information (e.g., posture information) of a portion of the work machine 10 that is different from the specific location 16 (e.g., the above-mentioned "reference location of the work machine 10"), and acquire position information of the specific location operating range Rb based on the acquired information and machine information 71a. Specifically, for example, as in the example shown in FIG. 1, the specific location 16 may be the tip attachment 15c, and the above-mentioned "reference location of the work machine 10" may be the machine main body 10a, boom 15a, and arm 15b. In this case, the controller 1c acquires position information (e.g., posture information) of the machine main body 10a, boom 15a, and arm 15b. For example, from this position information, the controller 1c acquires position information of the tip of the arm 15b. At this time, the controller 1c does not need to acquire position information (for example, posture information) of the end attachment 15c. Furthermore, the controller 1c acquires information (preset machine information 71a (see FIG. 2)) of the operating range of the end attachment 15c when the end attachment 15c rotates relative to the end of the arm 15b. Then, the controller 1c may acquire (calculate) position information of the specific location operating range Rb of the end attachment 15c.

[0109] Note that if it is possible to acquire position information of the specific location operating range Rb, the controller 1c does not need to acquire position information (e.g., posture information) of the work machine 10. For example, it is possible to imagine a case where the above-mentioned "reference location of the work machine 10" does not move (the coordinates do not change) on the coordinate axes used to calculate the relative distance L. Specifically, for example, if the coordinate axes used to calculate the relative distance L are machine coordinates based on a specific part of the upper rotating body 13, and the "reference location of the work machine 10" is the upper rotating body 13, then the above-mentioned "reference location of the work machine 10" does not move. In such a case, the controller 1c may acquire position information of the specific location operating range Rb from information on the operating range of the predetermined specific location 16 (machine information 71a (see FIG. 2)), without acquiring position information (e.g., posture information) of the work machine 10. If it is not necessary to acquire posture information of the work machine 10, the posture detection device 33b3 is not necessary.

[0110] The following mainly describes a method for acquiring the position information of the specific location 16. The following method for acquiring the position information of the specific location 16 may also be applied to a method for acquiring the position information of a part of the work machine 10 that is different from the specific location 16.

[0111] The controller 1c may acquire the position information of the specific location 16 by various methods. The controller 1c may acquire the position information of the specific location 16 by one method or by a combination of multiple methods. For example, the controller 1c may acquire the position information of the specific location 16 by one method or a combination of multiple methods from among the following [Acquisition Method B1] to [Acquisition Method B4].

[0112] [Acquisition Method B1] The controller 1c may acquire the position information of the specific location 16 based on the attitude of the work machine 10 detected by the attitude detection device 33b3 (see FIG. 2).

[0113] [Acquisition Method B1a] For example, the attitude detection device 33b3 (see FIG. 2) may detect the angle of the attachment 15. For example, the attitude detection device 33b3 detects the rotation angles of the boom 15a, the arm 15b, and the tip attachment 15c. Furthermore, the position detection device 33b2 (see FIG. 2) detects the position (e.g., global coordinates) of a reference part of the work machine 10 (e.g., a specific part of the upper rotating body 13) relative to the work site. The storage device 71 stores the above-mentioned machine information 71a in advance (before acquiring the position information of the specific range R). Here, for example, it is assumed that the specific location 16 is the tip attachment 15c. In this case, the controller 1c calculates the position (global coordinates) of the tip attachment 15c relative to the work site from the position detected by the position detection device 33b2, the attitude (specifically, the angle) detected by the attitude detection device 33b3, and the machine information 71a (see FIG. 2). As a result, the position information (coordinates) of the specific location 16 is calculated. It should be noted that, in cases where the position information of the specific location 16 is expressed in machine coordinates, the position detection device 33b2 does not need to detect the coordinates of the part of the work machine 10 that serves as the reference.

[0114] [Acquisition Method B1b] As described above, the imaging device 33b1 (see FIG. 2), which is the attitude detection device 33b3 (see FIG. 2), may detect one or both of a two-dimensional image and a three-dimensional image of the specific location 16. In this case, the controller 1c (e.g., the image processing device 75c (see FIG. 2)) recognizes where the specific location 16 is located in the detected image. The controller 1c then calculates position information (coordinates and / or shape) of the specific location 16 from the position of the specific location 16 in the image (similar to [Acquisition Method A3] described above).

[0115] [Acquisition Method B2] The controller 1c may acquire shape information of the specific location 16 stored in advance in the storage device 71 (see FIG. 2) (similar to the above [Acquisition Method A1]).

[0116] [Acquisition Method B3] The controller 1c may acquire the shape of the specific location 16 from information (type information) about the type of the specific location 16 (similar to [Acquisition Method A2] above). In particular, the controller 1c may acquire the type (type information) of the specific location 16, and acquire information about the shape of the specific location 16 that corresponds to the acquired type of the specific location 16 and is stored in advance in the storage device 71 (see FIG. 2).

[0117] For example, the specific location 16 may be the tip attachment 15c. The tip attachment 15c may be changed depending on the type of work performed by the work machine 10. In this case, the type of the specific location 16 changes. Therefore, the controller 1c acquires type information of the specific location 16. The type information may be detected as in the above-mentioned [Acquisition method A2], or may be manually set by the worker using an operation device (for example, the information presentation operation device 53a1 (see FIG. 2)). The controller 1c then acquires (reads) from the storage device 71 the shape of the specific location 16 of the type corresponding to the acquired type information.

[0118] [Acquisition Method B4] The controller 1c may acquire (detect) the coordinates of the specific location 16 using a position detection device 33b2 (see FIG. 2) provided at the specific location 16 (similar to the above [Acquisition Method A4]).

[0119] (Calculation of relative distance L) The controller 1c (specifically, the relative distance calculation device 75b (see FIG. 2)) calculates the relative distance L of the specific range R of the work machine 10 with respect to the work-related facility 90, as shown in FIG. 4. The controller 1c calculates the relative distance L based on the position information of the work-related facility 90 and the position information of the specific range R acquired by the controller 1c (specifically, the relative position acquisition device 75a (see FIG. 2)).

[0120] The controller 1c may calculate the relative distance L of the specific location existence range Ra with respect to the work-related equipment 90, or may calculate the relative distance L of the specific location operation range Rb with respect to the work-related equipment 90, or may calculate both of these relative distances L. The following mainly describes a case where the controller 1c calculates the relative distance L of the specific location existence range Ra (specific location 16) with respect to the work-related equipment 90. The specific location 16 (i.e., specific location existence range Ra) in the following description may be read as specific location operation range Rb.

[0121] The relative distance L calculated by the controller 1c may be the shortest distance between the work-related facility 90 and the specific location 16 in a specific direction.

[0122] Specifically, the relative distance L calculated by the controller 1c may include the relative distance LX in the front-rear direction. The relative distance LX in the front-rear direction is the shortest distance in the front-rear direction X between the work-related equipment 90 and the specific location 16. The relative distance LX in the front-rear direction calculated by the controller 1c may also include the shortest distance in the front-rear direction X between the specific location 16 and the work-related equipment 90 (specifically, the front part 91b3 of the container) located on the rear side X1 of the specific location 16. The relative distance LX in the front-rear direction calculated by the controller 1c may also include the shortest distance in the front-rear direction X between the specific location 16 and the work-related equipment 90 (specifically, the rear part 91b1 of the container) located on the front side X2 of the specific location 16. When the work-related equipment 90 and the specific location 16 do not face each other in the front-rear direction X but are deviated by a predetermined value or more in the up-down direction Z or the lateral direction Y (largely deviated), there is a low possibility that the specific location 16 will come into contact with the work-related equipment 90 even if the specific location 16 moves in the front-rear direction X. In this case, the controller 1c may not need to calculate the relative distance LX in the front-rear direction, and may set the relative distance LX to infinity, for example.

[0123] As shown in FIG. 5, the relative distance L calculated by the controller 1c may include the lateral relative distance LY. The lateral relative distance LY is the shortest distance in the lateral direction Y between the work-related equipment 90 and the specific location 16. The lateral relative distance LY calculated by the controller 1c may also include the shortest distance in the lateral direction Y between the specific location 16 and the work-related equipment 90 (specifically, the right container side 91b2) to the right of the specific location 16 (the right side when looking at the rear side X1 from the work machine 10). The lateral relative distance LY calculated by the controller 1c may also include the shortest distance in the lateral direction Y between the specific location 16 and the work-related equipment 90 (specifically, the left container side 91b2) to the left of the specific location 16. If the work-related equipment 90 and the specific location 16 do not face each other in the lateral direction Y but are offset by a predetermined value or more in the up-down direction Z or the front-rear direction X, the controller 1c may, for example, not calculate the lateral relative distance LY, and may set the lateral relative distance LY to infinity.

[0124] As shown in FIG. 4, the relative distance L calculated by the controller 1c may include the vertical relative distance LZ. The vertical relative distance LZ may include the shortest distance in the vertical direction Z between the work-related facility 90 and the specific location 16. The vertical relative distance LZ calculated by the controller 1c may include the shortest distance in the vertical direction Z between the specific location 16 and the work-related facility 90 (specifically, the container bottom 91a) below the specific location 16 Z2. The vertical relative distance LZ calculated by the controller 1c may include the shortest distance in the vertical direction Z between the specific location 16 and the work-related facility 90 (e.g., the ceiling) above the specific location 16 Z1. If the work-related facility 90 and the specific location 16 are not opposite each other in the vertical direction Z but are offset by a predetermined value or more in the front-rear direction X or the lateral direction Y, the controller 1c may not calculate the vertical relative distance LZ, and may set the vertical relative distance LZ to infinity, for example.

[0125] The relative distance L calculated by the controller 1c may be only the relative distance L in some of the three directions, i.e., the forward / backward relative distance LX, the lateral relative distance LY, and the up / down relative distance LZ. The relative distance L calculated by the controller 1c may include the shortest distance between the work-related facility 90 and the specific location 16 in a specific direction other than the three directions of the forward / backward direction X, the lateral direction Y, and the up / down direction Z. Furthermore, the relative distance L calculated by the controller 1c may include the shortest distance in any direction between the work-related facility 90 and the specific location 16.

[0126] (information presentation) The information presentation device 57b presents information according to the relative distance L. The information presentation device 57b may present information according to the relative distance L of the specific location presence range Ra with respect to the work-related equipment 90, or may present information according to the relative distance L of the specific location operation range Rb with respect to the work-related equipment 90, or may present both of these pieces of information. The information presentation device 57b presents information according to the relative distance L in a direction calculated by the controller 1c. The information presentation device 57b does not need to present information according to the relative distance L in a direction not calculated by the controller 1c. The information presentation device 57b presents proximity information, which is information indicating that the relative distance L has become equal to or less than a threshold. The information presentation device 57b may present information other than proximity information.

[0127] The information presented by the information presentation device 57b may include display information (visual information, image information). In this case, the information presentation device 57b may be equipped with a monitor that displays images. The information presented by the information presentation device 57b may include audio information. In this case, the information presentation device 57b may be equipped with a speaker. The information presented by the information presentation device 57b may include vibration information. In this case, the information presentation device 57b may be equipped with a vibration generator.

[0128] The information presented by the information presentation device 57b may include a relative position display G21 and a relative distance display G23.

[0129] The relative position display G21 is a display (image) that allows a worker to intuitively grasp the relative position of the specific range R (in this example, the specific location existence range Ra and the specific location 16) with respect to the work-related equipment 90. Specifically, the relative position display G21 includes an image that shows the relative position of the specific location 16 with respect to the work-related equipment 90. The relative position display G21 is an image of the work-related equipment 90 and the specific location 16 viewed from a specific direction. This "specific direction" may be, for example, the horizontal direction Y, the vertical direction Z, or the front-to-back direction X. The information presentation device 57b may display only one relative position display G21 viewed from one direction, or may display multiple relative position displays G21 viewed from multiple directions. The relative position display G21 may include a display of the specific location 16 (specific location existence range Ra), a display of the specific location operation range Rb (see FIG. 1), or both of these displays.

[0130] The relative position display G21 may include an image of the work-related equipment 90 and the specific location 16 viewed from the lateral direction Y. In this case, the worker can grasp the relative positions of the work-related equipment 90 and the specific location 16 in the front-to-back direction X (depth direction) and the up-to-down direction Z. Furthermore, as shown in FIG. 5 , the relative position display G21 may include an image of the work-related equipment 90 and the specific location 16 viewed from the up-to-down direction Z (for example, from the upper side Z1 or the lower side Z2). In this case, the worker can grasp the relative positions of the work-related equipment 90 and the specific location 16 in the front-to-back direction X (depth direction) and the lateral direction Y. Furthermore, the relative position display G21 may include an image of the work-related equipment 90 and the specific location 16 viewed from the front-to-back direction X (for example, from the front side X2 or the rear side X1). In this case, the worker can grasp the relative positions of the work-related equipment 90 and the specific location 16 in the lateral direction Y and the up-to-down direction Z.

[0131] As shown in FIG. 4, the relative position display G21 includes an image of the work-related equipment 90 (the information presentation device 57b displays the image of the work-related equipment 90). For example, as shown in FIG. 3, when viewed from a certain direction (for example, when viewed from inside the cab 13c), part or all of the work-related equipment 90 may be hidden by an object that conceals the work-related equipment 90. Even in such cases, as shown in FIG. 4, the relative position display G21 includes an image of the work-related equipment 90. Therefore, even when part or all of the work-related equipment 90 is hidden, the worker can grasp the relative position of the work-related equipment 90 and the specific location 16. The "object that conceals the work-related equipment 90" may be, for example, the work target object W, as shown in FIG. 3, or the work machine 10 (for example, the frame of the cab 13c).

[0132] Specifically, as shown in FIG. 4, the information presentation device 57b may display the work-related equipment 90 without displaying objects that obscure the work-related equipment 90 (such as the work target object W or the work machine 10). As shown in FIG. 7, the information presentation device 57b may display the objects that obscure the work-related equipment 90, but may also display the work-related equipment 90 so as to allow the worker to grasp the position of the work-related equipment 90. Specifically, for example, the information presentation device 57b may display a cross section of the object that obscures the work-related equipment 90 and also display the work-related equipment 90. The information presentation device 57b may display the work-related equipment 90 while also displaying the object that obscures the work-related equipment 90 in a see-through manner. The information presentation device 57b may display a cross section of the work-related equipment 90, or may display the external appearance of the work-related equipment 90.

[0133] The relative position display G21 may include an image of a portion of the work machine 10 other than the specific location 16. For example, if only the tip attachment 15c of the attachment 15 shown in Figure 1 is the specific location 16, the relative position display G21 (see Figure 4) may include an image of the arm 15b or an image of the boom 15a.

[0134] 4 is not detected, the information presentation device 57b may not display the work-related facility 90. In this case, the information presentation device 57b may present (notify) that the work-related facility 90 is not detected. Specifically, if the work-related facility 90 is a dump truck (an example of the vehicle 90a), the information presentation device 57b may display the words "Dump truck not detected."

[0135] The relative distance display G23 is an image that notifies the worker of the relative distance L. The relative distance display G23 includes an image corresponding to the relative distance L. This "image corresponding to the relative distance L" is a different image from the relative position display G21, i.e., the image showing the relative positions of the work-related equipment 90 and the specific location 16. The relative position display G21 and the relative distance display G23 are displayed on a common screen. The relative position display G21 and the relative distance display G23 may be displayed side by side on a common screen, or may be displayed superimposed on each other. Note that the relative position display G21 and the relative distance display G23 may be displayed on different screens.

[0136] This relative distance display G23 may include a display according to the relative distance L of the specific location presence range Ra (i.e., the specific location 16) with respect to the work-related equipment 90 (referred to as a relative distance display G23 of the specific location presence range Ra). The relative distance display G23 may also include a display according to the relative distance L of the specific location operation range Rb (see FIG. 1) with respect to the work-related equipment 90 (referred to as a relative distance display G23 of the specific location operation range Rb). The relative distance display G23 may include both the relative distance display G23 of the specific location presence range Ra and the relative distance display G23 of the specific location operation range Rb. In this case, the display mode (e.g., shape, color, etc.) of the relative distance display G23 of the specific location presence range Ra may differ from the display mode of the relative distance display G23 of the specific location operation range Rb.

[0137] This relative distance display G23 includes a display (guidance display) indicating that the relative distance L has become equal to or less than a threshold value. The relative distance display G23 is an example of the above-mentioned "proximity information." The form of the relative distance display G23 may change depending on the relative distance L. For example, the relative distance display G23 may include a graphic that changes depending on the relative distance L (see FIG. 8), a color that changes depending on the relative distance L, or text that changes depending on the relative distance L. For example, the relative distance display G23 may include a numerical value that indicates the relative distance L. The relative distance display G23 may include wording (such as a warning message) that changes depending on the relative distance L.

[0138] The information presented by the information presentation device 57b may include sound (see FIG. 8). In this case, the information presentation device 57b may change the sound to be output (presented) depending on the relative distance L. The information presentation device 57b may present a sound (also referred to as the sound of the specific location presence range Ra) according to the relative distance L of the specific location presence range Ra from the work-related equipment 90 (the same applies to vibration). The information presentation device 57b may present a sound (also referred to as the sound of the specific location presence range Rb) according to the relative distance L of the specific location operation range Rb (see FIG. 1) from the work-related equipment 90 (the same applies to vibration). The information presentation device 57b may present both the sound of the specific location presence range Ra and the sound of the specific location operation range Rb (the same applies to vibration). In this case, the information presentation device 57b may differentiate the manner of the sound (presence or absence of sound, sound pattern, etc.) in the specific location presence range Ra from the manner of the sound in the specific location operation range Rb (the same applies to vibration). For example, the information display device 57b may change the manner of sound in accordance with the relative distance L (a specific example will be described later). The information displayed by the information display device 57b may include vibration. In this case, the information display device 57b may change the manner of vibration to be output (displayed) in accordance with the relative distance L (see the example of sound for a specific example).

[0139]

[0063] A further specific example of presentation of proximity information by the information presentation device 57b will be described. The information presentation device 57b may change the graphic of the relative distance display G23 according to the relative distance L. In the example shown in FIG. 8, the relative distance display G23 of the information presentation device 57b (see FIG. 4) includes a plurality of bars (rod-shaped graphic). The information presentation device 57b displays more bars as the relative distance L increases, and displays fewer bars as the relative distance L decreases. The information presentation device 57b may change the color of the relative distance display G23 according to the relative distance L. For example, the information presentation device 57b may display a cooler color (such as green) for the relative distance display G23 as the relative distance L increases, and a warmer color (such as yellow or red) for the relative distance L decreases. For example, the information presentation device 57b may change the sound to be output according to the relative distance L. For example, the information presentation device 57b outputs a warning sound. When the relative distance L is long, the information presentation device 57b outputs an intermittent sound with long intervals (silence portions) such as beep, beep, beep. When the relative distance L is short, the information presentation device 57b outputs an intermittent sound with short intervals (fast) such as beep, beep. When the relative distance L is even shorter, the information presentation device 57b outputs a continuous sound with no intervals such as beep.

[0140] The controller 1c is set with a plurality of thresholds for the relative distance L related to the proximity information. The information presentation device 57b changes the manner of the proximity information according to the relative distance L each time the relative distance L becomes equal to or less than the threshold (or becomes less than the threshold). Note that the information presentation device 57b may not present the proximity information when the relative distance L is greater than a certain threshold. Specifically, when the relative distance L is greater than a certain threshold, the information presentation device 57b may not display the relative distance display G23, may not output a sound, or may not output a vibration.

[0141] The threshold value of the relative distance L may differ depending on the direction of the relative distance L. For example, the threshold value of the relative distance LX in the front-rear direction may differ from the threshold value of the relative distance LZ in the up-down direction. Specifically, for example, the threshold value of the relative distance LX in the front-rear direction between the container front part 91b3 (torii gate surface) and the specific location 16 shown in FIG. 4 may differ from the threshold value of the relative distance LZ in the up-down direction between the container bottom part 91a (lower surface) and the specific location 16. Furthermore, the controller 1c may change the threshold value of the relative distance L depending on the conditions (e.g., type) of the work-related equipment 90, or the conditions (e.g., type) of the specific location 16.

[0142] Specific examples of processing by the controller 1c (specifically, the actual machine control device 35, the remote control device 55, and the information processing system 70) and the like will be described with reference to the flowchart shown in FIG. 9. The following description follows the order of processing, unless otherwise specified. Note that the order of processing can be changed in various ways. Also, which of the actual machine control device 35, the remote control device 55, and the information processing system 70 performs a certain process can be changed in various ways. The actual machine control device 35, the remote control device 55, and the information processing system 70 will be described with reference to FIG. 2 below, and each step of the flowchart will be described with reference to FIG. 9 below. As mentioned above, the work-related equipment 90 will be described with reference to FIG. 1 below.

[0143] In step S10, the remote control device 55 shown in FIG. 2 determines whether a designation operation has been performed on the remote operation device 53a. The "designation operation" is, for example, an operation of selecting to perform (e.g., start) a remote operation. If the remote control device 55 determines that a designation operation has been performed (YES in step S10), it performs the process of step S11. If the remote control device 55 determines that a designation operation has not been performed (NO in step S10), it ends the current process on the remote control device 55. If the remote control device 55 ends the current process, it may start the next process (may return to "START") (the same applies when the real machine control device 35 and the information processing system 70 end the current process).

[0144] In step S11, the remote control device 55 transmits an environment confirmation request to the actual machine control device 35. The above-mentioned "environment confirmation request" is a command to the actual machine control device 35 to transmit the situation around the work machine 10 (work environment information) to the remote control device 55. The "environment confirmation request" is a command to cause the actual machine control device 35 to perform the processing of step S21.

[0145] In step S20, the actual device control device 35 determines whether or not an environment confirmation request has been received. If the actual device control device 35 determines that an environment confirmation request has been received (YES in step S20), it performs the process of step S21. If the actual device control device 35 determines that an environment confirmation request has not been received (NO in step S20), it performs the process of step S70.

[0146] In step S21, the actual machine control device 35 acquires work environment information and transmits the acquired work environment information to the remote control device 55. The work environment information is information about the situation around the work machine 10. The work environment information may include images (work environment images) captured by the imaging device 33b1. The work environment information may also include information about sounds around the work machine 10 (work environment sounds). The work environment information may also include other information detected by the actual machine sensor group 33b.

[0147] In step S30, the remote control device 55 determines whether or not the work environment information has been received. If the remote control device 55 determines that the work environment information has been received (YES in step S30), it performs the process of step S31. If the remote control device 55 determines that the work environment information has not been received (NO in step S30), it performs the process of step S60.

[0148] In step S31, the remote control device 55 causes the remote output interface 57 to output work environment information. For example, the remote control device 55 causes the remote image output device 57a to output a work environment image. The remote control device 55 may also cause the remote output interface 57 (e.g., a speaker) to output work environment sound. The remote control device 55 may also cause the remote output interface 57 to output (e.g., display) other information detected by the actual machine sensor group 33b. When the work machine 10 is remotely operated by the remote operation device 53a, the remote control device 55 causes the remote output interface 57 to continuously output work environment information.

[0149] In step S41, the actual machine control device 35 acquires information on the specific range R (see FIG. 1 ) and transmits (outputs) the acquired information to the information processing system 70. The "information on the specific range R" acquired by the actual machine control device 35 may be, for example, the position information of the specific location 16 itself, or may be information for calculating the position information of the specific location 16. The "information on the specific range R" acquired by the actual machine control device 35 may be position information of a part of the work machine 10 that is different from the specific location 16, and may be information for calculating the specific range R. The above "information for calculating the position information of the specific location 16" or "information for calculating the specific range R" may be, for example, posture information of the work machine 10 detected by the posture detection device 33b3.

[0150] In step S42, the actual machine control device 35 determines whether or not the work-related facility 90 has been detected (presence or absence of the work-related facility 90). For example, when the work-related facility 90 is moving (such as in the case of a vehicle 90a), the actual machine control device 35 may detect the presence or absence of the work-related facility 90 by determining whether or not the work-related facility 90 has entered an area within a predetermined range around the work machine 10. Furthermore, for example, the actual machine control device 35 may detect the presence or absence of the work-related facility 90 by determining (recognizing) whether or not the work-related facility 90 is captured within a predetermined area of ​​the image captured by the imaging device 33b1. If the actual machine control device 35 determines that the work-related facility 90 is present, it performs the processing of step S43. If the actual machine control device 35 determines that the work-related facility 90 is not present, it performs the processing of step S70.

[0151] In step S43, the actual machine control device 35 transmits (outputs) information about the work-related facility 90 to the information processing system 70. The "information about the work-related facility 90" transmitted by the actual machine control device 35 may be the position information of the work-related facility 90 itself, or may be information for calculating the position information of the work-related facility 90. The information for calculating the position information of the work-related facility 90 may be, for example, an image showing an identifier (such as a mark 90a1 (see FIG. 3)), or may be electronic information received from an identifier (such as an RFID tag).

[0152] In step S50, the information processing system 70 determines whether or not it has received information about the work-related facility 90. If it determines that it has received information about the work-related facility 90 (YES in step S50), the information processing system 70 performs the process of step S51. If it determines that it has not received information about the work-related facility 90 (NO in step S50), the information processing system 70 ends the current process.

[0153] In step S51, the information processing system 70 (more specifically, the relative position acquisition device 75a) acquires position information of the work-related facility 90 and position information of the specific range R (see FIG. 1). For example, the information processing system 70 acquires (e.g., calculates) the position information of the specific range R from the information of the specific range R received from the actual machine control device 35 (see step S41). For example, the information processing system 70 acquires (e.g., calculates) the position information of the specific location 16 from the information of the specific location 16 received from the actual machine control device 35. For example, the information processing system 70 acquires (e.g., calculates) the position information of the work-related facility 90 from the information of the work-related facility 90 received from the actual machine control device 35 (see step S43).

[0154] In step S52, the information processing system 70 (more specifically, the relative distance calculation device 75b) calculates the relative distance L (see FIG. 1) (details described above) and transmits information on the calculated relative distance L to the remote control device 55. Then, the information processing system 70 ends the current processing in the information processing system 70.

[0155] In step S60, the remote control device 55 determines whether or not information on the relative distance L (see FIG. 1) has been received. If the remote control device 55 determines that information on the relative distance L has been received (YES in step S60), it performs the process of step S61. If the remote control device 55 determines that information on the relative distance L has not been received (NO in step S60), it performs the process of step S62.

[0156] In step S61, the remote control device 55 causes the information presentation device 57b to present information. For example, when the relative distance L (see FIG. 1) is equal to or less than a certain threshold, the remote control device 55 causes the information presentation device 57b to present (output) proximity information (as described above in detail).

[0157] In step S62, the remote control device 55 detects (recognizes) an operation (operation mode) performed on the remote operation device 53a. Then, the remote control device 55 transmits a command (remote operation command) corresponding to the detected operation to the real machine control device 35. Then, the remote control device 55 ends the current processing of the remote control device 55.

[0158] In step S70, the actual device control device 35 determines whether or not a remote operation command has been received. If the actual device control device 35 determines that a remote operation command has been received (YES in step S70), it performs the process of step S71. If the actual device control device 35 determines that a remote operation command has not been received (NO in step S70), it ends the current process in the actual device control device 35.

[0159] In step S71, the actual machine control device 35 controls the movement of the work machine 10 in accordance with the received remote operation command. Specifically, in accordance with the remote operation command, the actual machine control device 35 sends a command to the drive control device 17 (for example, a hydraulic circuit, an electric circuit, etc.) to drive the actuator that moves the work machine 10. Then, the actual machine control device 35 ends the current processing at the actual machine control device 35.

[0160] (Effects of the first invention) The information presentation system 1 shown in Fig. 2 has the following advantages: The information presentation system 1 includes an information presentation device 57b.

[0161] [Configuration 1] As shown in Figure 4, the information presentation device 57b presents information according to the relative distance L of a specific range R, which is the range relating to a specific location 16 of the work machine 10, to work-related equipment 90, which is equipment located near the work position of the work machine 10.

[0162] In the above [Configuration 1], information according to the relative distance L of the specific range R with respect to the work-related facility 90 is presented by the information presentation device 57b. Therefore, the information presentation system 1 can allow the worker to grasp information on the relative distance L of the specific range R of the work machine 10 with respect to the work-related facility 90.

[0163] For example, as shown in Fig. 1, it is conceivable that it is impossible or difficult for a worker to grasp the relative distance L from the outside, such as when part or all of the work-related equipment 90 is hidden by some object (work target W, work machine 10, etc.). Even in such cases, information according to the relative distance L is presented by the information presentation device 57b, so that the worker can grasp the information on the relative distance L. Note that when part or all of the work-related equipment 90 is not hidden by some object, the information presentation device 57b may present information according to the relative distance L.

[0164] (Effects of the second invention) As shown in FIG. 4, the information presentation device 57b displays a relative position display G21 and a relative distance display G23.

[0165] [Configuration 2-1] The relative position display G21 is an image showing the relative position of the specific range R with respect to the work-related facility 90, and includes an image of the work-related facility 90 and the specific range R viewed from a specific direction.

[0166] [Configuration 2-2] The relative distance display G23 includes an image according to the relative distance L, separate from the relative position display G21.

[0167] In the above [Configuration 2-1], the information presentation device 57b displays an image showing the relative position of the specific range R with respect to the work-related equipment 90. Therefore, the image allows the worker to intuitively understand the relative position of the specific range R with respect to the work-related equipment 90. In the above [Configuration 2-2], the information presentation device 57b further displays a relative distance display G23 including an image according to the relative distance L. Therefore, the image allows the worker to intuitively understand the relative distance L.

[0168] (Effect of the third invention) As shown in FIG. 2, the information presentation system 1 includes an attitude detection device 33b3, a relative position acquisition device 75a, and a relative distance calculation device 75b.

[0169] [Configuration 3-1] The attitude detection device 33b3 detects the attitude of the work machine 10. The relative position acquisition device 75a acquires the position in the specific range R (see FIG. 1) calculated based on the attitude detected by the attitude detection device 33b3.

[0170] [Configuration 3-2] The relative position acquisition device 75a acquires the position of the work-related equipment 90. The relative distance calculation device 75b calculates the relative distance L (see FIG. 1) based on the position of the work-related equipment 90 (see FIG. 1) and the position of the specific range R (see FIG. 1) acquired by the relative position acquisition device 75a.

[0171] In the above [Configuration 3-1], the position within the specific range R can be reliably acquired from the attitude of the work machine 10 shown in Fig. 1. As a result, the relative position acquisition device 75a can reliably calculate the relative distance L of the specific range R with respect to the work-related equipment 90 (the above [Configuration 3-2]).

[0172] (Effect of the fourth invention) [Configuration 4] The specific range R is the entire or part of the operating range of the specific location 16 (specific location operating range Rb).

[0173] The above [Configuration 4] allows the worker to grasp information about the relative distance L of all or part of the operating range of the specific location 16 relative to the work-related equipment 90. Therefore, for example, even if the specific location 16 is not actually moved, the worker can grasp information indicating how close the work-related equipment 90 will be to the specific location 16 when the specific location 16 is moved.

[0174] (Effect of the fifth invention) The information presentation system 1 includes a relative position acquisition device 75a (see FIG. 2) and a relative distance calculation device 75b (see FIG. 2).

[0175] [Configuration 5-1] The relative position acquisition device 75a (see Figure 2) acquires the position of the specific range R (specific location operating range Rb) acquired based on information on the operating range of the predetermined specific location 16, and the position of the work-related equipment 90.

[0176] [Configuration 5-2] The relative distance calculation device 75b (see FIG. 2) calculates the relative distance L based on the position of the work-related facility 90 and the position of the specific range R acquired by the relative position acquisition device 75a.

[0177] In the above [Configuration 5-1], the position of the specific range R (more specifically, the specific location operating range Rb) can be acquired from information on the operating range of the preset specific location 16. As a result, the relative position acquisition device 75a (see FIG. 2) can reliably calculate the relative distance L of the specific range R (specific location operating range Rb) to the work-related equipment 90 (the above [Configuration 5-2]).

[0178] (Effect of the sixth aspect of the invention) The information presentation system 1 includes a relative position acquisition device 75a and a relative distance calculation device 75b. The relative position acquisition device 75a acquires the position of work-related equipment 90 (see FIG. 1) and the position of a specific range R (see FIG. 1). The relative distance calculation device 75b calculates the relative distance L (see FIG. 1) based on the position of the work-related equipment 90 (see FIG. 1) and the position of the specific range R acquired by the relative position acquisition device 75a.

[0179] [Configuration 6] The relative position acquisition device 75a acquires information on the position (position information) of the work-related facility 90 shown in Figure 1 by one of the following methods [Method 6A] to [Method 6E] or a combination of multiple methods. [Method 6A] A method of acquiring one or both of the coordinates and shape of the work-related facility 90 that have been stored in advance. [Method 6B] A method of acquiring the type of work-related facility 90 and acquiring pre-stored shape information of the work-related facility 90 that corresponds to the type of work-related facility 90. [Method 6C] A method of detecting one or both of a two-dimensional image and a three-dimensional image of the work-related facility 90 and calculating one or both of the coordinates and shape of the work-related facility 90 from the detected image. [Method 6C] A method of detecting the coordinates of the work-related facility 90 by a position detection device 33b2 provided on the work-related facility 90. [Method 6D] A method of manually setting one or both of the coordinates and shape of the work-related facility 90 by the worker.

[0180] According to the above [Configuration 6], the relative position acquisition device 75a (see FIG. 2) can reliably acquire information on the position of the work-related equipment 90, and therefore can reliably acquire the position (position information) of the work-related equipment 90.

[0181] (Variation) The above-described embodiments (including modified examples within the embodiments (the same applies hereinafter)) may be modified in various ways. For example, the number of components in the above-described embodiments may be changed, or some of the components may not be provided. For example, the arrangement of the components may be changed. For example, components may be fixed or connected directly or indirectly to one another. For example, the connections between the components shown in FIG. 2 may be changed. For example, the inclusion relationships between the components may be changed in various ways. For example, a component described as a lower-level component included in a higher-level component may not be included in this higher-level component, but may be included in another component. For example, a component described as multiple different elements may be combined into a single element. For example, a component described as a single element may be provided as multiple different elements. For example, the order of the steps in the flowchart shown in FIG. 9 may be changed, or some of the steps may not be performed.

[0182] For example, various pieces of information (values, ranges, etc.) may be preset in the controller 1c, or may be set by being read into the controller 1c from an external device. Various pieces of information may be set in the controller 1c based on information manually set by an operator. Various pieces of information may be set in the controller 1c based on information detected by a detection device. For example, various pieces of information may not be changeable, may be changeable manually, or may be automatically changed by the controller 1c in response to certain conditions. For example, the controller 1c may perform substantially the same processing (calculation, determination, etc.) as the processing of the above-described embodiment. For example, the processing procedure, the information used in the processing, etc. may be changed in various ways. Specifically, the controller 1c may perform processing using information that can be converted into the various pieces of information used in the above-described embodiment. The processing performed by the controller 1c may be combined in various ways. For example, each component may have only a part of its characteristics (function, arrangement, shape, operation, etc.).

[0183] The information presentation system 1 is configured to perform each of the above operations. An information presentation program may be set that causes the controller 1c (computer) to execute processing to perform each of the above operations. An information presentation method may be performed to perform each of the above operations. Each of the above operations may be considered a "step" in the program and method. For example, the presentation of information by the information presentation device 57b may be considered an "information presentation step." [Explanation of symbols]

[0184] 1 Information presentation system 10. Work Machinery 16 Specific Locations 33b3 Attitude detection device 57b Information presentation device 75a Relative position acquisition device 75b Relative distance calculation device 90 Work-related equipment G21 Relative position display G23 Relative distance display L Relative distance R Specific range

Claims

1. an information presentation device that presents information according to a relative distance of a specific range, which is a range relating to a specific location of the work machine, with respect to work-related equipment, which is equipment arranged in the vicinity of the work machine; Information presentation system.

2. The information presentation system according to claim 1, The information presentation device includes: a relative position display including an image showing a relative position of the specific area with respect to the work-related equipment, the image showing the work-related equipment and the specific area from a specific direction; a relative distance display including an image according to the relative distance, which is separate from the relative position display; Displaying Information presentation system.

3. The information presentation system according to claim 1, an attitude detection device that detects the attitude of the work machine; a relative position acquisition device that acquires the position of the specific range calculated based on the attitude detected by the attitude detection device and the position of the work-related equipment; a relative distance calculation device that calculates the relative distance based on the position of the work-related equipment and the position of the specific range acquired by the relative position acquisition device; and Equipped with Information presentation system.

4. The information presentation system according to claim 1, The specific range is the entire or part of the operating range of the specific location. Information presentation system.

5. The information presentation system according to claim 4, a relative position acquisition device that acquires the position of the specific range acquired based on information on the operating range of the specific location that has been set in advance, and the position of the work-related equipment; a relative distance calculation device that calculates the relative distance based on the position of the work-related equipment and the position of the specific range acquired by the relative position acquisition device; and Equipped with Information presentation system.

6. The information presentation system according to claim 1, a relative position acquisition device that acquires the position of the work-related equipment and the position of the specific range; a relative distance calculation device that calculates the relative distance based on the position of the work-related equipment and the position of the specific range acquired by the relative position acquisition device; and Equipped with The relative position acquisition device A method for acquiring one or both of the coordinates and the shape of the work-related equipment stored in advance; A method of acquiring the type of the work-related equipment and acquiring pre-stored information on the shape of the work-related equipment corresponding to the type of the work-related equipment; a method for detecting one or both of a two-dimensional image and a three-dimensional image of the work-related equipment, and calculating one or both of the coordinates and the shape of the work-related equipment from the detected image; a method for detecting coordinates of the work-related equipment by a position detection device provided on the work-related equipment; a method of manually setting one or both of the coordinates and the shape of the work-related equipment by an operator; Acquire information on the location of the work-related equipment by one of the methods or a combination of multiple methods. Information presentation system.

Images