Construction machinery work support system

Abstract

To provide a work support system of construction machinery capable of freely and easily setting up a virtual wall.SOLUTION: A work support system of construction machinery includes a piece of construction machinery, a management unit, and a support unit. The management unit has a set instruction unit that instructs the construction machinery to set, as a virtual wall, a plane specified based on the coordinates of multiple points in a space acquired in the support unit.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present invention relates to a work support system for construction machinery.

Background Art

[0002] Conventionally, there is known a technique of setting a virtual wall, which is a virtual wall that divides the working range of an excavator such as a shovel, and preventing contact between an object outside the virtual wall and the shovel.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In the above-described conventional technology, in order to set a virtual wall, it is necessary to install a mark such as a load cone in the working range of the shovel, and the virtual wall cannot be set freely and easily.

[0005] Therefore, in view of the above problems, an object is to set a virtual wall freely and easily. [[ID=4*]]

Means for Solving the Problems

[0006] A work support system for construction machinery according to an embodiment of the present invention is a work support system for construction machinery including a construction machinery, a management device, and a support device, wherein the management device has The aforementioned support device displays an AR image that guides the setting of a new virtual wall, a setting instruction unit that causes the construction machinery to set, as a virtual wall, a plane specified based on coordinate values of a plurality of points in space acquired by the support device. , set according to the AR image a work support system for construction machinery.

[0007] An embodiment of the present invention is a work support system for construction machinery comprising a construction machine and a support device, wherein the support device is An AR image is displayed to guide you through setting up a new virtual wall, Acquired by the recording support device , set according to the AR image This is a work support system for construction machinery, which has a setting instruction unit that sets a surface identified based on the coordinate values ​​of multiple points in space as a virtual wall for the construction machinery. [Effects of the Invention]

[0008] Virtual walls can be set up freely and easily. [Brief explanation of the drawing]

[0009] [Figure 1] This figure shows an example of a system configuration for a work support system for construction machinery. [Figure 2] This diagram shows an example of a hydraulic system configuration installed in an excavator. [Figure 3] This figure shows an example of the hardware configuration of the management device. [Figure 4] This diagram illustrates the functional configuration of each device in the work support system. [Figure 5] This is a diagram explaining how to set up virtual walls. [Figure 6] This is a sequence diagram illustrating the operation of the work support system. [Figure 7] This is the first diagram illustrating the screen transitions of the support device. [Figure 8] This is the second diagram illustrating the screen transitions of the support device. [Figure 9] This is the third diagram illustrating the screen transitions of the support device. [Figure 10] This is a diagram showing an example of a shovel display. [Modes for carrying out the invention]

[0010] Hereinafter, with reference to the drawings, a work support system for a construction machine according to this embodiment will be described. FIG. 1 is a diagram showing an example of the system configuration of a work support system for a construction machine. In this embodiment, an excavator 100 will be described as an example of a construction machine.

[0011] The work support system SYS for a construction machine according to this embodiment includes an excavator 100, a management device 200, and a support device 300. In the following description, the work support system SYS for a construction machine will be simply referred to as the work support system SYS.

[0012] In the work support system SYS according to this embodiment, the excavator 100, the management device 200, and the support device 300 are connected via a network or the like.

[0013] The excavator 100 according to this embodiment acquires operation information indicating the operation status of the machine itself, transmits it to the management device 200, and receives various types of information from the management device 200.

[0014] Specifically, the operation information of the excavator 100 includes position information indicating the current position of the machine itself, orientation information indicating the orientation of the machine itself, attitude information indicating the attitude of the machine itself, work content information indicating the work content, load rate information recording the load rate, cumulative time information indicating the cumulative operation time, fuel information including the fuel injection amount, CO2 emission amount, work volume, and the like.

[0015] In addition, the excavator 100 according to this embodiment controls its operation so that the machine itself does not contact a virtual wall set using the support device 300. The virtual wall according to this embodiment is a virtual wall that divides the working range of the excavator 100. In other words, the virtual wall is a surface that the excavator 100 is prohibited from straddling.

[0016] When a virtual wall is set using the support device 300, the excavator 100 of the present embodiment recognizes the surrounding environment as if there is an actual wall at the position of the virtual wall, and restricts the movement of the excavator 100 so that the excavator 100 does not contact the (non-existent) wall. That is, the virtual wall functions as a virtual protective wall that prevents contact between an object in front of the virtual wall and the excavator 100.

[0017] The management device 200 receives operation information from the excavator 100, and aggregates the operation information for each work content of the excavator 100 indicated by the state information included in the operation information. Then, the management device 200 specifies a period during which each actuator operation is stopped while the engine is running (idling period) within a period when the excavator 100 is in a state of pausing work, and causes the display device 40 of the excavator 100 to display information regarding fuel during the specified period.

[0018] During a period when the excavator 100 is in a state of pausing work, in addition to a period during which each actuator operation is stopped while the engine is running (idling period), there is also a period during which the engine is stopped.

[0019] As described above, the state in which each actuator operation of the excavator 100 is stopped includes a state in which the engine 11 as the prime mover of the excavator 100 is turned on and no lever operation is performed, and a state in which the engine 11 is turned off.

[0020] Note that an electric motor may be used as the prime mover instead of the engine. In this case, a power storage device is also mounted to supply power to the electric motor. The power storage device is a device for storing power, and examples thereof include an electric double layer capacitor, a lithium ion battery, and a nickel hydrogen battery. In this case, the excavator 100 controls the electric motor using an inverter to rotationally drive the main pump 14 with the power stored in the power storage device.

[0021] Furthermore, the management device 200 in this embodiment displays a guide image to the support device 300 for setting multiple points in space. When the support device 300 has set multiple points and input the coordinate values ​​of each point, the management device 200 identifies a surface from the coordinate values ​​of each point and sets the identified surface as a virtual wall on the shovel 100.

[0022] The support device 300 is, for example, a terminal device for assisting the operation of the shovel 100. Specifically, the support device 300 in this embodiment is a glasses-type wearable terminal worn by worker P who assists the operation of the shovel 100, and may be AR (Augmented Reality) glasses with augmented reality functionality. Worker P may be a different person from the operator who operates the shovel 100.

[0023] The display device of the support device 300 in this embodiment displays a guide image for setting multiple points in space that identify a surface to be a virtual wall. When the user of the support device 300 performs the operation to set multiple points, the device acquires the coordinate values ​​of each point and transmits them to the management device 200. In this embodiment, the coordinate values ​​are three-dimensional coordinates representing points in space, and are coordinate values ​​in the world coordinate system.

[0024] In the example shown in Figure 1, the management device 200 is assumed to be implemented by a single information processing device, but this is not limited to that. The management device 200 may be implemented by multiple information processing devices. In other words, the functions implemented by the management device 200 may be implemented by multiple information processing devices.

[0025] Furthermore, in the example shown in Figure 1, the support device 300 is an AR glasses worn by the worker P, but it is not limited to this. The support device 300 in this embodiment may be, for example, a portable tablet terminal such as a smartphone.

[0026] Next, the shovel 100 of this embodiment will be described. Figure 1 shows a side view of the shovel 100.

[0027] Excavator 100 has a lower traveling body 1, a slewing mechanism 2, and an upper slewing body 3. In excavator 100, the upper slewing body 3 is rotatably mounted on the lower traveling body 1 via the slewing mechanism 2. A boom 4 is attached to the upper slewing body 3. An arm 5 is attached to the tip of the boom 4, and a bucket 6 is attached to the tip of the arm 5 as an end attachment.

[0028] The boom 4, arm 5, and bucket 6 constitute an excavation attachment as an example of an attachment. The boom 4 is driven by the boom cylinder 7, the arm 5 is driven by the arm cylinder 8, and the bucket 6 is driven by the bucket cylinder 9. A boom angle sensor S1 is attached to the boom 4, an arm angle sensor S2 is attached to the arm 5, and a bucket angle sensor S3 is attached to the bucket 6.

[0029] The boom angle sensor S1 is configured to detect the rotation angle of the boom 4. In this embodiment, the boom angle sensor S1 is an acceleration sensor and can detect the rotation angle of the boom 4 relative to the upper slewing body 3 (hereinafter referred to as "boom angle"). The boom angle is smallest when the boom 4 is lowered to its lowest position, and increases as the boom 4 is raised.

[0030] The arm angle sensor S2 is configured to detect the rotation angle of the arm 5. In this embodiment, the arm angle sensor S2 is an acceleration sensor and can detect the rotation angle of the arm 5 relative to the boom 4 (hereinafter referred to as "arm angle"). The arm angle is smallest when the arm 5 is closed to its shortest extent, and increases as the arm 5 is opened.

[0031] The bucket angle sensor S3 is configured to detect the rotation angle of the bucket 6. In this embodiment, the bucket angle sensor S3 is an acceleration sensor and can detect the rotation angle of the bucket 6 relative to the arm 5 (hereinafter referred to as the "bucket angle"). The bucket angle is smallest when the bucket 6 is closed to its fullest extent, and increases as the bucket 6 is opened.

[0032] The boom angle sensor S1, arm angle sensor S2, and bucket angle sensor S3 may each be a potentiometer using a variable resistor, a stroke sensor for detecting the stroke amount of the corresponding hydraulic cylinder, a rotary encoder for detecting the rotation angle around the connecting pin, a gyro sensor, or a combination of an acceleration sensor and a gyro sensor.

[0033] The boom cylinder 7 is equipped with a boom rod pressure sensor S7R and a boom bottom pressure sensor S7B. The arm cylinder 8 is equipped with an arm rod pressure sensor S8R and an arm bottom pressure sensor S8B.

[0034] The bucket cylinder 9 is equipped with a bucket rod pressure sensor S9R and a bucket bottom pressure sensor S9B. The boom rod pressure sensor S7R, boom bottom pressure sensor S7B, arm rod pressure sensor S8R, arm bottom pressure sensor S8B, bucket rod pressure sensor S9R, and bucket bottom pressure sensor S9B are collectively referred to as "cylinder pressure sensors".

[0035] The boom rod pressure sensor S7R detects the pressure in the rod-side oil chamber of the boom cylinder 7 (hereinafter referred to as "boom rod pressure"), and the boom bottom pressure sensor S7B detects the pressure in the bottom-side oil chamber of the boom cylinder 7 (hereinafter referred to as "boom bottom pressure"). The arm rod pressure sensor S8R detects the pressure in the rod-side oil chamber of the arm cylinder 8 (hereinafter referred to as "arm rod pressure"), and the arm bottom pressure sensor S8B detects the pressure in the bottom-side oil chamber of the arm cylinder 8 (hereinafter referred to as "arm bottom pressure").

[0036] The bucket rod pressure sensor S9R detects the pressure in the rod-side oil chamber of the bucket cylinder 9 (hereinafter referred to as "bucket rod pressure"), and the bucket bottom pressure sensor S9B detects the pressure in the bottom-side oil chamber of the bucket cylinder 9 (hereinafter referred to as "bucket bottom pressure").

[0037] The upper rotating body 3 is equipped with a cabin 10, which serves as the driver's cab, and a power source such as an engine 11. A sensor for detecting CO2 emissions may also be provided near the exhaust mechanism of the engine 11.

[0038] Furthermore, the upper rotating body 3 is equipped with a controller 30, a display device 40, an input device 42, an audio output device 43, a storage device 47, a positioning device P1, an aircraft tilt sensor S4, a rotation angular velocity sensor S5, an imaging device S6, and a communication device T1.

[0039] The upper rotating body 3 may be equipped with a power storage unit for supplying electricity, and a motor-generator that generates electricity using the rotational driving force of the engine 11. The power storage unit may be, for example, a capacitor or a lithium-ion battery. The motor-generator may function as an electric motor to drive a mechanical load, or as a generator to supply power to an electrical load.

[0040] The controller 30 functions as a main control unit that controls the drive of the shovel 100. In this embodiment, the controller 30 is composed of a computer including a CPU, RAM, and ROM. Various functions of the controller 30 are realized, for example, by the CPU executing a program stored in ROM. These functions may include, for example, at least one of a machine guidance function that guides the operator in manually operating the shovel 100, and a machine control function that automatically assists the operator in manually operating the shovel 100.

[0041] The display device 40 is configured to display various types of information. The display device 40 may be connected to the controller 30 via a communication network such as CAN, or it may be connected to the controller 30 via a dedicated line.

[0042] The input device 42 is configured to allow the operator to input various types of information to the controller 30. The input device 42 includes at least one of the following: a touch panel, a knob switch, and a membrane switch, all of which are installed inside the cabin 10.

[0043] The audio output device 43 is configured to output sound. The audio output device 43 may be, for example, an in-vehicle speaker connected to the controller 30, or an alarm device such as a buzzer. In this embodiment, the audio output device 43 is configured to output various information as sound in response to an audio output command from the controller 30.

[0044] The storage device 47 is configured to store various types of information. The storage device 47 is, for example, a non-volatile storage medium such as a semiconductor memory. The storage device 47 may store information output by various devices during the operation of the shovel 100, or it may store information acquired via various devices before the operation of the shovel 100 begins.

[0045] The storage device 47 may store data relating to the target construction surface, for example, obtained via a communication device T1. The target construction surface may be set by the operator of the shovel 100, or by the construction manager or the like.

[0046] The positioning device P1 is configured to measure the position of the upper rotating body 3. The positioning device P1 may also be configured to measure the orientation of the upper rotating body 3. In this embodiment, the positioning device P1 is, for example, a GNSS compass, which detects the position and orientation of the upper rotating body 3 and outputs the detected values ​​to the controller 30. Therefore, the positioning device P1 can also function as an orientation detection device to detect the orientation of the upper rotating body 3. The orientation detection device may be an orientation sensor attached to the upper rotating body 3.

[0047] The machine body tilt sensor S4 is configured to detect the tilt of the upper rotating body 3. In this embodiment, the machine body tilt sensor S4 is an acceleration sensor that detects the longitudinal tilt angle of the upper rotating body 3 around the longitudinal axis and the lateral tilt angle around the lateral axis with respect to a virtual horizontal plane. The longitudinal axis and lateral axis of the upper rotating body 3 are orthogonal to each other at the shovel center point, which is a point on the rotation axis of the shovel 100.

[0048] The rotational angular velocity sensor S5 is configured to detect the rotational angular velocity of the upper rotating body 3. The rotational angular velocity sensor S5 may also be configured to detect or calculate the rotation angle of the upper rotating body 3. In this embodiment, the rotational angular velocity sensor S5 is a gyro sensor. The rotational angular velocity sensor S5 may also be a resolver, a rotary encoder, or the like.

[0049] The imaging device S6 is an example of a spatial recognition device and is configured to acquire images of the area around the shovel 100. In this embodiment, the imaging device S6 includes a front camera S6F for imaging the space in front of the shovel 100, a left camera S6L for imaging the space to the left of the shovel 100, a right camera S6R for imaging the space to the right of the shovel 100, and a rear camera S6B for imaging the space behind the shovel 100.

[0050] The imaging device S6 is, for example, a monocular camera having an image sensor such as a CCD or CMOS, and outputs the captured image to the display device 40. The imaging device S6 may also be a stereo camera, a depth image camera, etc. Furthermore, the imaging device S6 may be replaced with other spatial recognition devices such as a 3D depth image sensor, an ultrasonic sensor, a millimeter-wave radar, a LiDAR or an infrared sensor, or it may be replaced with a combination of other spatial recognition devices and a camera.

[0051] The front camera S6F is mounted, for example, on the ceiling of the cabin 10, i.e., inside the cabin 10. However, the front camera S6F may also be mounted on the roof of the cabin 10, the side of the boom 4, or other external locations within the cabin 10. The left camera S6L is mounted on the upper left end of the upper surface of the upper slewing body 3, the right camera S6R is mounted on the upper right end of the upper surface of the upper slewing body 3, and the rear camera S6B is mounted on the upper rear end of the upper surface of the upper slewing body 3.

[0052] The communication device T1 is configured to control communication with external devices located outside the excavator 100. In this embodiment, the communication device T1 controls communication with external devices via a satellite communication network, a mobile phone communication network, or the Internet network. The external devices are, for example, a management device 200 such as a server installed in an external facility, or a support device 300 such as a smartphone carried by a worker around the excavator 100.

[0053] Next, with reference to Figure 2, an example of the configuration of the hydraulic system installed in the excavator 100 will be described. Figure 2 is a diagram showing an example of the configuration of the hydraulic system installed in the excavator. In Figure 2, the mechanical power transmission line, hydraulic fluid line, pilot line, and electrical control line are shown with double lines, solid lines, dashed lines, and dotted lines, respectively.

[0054] The hydraulic system circulates hydraulic fluid from the left main pump 14L, driven by engine 11, through the left center bypass pipe 40L or the left parallel pipe 42L to the hydraulic fluid tank, and also circulates hydraulic fluid from the right main pump 14R, driven by engine 11, through the right center bypass pipe 40R or the right parallel pipe 42R to the hydraulic fluid tank.

[0055] The left center bypass pipeline 40L is a hydraulic fluid line that passes through control valves 171, 173, 175L, and 176L located within the control valve unit 17. The right center bypass pipeline 40R is a hydraulic fluid line that passes through control valves 172, 174, 175R, and 176R located within the control valve unit 17.

[0056] The control valve 171 is a spool valve that switches the flow of hydraulic fluid to supply the hydraulic fluid discharged by the left main pump 14L to the left-side travel hydraulic motor 1L, and to discharge the hydraulic fluid discharged by the left-side travel hydraulic motor 1L to the hydraulic fluid tank.

[0057] The control valve 172 is a spool valve that supplies the hydraulic fluid discharged by the right main pump 14R to the right-side travel hydraulic motor 1R, and also switches the flow of hydraulic fluid to discharge the hydraulic fluid discharged by the right-side travel hydraulic motor 1R to the hydraulic fluid tank.

[0058] The control valve 173 is a spool valve that supplies the hydraulic fluid discharged by the left main pump 14L to the swing hydraulic motor 2A, and also switches the flow of the hydraulic fluid discharged by the swing hydraulic motor 2A to the hydraulic fluid tank.

[0059] The control valve 174 is a spool valve that supplies the hydraulic fluid discharged by the right main pump 14R to the bucket cylinder 9 and switches the flow of the hydraulic fluid in order to discharge the hydraulic fluid in the bucket cylinder 9 to the hydraulic fluid tank.

[0060] The control valve 175L is a spool valve that switches the flow of hydraulic fluid in order to supply the hydraulic fluid discharged by the left main pump 14L to the boom cylinder 7.

[0061] The control valve 175R is a spool valve that supplies the hydraulic fluid discharged by the right main pump 14R to the boom cylinder 7 and also switches the flow of hydraulic fluid in order to discharge the hydraulic fluid in the boom cylinder 7 to the hydraulic fluid tank.

[0062] The control valve 176L is a spool valve that supplies the hydraulic fluid discharged by the left main pump 14L to the arm cylinder 8, and also switches the flow of the hydraulic fluid in order to discharge the hydraulic fluid in the arm cylinder 8 to the hydraulic fluid tank.

[0063] The control valve 176R is a spool valve that supplies the hydraulic fluid discharged by the right main pump 14R to the arm cylinder 8 and switches the flow of the hydraulic fluid in order to discharge the hydraulic fluid in the arm cylinder 8 to the hydraulic fluid tank.

[0064] The left parallel pipeline 42L is a hydraulic fluid line running parallel to the left center bypass pipeline 40L. The left parallel pipeline 42L can supply hydraulic fluid to a control valve further downstream if the flow of hydraulic fluid through the left center bypass pipeline 40L is restricted or blocked by any of the control valves 171, 173, or 175L. The right parallel pipeline 42R is a hydraulic fluid line running parallel to the right center bypass pipeline 40R. The right parallel pipeline 42R can supply hydraulic fluid to a control valve further downstream if the flow of hydraulic fluid through the right center bypass pipeline 40R is restricted or blocked by any of the control valves 172, 174, or 175R.

[0065] The left regulator 13L is configured to control the discharge rate of the left main pump 14L. In this embodiment, the left regulator 13L controls the discharge rate of the left main pump 14L by adjusting the swash plate tilt angle of the left main pump 14L in accordance with the discharge pressure of the left main pump 14L. The right regulator 13R is configured to control the discharge rate of the right main pump 14R. In this embodiment, the right regulator 13R controls the discharge rate of the right main pump 14R by adjusting the swash plate tilt angle of the right main pump 14R in accordance with the discharge pressure of the right main pump 14R. The left regulator 13L reduces the discharge rate by adjusting the swash plate tilt angle of the left main pump 14L in accordance with an increase in the discharge pressure of the left main pump 14L. The same applies to the right regulator 13R. This is to ensure that the pump absorption horsepower, which is expressed as the product of the discharge pressure and the discharge rate, does not exceed the output horsepower of the engine 11. Note that the pump absorption horsepower is the sum of the absorption horsepower of the left main pump (14L) and the absorption horsepower of the right main pump (14R).

[0066] The left discharge pressure sensor 28L is an example of a discharge pressure sensor 28, which detects the discharge pressure of the left main pump 14L and outputs the detected value to the controller 30. The same applies to the right discharge pressure sensor 28R.

[0067] Here, we will explain the negative control system employed in the hydraulic system shown in Figure 2.

[0068] In the left center bypass pipeline 40L, a left throttle 18L is located between the downstream control valve 176L and the hydraulic oil tank. The flow of hydraulic oil discharged by the left main pump 14L is restricted by the left throttle 18L. The left throttle 18L then generates a control pressure to control the left regulator 13L. The left control pressure sensor 19L is a sensor for detecting the control pressure and outputs the detected value to the controller 30. In the right center bypass pipeline 40R, a right throttle 18R is located between the downstream control valve 176R and the hydraulic oil tank. The flow of hydraulic oil discharged by the right main pump 14R is restricted by the right throttle 18R. The right throttle 18R then generates a control pressure to control the right regulator 13R. The right control pressure sensor 19R is a sensor for detecting the control pressure and outputs the detected value to the controller 30.

[0069] The controller 30 controls the discharge volume of the left main pump 14L by adjusting the swash plate tilt angle of the left main pump 14L in accordance with the control pressure. The controller 30 decreases the discharge volume of the left main pump 14L as the control pressure increases, and increases the discharge volume of the left main pump 14L as the control pressure decreases. The discharge volume of the right main pump 14R is controlled in the same manner.

[0070] Specifically, as shown in Figure 2, when none of the hydraulic actuators in the shovel 100 are operated and the system is in standby mode, the hydraulic fluid discharged from the left main pump 14L passes through the left center bypass pipe 40L to the left constrictor 18L. The flow of hydraulic fluid discharged from the left main pump 14L increases the control pressure generated upstream of the left constrictor 18L. As a result, the controller 30 reduces the discharge volume of the left main pump 14L to the minimum allowable discharge volume, suppressing pressure loss (pumping loss) as the discharged hydraulic fluid passes through the left center bypass pipe 40L. On the other hand, when any of the hydraulic actuators are operated, the hydraulic fluid discharged from the left main pump 14L flows into the hydraulic actuator being operated via the control valve corresponding to that actuator. The flow of hydraulic fluid discharged from the left main pump 14L reduces or eliminates the amount reaching the left constrictor 18L, lowering the control pressure generated upstream of the left constrictor 18L. As a result, the controller 30 increases the discharge volume of the left main pump 14L, circulating sufficient hydraulic fluid to the hydraulic actuator being operated, and ensuring reliable operation of the hydraulic actuator. The same applies to the hydraulic fluid discharged by the right main pump 14R.

[0071] With the configuration described above, the hydraulic system in Figure 2 can suppress wasted energy consumption in both the left main pump 14L and the right main pump 14R when in standby mode. Wasted energy consumption includes pumping losses caused by the hydraulic fluid discharged by the left main pump 14L in the left center bypass pipeline 40L, and pumping losses caused by the hydraulic fluid discharged by the right main pump 14R in the right center bypass pipeline 40R. Furthermore, when operating a hydraulic actuator, the hydraulic system in Figure 2 can supply the necessary and sufficient amount of hydraulic fluid to the hydraulic actuator being operated from both the left main pump 14L and the right main pump 14R.

[0072] Next, a configuration for automatically operating the actuator will be described. The boom operating lever 26A is an example of an electric operating lever as an operating device 26, and is used to operate the boom 4. The boom operating lever 26A detects the direction and amount of operation, and outputs the detected direction and amount of operation as operation data (electrical signal) to the controller 30. When manually controlled, if the boom operating lever 26A is operated in the boom raising direction, the controller 30 controls the opening degree of the proportional valve 31AL according to the amount of operation of the boom operating lever 26A.

[0073] This allows the pilot pump 15 to use the hydraulic fluid it discharges to apply a pilot pressure corresponding to the amount the boom operating lever 26A is operated to the right pilot port of the control valve 175L and the left pilot port of the control valve 175R. Furthermore, when manually controlled, if the boom operating lever 26A is operated in the boom lowering direction, the controller 30 controls the opening degree of the proportional valve 31AR according to the amount the boom operating lever 26A is operated to. This allows the pilot pump 15 to use the hydraulic fluid it discharges to apply a pilot pressure corresponding to the amount the boom operating lever 26A is operated to the right pilot port of the control valve 175R.

[0074] The proportional valves 31AL and 31AR constitute a boom proportional valve 31A, which is an example of a proportional valve 31 as a solenoid valve. The proportional valve 31AL operates in response to a current command regulated by the controller 30. The controller 30 adjusts the pilot pressure using hydraulic fluid introduced from the pilot pump 15 through the proportional valve 31AL to the right pilot port of the control valve 175L and the left pilot port of the control valve 175R.

[0075] The proportional valve 31AR operates in response to a current command regulated by the controller 30. The controller 30 regulates the pilot pressure by the hydraulic fluid introduced from the pilot pump 15 through the proportional valve 31AR to the right-side pilot port of the control valve 175R. The proportional valves 31AL and 31AR are capable of adjusting the pilot pressure so that the control valves 175L and 175R can be stopped at any valve position.

[0076] With this configuration, during automatic excavation control, the controller 30 can supply the hydraulic fluid discharged by the pilot pump 15 to the right pilot port of the control valve 175L and the left pilot port of the control valve 175R via the proportional valve 31AL, independently of the boom raising operation by the operator. In other words, the controller 30 can automatically raise the boom 4. Furthermore, the controller 30 can supply the hydraulic fluid discharged by the pilot pump 15 to the right pilot port of the control valve 175R via the proportional valve 31AR, independently of the boom lowering operation by the operator. In other words, the controller 30 can automatically lower the boom 4.

[0077] The arm operating lever 26B is another example of an electrically operated lever as an operating device 26, and is used to operate the arm 5. The arm operating lever 26B detects the direction and amount of operation and outputs the detected direction and amount of operation as operation data (electrical signal) to the controller 30. When manually controlled, if the arm operating lever 26B is operated in the arm opening direction, the controller 30 controls the opening degree of the proportional valve 31BR according to the amount of operation of the arm operating lever 26B.

[0078] This allows the pilot pump 15 to use the hydraulic fluid it discharges to apply pilot pressure corresponding to the amount the arm operating lever 26B is operated to the left pilot port of the control valve 176L and the right pilot port of the control valve 176R. Furthermore, when manually controlled, if the arm operating lever 26B is operated in the arm closing direction, the controller 30 controls the opening degree of the proportional valve 31BL according to the amount the arm operating lever 26B is operated. This allows the pilot pump 15 to use the hydraulic fluid it discharges to apply pilot pressure corresponding to the amount the arm operating lever 26B is operated to the right pilot port of the control valve 176L and the left pilot port of the control valve 176R.

[0079] The proportional valves 31BL and 31BR constitute an arm proportional valve 31B, which is an example of the proportional valve 31. The proportional valve 31BL operates in response to a current command regulated by the controller 30. The controller 30 regulates the pilot pressure using hydraulic fluid introduced from the pilot pump 15 through the proportional valve 31BL to the right pilot port of the control valve 176L and the left pilot port of the control valve 176R. The proportional valve 31BR operates in response to a current command regulated by the controller 30. The controller 30 regulates the pilot pressure using hydraulic fluid introduced from the pilot pump 15 through the proportional valve 31BR to the left pilot port of the control valve 176L and the right pilot port of the control valve 176R. The proportional valves 31BL and 31BR can adjust the pilot pressure so that the control valves 176L and 176R can be stopped at any valve position.

[0080] With this configuration, the controller 30 can supply the hydraulic fluid discharged by the pilot pump 15 to the right pilot port of the control valve 176L and the left pilot port of the control valve 176R via the proportional valve 31BL, independently of the operator's arm closing operation. In other words, the controller 30 can automatically close the arm 5. Furthermore, the controller 30 can supply the hydraulic fluid discharged by the pilot pump 15 to the left pilot port of the control valve 176L and the right pilot port of the control valve 176R via the proportional valve 31BR, independently of the operator's arm opening operation. In other words, the controller 30 can automatically open the arm 5.

[0081] As a result, in automatic excavation control, the arm cylinder 8 and boom cylinder 7 operate automatically according to the amount of movement of the arm operation lever 26B, thereby controlling the speed or position of the work area.

[0082] The shovel 100 may be equipped with a configuration for automatically rotating the upper slewing body 3 left and right, a configuration for automatically opening and closing the bucket 6, and a configuration for automatically moving the lower traveling body 1 forward and backward. In this case, the hydraulic system portion related to the slewing hydraulic motor 2A, the hydraulic system portion related to the operation of the bucket cylinder 9, the hydraulic system portion related to the operation of the left-side traveling hydraulic motor 1L, and the hydraulic system portion related to the operation of the right-side traveling hydraulic motor 1R may be configured in the same way as the hydraulic system portion related to the operation of the boom cylinder 7, etc.

[0083] Next, the hardware configuration of the management device 200 in this embodiment will be described with reference to Figure 3. Figure 3 is a diagram showing an example of the hardware configuration of the management device.

[0084] The management device 200 in this embodiment is a computer that includes an input device 201, an output device 202, a drive device 203, an auxiliary storage device 204, a memory device 205, an arithmetic processing unit 206, and an interface device 207, all of which are interconnected via bus B.

[0085] The input device 201 is a device for inputting various types of information and can be implemented, for example, by a touch panel. The output device 202 is for outputting various types of information and can be implemented, for example, by a display. The interface device 207 is used to connect to a network.

[0086] The work support program implemented by the various components described later is at least a part of the various programs that control the management device 200. The work support program is provided, for example, by distribution of the storage medium 208 or by downloading it from a network. The storage medium 208 on which the work support program is recorded can be of various types, such as storage media that record information optically, electrically, or magnetically, or semiconductor memory that records information electrically, such as ROM or flash memory.

[0087] Furthermore, when the storage medium 208 containing the work support program is set in the drive device 203, the work support program is installed from the storage medium 208 to the auxiliary storage device 204 via the drive device 203. Work support programs downloaded from the network are installed to the auxiliary storage device 204 via the interface device 207.

[0088] The auxiliary storage device 204 stores the work support program installed on the management device 200, as well as various necessary files and data from the management device 200. The memory device 205 reads the work support program from the auxiliary storage device 204 and stores it when the management device 200 starts up. The arithmetic processing unit 206 then performs various processes as described later, according to the work support program stored in the memory device 205.

[0089] Furthermore, the support device 300 in this embodiment is a computer including a processing unit and a memory device, similar to the management device 200. The support device 300 may also include a display device as an output device and have hardware switches or the like as input devices. The hardware switches may be, for example, operation buttons that are operated when setting a point in space in the support device 300.

[0090] Furthermore, in addition to the devices shown in Figure 3, the support device 300 includes a GPS receiver that implements GPS (Global Positioning System) functionality, an IMU (Inertial Measurement Unit), and an imaging device. The GPS receiver receives GPS signals from GPS satellites. The IMU is an inertial measurement device that detects the acceleration of the support device 300 and the direction in which the support device 300 is facing. Moreover, the support device 300 in this embodiment may also have a sound collection device to acquire audio data.

[0091] Next, with reference to Figure 4, the functional configuration of each device in the SYS work support system of this embodiment will be described. Figure 4 is a diagram illustrating the functional configuration of each device in the work support system.

[0092] First, the functional configuration of the support device 300 will be described. The support device 300 of this embodiment includes an input receiving unit 310, a display control unit 320, a coordinate value acquisition unit 330, and a communication control unit 340.

[0093] The input receiving unit 310 receives various inputs to the support device 300. Specifically, the input receiving unit 310 receives input operations from the operator P to the input device and operation instructions via voice data acquired by the sound collection device.

[0094] The display control unit 320 causes the display device to display the image data received from the management device 200 via the communication control unit 340. The image data received from the management device 200 may be a guide image that guides the setting of points in space, or it may be an image of a virtual wall set in the management device 200.

[0095] When the coordinate value acquisition unit 330 receives an operation to set a point in space from the input reception unit 310, it acquires the coordinate values ​​of the point in space based on position information indicating the position of the support device 300 at that time and the line of sight of the worker P, who is wearing the support device 300.

[0096] In other words, the coordinate value acquisition unit 330 acquires the coordinate values ​​of a point in space based on positional information indicating the position of the support device 300 and the direction in which the support device 300 is facing. The direction in which the support device 300 is facing is the direction of the worker P's line of sight. In the following explanation, the direction of the worker P's line of sight will be expressed as the line of sight vector.

[0097] The communication control unit 340 controls communication between the support device 300 and the management device 200. The communication control unit 340 may also control communication between the support device 300 and the shovel 100.

[0098] Next, the functional configuration of the management device 200 will be described. The management device 200 includes a display control unit 210, a communication control unit 220, a coordinate value holding unit 230, a surface identification unit 240, and a setting instruction unit 250.

[0099] The display control unit 210, in response to a request from the support device 300, transmits image data showing a guide image that guides the setting of coordinate points to the support device 300, and has it displayed on the display device of the support device 300. The display control unit 210 also has the display of the virtual wall image set by the setting instruction unit 250 displayed on the display device of the support device 300 and on the display device 40 of the shovel 100.

[0100] The communication control unit 220 controls communication between the management device 200 and the support device 300. The communication control unit 220 also controls communication between the management device 200 and the shovel 100.

[0101] The coordinate value holding unit 230 acquires and holds the coordinate values ​​set in the support device 300. The coordinate value holding unit 230 holds multiple coordinate values ​​necessary for setting the virtual wall.

[0102] The surface identification unit 240 identifies a surface from multiple points indicated by each of the multiple coordinate values ​​held in the coordinate value holding unit 230.

[0103] In this embodiment, three coordinate values ​​may be set as multiple coordinate values ​​for identifying a surface. Alternatively, in this embodiment, two coordinate values ​​may be set as coordinate values ​​for identifying a surface, and the surface may be identified based on the coordinate values ​​of the two points and the direction of gravity.

[0104] The setting instruction unit 250 designates the identified surface as a virtual wall and transmits information indicating the virtual wall to the support device 300 and the shovel 100. The information indicating the virtual wall may be the coordinate values ​​of three points that identify the surface. Alternatively, the information indicating the virtual wall may be the coordinate values ​​of two points that identify the surface and information indicating the direction of gravity.

[0105] Next, the functional configuration of the shovel 100 will be explained. The functional configuration of the shovel 100 is realized by the controller 30 reading a program stored in a storage device such as ROM.

[0106] The shovel 100 includes a virtual wall setting unit 110, an operation control unit 120, and a display control unit 130.

[0107] The virtual wall setting unit 110 acquires information indicating a virtual wall from the management device 200 and sets the virtual wall. Specifically, the virtual wall setting unit 110 stores the information indicating a virtual wall acquired from the management device 200 in the storage device 47 and controls the operation of the shovel 100 based on the positional relationship between the surface that will become the virtual wall and the shovel 100 when the shovel is operating.

[0108] The motion control unit 120 controls the operation so that the machine does not come into contact with the virtual wall set by the virtual wall setting unit 110. In other words, the motion control unit 120 in this embodiment restricts the movement of the actuator based on the positional relationship between the virtual wall and the shovel 100. Specifically, the motion control unit 120 may stop the operation of the shovel 100 when the shovel 100 approaches the virtual wall to a position where the distance to the virtual wall is within a predetermined distance.

[0109] The display control unit 130 controls the display on the display device 40 of the shovel 100. Specifically, the display control unit 130 causes the display device 40 of the shovel 100 to display an image of a virtual wall that is set based on information indicating a virtual wall.

[0110] Now, with reference to Figure 5, the method for setting up virtual walls in this embodiment will be explained. Figure 5 is a diagram illustrating how to set up virtual walls.

[0111] In this embodiment, for example, worker P is made to view a point in space from multiple different locations while wearing the support device 300, and the position where the line-of-sight vectors intersect is defined as the point in space.

[0112] In this embodiment, for example, markers are placed on objects such as utility poles, fences, and the ground installed in the work environment, and by making these markers visible to worker P, the position of the markers is defined as the position of a point in space.

[0113] In the example shown in Figure 5, markers M1, M2, and M3 are attached to utility poles 51a and 51b, respectively.

[0114] In this embodiment, first, worker P visually identifies markers M1, M2, and M3 from point E1 via the support device 300. At this time, worker P adjusts their line of sight so that markers M1, M2, and M3 are all within the guide image displayed on the support device 300.

[0115] At this time, the support device 300 identifies the line-of-sight vector Ym11 from point E1 to marker M1, the line-of-sight vector Ym21 from point E1 to marker M2, and the line-of-sight vector Ym31 from point E1 to marker M3.

[0116] Next, worker P visually inspects markers M1, M2, and M3 from point E2 via the support device 300. At this time, worker P adjusts their line of sight so that markers M1, M2, and M3 are all within the guide image displayed on the support device 300.

[0117] At this time, the support device 300 identifies the line-of-sight vector Ym12 from point E2 to marker M1, the line-of-sight vector Ym22 from point E2 to marker M2, and the line-of-sight vector Ym32 from point E2 to marker M3.

[0118] In this way, when each marker is viewed from multiple different locations, the line-of-sight vectors intersect at the marker. In the example in Figure 5, worker P is shown viewing the markers from multiple different locations, but this task may be performed by multiple workers. Specifically, for example, the worker viewing each marker from location E1 and the worker viewing each marker from location E2 may be different people. Furthermore, the task of viewing the markers from each location may be performed at an independent time for each location.

[0119] In the example in Figure 5, the line-of-sight vectors Ym11 and Ym21 intersect at the position of marker M1, the line-of-sight vectors Ym12 and Ym22 intersect at the position of marker M2, and the line-of-sight vectors Ym13 and Ym23 intersect at the position of marker M3.

[0120] In this embodiment, the coordinate values ​​of three points where the line-of-sight vectors intersect in space are obtained. In other words, in this embodiment, the points where the line-of-sight vectors intersect in space are used as points in space for setting up a virtual wall. Then, in this embodiment, the plane containing the three points whose coordinate values ​​were obtained is defined as the virtual wall VM.

[0121] In this embodiment, markers are described as being placed at positions indicating points in space, but this is not the only option. In this embodiment, markers do not need to be installed; anything that can be seen by worker P is sufficient. Specifically, existing elements such as patterns on utility poles 51a and 51b may be used as substitutes for markers.

[0122] Furthermore, while the example in Figure 5 assumes that a utility pole is installed within the work environment, this is not the only limitation. In this embodiment, there does not need to be an object to which a marker is to be placed within the work environment. In such cases, for example, a drone or other flying object may be flown within the work environment and used as a substitute for a marker.

[0123] Thus, in this embodiment, a virtual wall can be set without placing markers within the work environment. Furthermore, in this embodiment, the worker P can set a virtual wall simply by visually observing a specific point within the work area, eliminating the need for equipment (such as road cones) to set the virtual wall.

[0124] Furthermore, in the example shown in Figure 5, worker P is assumed to view three points in space from two different locations, but the embodiment is not limited to this. In this embodiment, four or more points may be set in space. In this case, the surface identification unit 240 of the management device 200 identifies the surface that is closest to the points, and the setting instruction unit 250 sets the identified surface as a virtual wall.

[0125] Furthermore, in this embodiment, there may be two points in the space that the worker P can see. In this case, the support device 300 can identify the surface that becomes the virtual wall VM based on the coordinate values ​​of the two points and the direction of gravity.

[0126] Next, the operation of the work support system SYS of this embodiment will be described with reference to Figure 6. Figure 6 is a sequence diagram illustrating the operation of the work support system.

[0127] In the SYS work support system of this embodiment, the support device 300 transmits a request to the management device 200 to start the virtual wall setting (step S601). The request to start the setting may also be made by operating the input device of the support device 300.

[0128] Upon receiving a start request, the management device 200 transmits a display instruction to the support device 300 (step S602). Upon receiving the display instruction, the support device 300 displays the menu screen on the display device (step S603).

[0129] When the support device 300 receives a selection for virtual wall settings on the menu screen, it sends a virtual wall setting request to the management device 200 (step S604). Upon receiving this request, the management device 200 sends an instruction to the support device 300 to display the setting screen (step S605).

[0130] The support device 300 receives a display instruction and displays the virtual wall setting screen on the display device (step S606). Subsequently, the support device 300 accepts an operation to set the virtual wall (step S607).

[0131] Specifically, the support device 300 receives an operation to set multiple points via the input reception unit 310, and acquires the coordinate values ​​of each point via the coordinate value acquisition unit 330.

[0132] Next, the support device 300 transmits the acquired coordinate values ​​of each point to the management device 200 (step S608).

[0133] When the management device 200 acquires coordinate values, it stores the acquired coordinate values ​​using the coordinate value holding unit 230 (step S609). Subsequently, the management device 200 uses the surface identification unit 240 to identify the surface that will become the virtual wall based on the stored coordinate values ​​(step S610).

[0134] Next, the management device 200, via the display control unit 210, transmits a virtual wall display instruction to the support device 300 (step S611). The virtual wall display instruction may include information indicating the virtual wall.

[0135] The support device 300 receives this display instruction and, based on the information indicating the virtual wall, causes the display control unit 320 to display an image of the virtual wall on the display device (step S612). Alternatively, the image of the virtual wall may be generated in the management device 200 and image data indicating the image of the virtual wall may be transmitted to the support device 300.

[0136] Furthermore, the control device 200 transmits a virtual wall setting instruction to the shovel 100 via the setting instruction unit 250 (step S613). The virtual wall setting instruction may include information indicating the virtual wall.

[0137] When the excavator 100 receives a virtual wall setting instruction from the virtual wall setting unit 110, the operation control unit 120 sets the virtual wall based on information indicating the virtual wall (step S614). At this time, the excavator 100 may have the display control unit 130 display an image of the set virtual wall on the display device 40.

[0138] Next, the excavator 100, controlled by the motion control unit 120, starts motion control to stop its operation when the distance from the virtual wall to itself falls below a predetermined distance (step S615).

[0139] Next, the screen transitions in the display device of the support device 300 will be explained with reference to Figures 7 to 9. Figure 7 is the first diagram illustrating the screen transitions of the support device.

[0140] The screen 301 shown in Figure 7 is an example of a menu screen displayed on the display device of the support device 300 in step S603 of Figure 6.

[0141] Screen 301 displays a list 301a showing the types of work support. These types of work support include, for example, displaying existing virtual walls, setting up new virtual walls, and displaying maintenance information. In Screen 301, "Setting up new virtual walls" is selected.

[0142] Furthermore, if the support device 300 is AR glasses with a transparent display device, list 301a will be displayed as an AR image superimposed on the scenery seen by the worker P wearing the support device 300.

[0143] Furthermore, if the support device 300 is a typical smartphone, list 301a will be displayed on the smartphone's screen.

[0144] When "New Virtual Wall Setting" is selected on screen 301, and the support device 300 sends a setting request, screen 301 transitions to screen 302.

[0145] Screen 302 is an example of a settings screen displayed on the support device 300 in step S606 of Figure 6. Furthermore, screens 302 through 305, which will be described later, show examples of screens displayed on the support device 300 during the settings performed in step S607 of Figure 6.

[0146] Screen 302 displays a guide image 302a for setting a point in space, and a message 302b instructing worker P to place a marker within the guide image.

[0147] Screen 303 shows the state in which operator P is facing in a direction that causes marker M1 to enter (overlap with) the guide image 302a, in accordance with message 302b. In this embodiment, when marker M1 overlaps with the guide image 302a, operator P performs an operation to set the first eye point on the support device 300.

[0148] On screen 303, message 303a is displayed, indicating that the first point has been set. This state corresponds, for example, to the state in Figure 5 where the line-of-sight vector Ym11 from point E1 to marker M1 has been identified.

[0149] In this embodiment, similarly, worker P sets the second and third points at point E1, with markers M2 and M3 facing in a direction that overlaps with the guide image 302a.

[0150] In this case, if the support device 300 is AR glasses with a transparent display device, the guide image 302a and messages 302b and 303a are displayed on screens 302 and 303 as AR images superimposed on the scenery seen by the worker P wearing the support device 300. In other words, the guide image 302a and messages 302b and 303a are visible to the worker P superimposed on the actual scenery, namely the utility pole 51a and markers M1 and M3.

[0151] Furthermore, if the support device 300 is a typical smartphone, the images displayed on screens 302 and 303 will be images in which the landscape image captured by the imaging device of the support device 300 is superimposed with the AR images, namely the guide image 302a and messages 302b and 303a.

[0152] Figure 8 is a second diagram illustrating the screen transitions of the support device. Screen 304 shown in Figure 8 is an example of a screen displayed on the support device 300 after three points have been set at point E1. Screen 304 displays message 304a instructing the user to move to another measurement point.

[0153] Screen 305 shows the state at point E2 where worker P has performed the operation to set marker M2 as the third point, with marker M2 facing in a direction that overlaps with guide image 302a. Screen 305 displays message 305a indicating that the third point at point E2 has been set.

[0154] This state, for example in Figure 5, is one in which the line-of-sight vector Ym12 from point E2 to marker M1, the line-of-sight vector Ym22 from point E2 to marker M2, and the line-of-sight vector Ym32 from point E2 to marker M2 have been identified.

[0155] In this embodiment, when multiple points in space are set at multiple locations of different positions, the support device 300 uses the coordinate value acquisition unit 330 to determine the line-of-sight vector from each point to each point in space. The support device 300 then uses the coordinate value acquisition unit 330 to acquire the three-dimensional coordinates of the points where the line-of-sight vectors intersect.

[0156] Furthermore, if the support device 300 is AR glasses with a transparent display device, the guide image 302a and messages 304a and 305a are displayed on screens 304 and 305 as AR images superimposed on the scenery seen by the worker P wearing the support device 300. In other words, the guide image 302a and messages 304a and 305a are visible to the worker P superimposed on the actual scenery, namely the utility pole 51b and the marker M2.

[0157] Furthermore, if the support device 300 is a typical smartphone, the images displayed on screens 304 and 305 will be images in which the landscape image captured by the imaging device of the support device 300 is superimposed with the AR images, namely the guide image 302a and messages 304a and 305a.

[0158] Figure 9 is a third diagram illustrating the screen transitions of the support device. Screen 306 shown in Figure 9 is an example of a screen displayed on the support device 300 in step S612 of Figure 6. On screen 306, the surface identified based on the coordinate values ​​of three points obtained through the settings up to Figure 8 is displayed as a virtual wall VM. In other words, on screen 306, an image of the virtual wall VM is displayed based on information indicating the virtual wall.

[0159] In this embodiment, when the virtual wall VM is set, the directions that are restricted to the virtual wall VM may be set. Specifically, for example, in screen 306, the virtual wall VM may be set to prohibit entry in the direction indicated by arrow B1 and allow entry in the direction of arrow B2, or to prohibit entry in the direction indicated by arrow B2 and allow entry in the direction of arrow B1. This setting may be provided as a setting item in, for example, the menu screen shown in screen 301, or it may be performed by operator P when the image of the virtual wall VM is displayed, as shown in screen 306.

[0160] Furthermore, in this embodiment, if the worker P wearing the support device 300 moves after the image of the virtual wall VM is displayed, the image of the virtual wall VM is displayed in a shape that matches the viewpoint of the worker P.

[0161] Screen 307 in Figure 8 shows an example of a screen displayed on the support device 300 when worker P, wearing the support device 300, moves to the side of the utility pole 51b and faces the direction of the utility pole 51b (in the direction of arrow A shown in screen 306).

[0162] In screen 307, an image of the virtual wall VM is displayed to the left of the utility pole 51b. In this embodiment, by setting the virtual wall VM in this way, for example, if the excavator 100 approaches the virtual wall VM to a position where the distance to the virtual wall VM is within a predetermined distance, the operation of the excavator 100 can be stopped. Therefore, it is possible to prevent the excavator 100 from coming into contact with the utility pole 51b.

[0163] Furthermore, in this embodiment, after the image of the virtual wall VM is displayed based on multiple coordinate values, the system may receive an instruction to move the display position of the image of the virtual wall VM and move the display position of the image of the virtual wall VM.

[0164] Screen 308 shows the state in which the display position of the virtual wall VM image shown in Screen 307 has been moved to a location a predetermined distance away from the utility pole 51b.

[0165] In this embodiment, for example, with the screen 307 displayed on the support device 300, the support device 300 may receive an operation from the operator P to indicate the distance and direction to move the display position of the virtual wall VM image, and display the virtual wall VM image at the position corresponding to the instruction. In other words, with the screen 307 displayed, the support device 300 of this embodiment may receive an operation from the operator P to change the display position of the virtual wall VM image, and change the display position of the virtual wall VM image.

[0166] Changing the display position of the image of the virtual wall VM specifically means converting the coordinate values ​​included in the information indicating the virtual wall to the coordinate values ​​corresponding to the changed display position.

[0167] The operation to indicate the distance and direction to move the image of the virtual wall VM (the operation to change the position of the virtual wall) may be performed, for example, by voice data. In this case, the support device 300 may acquire voice data using a sound collection device, perform voice recognition, and move the image of the virtual wall VM according to the recognition result.

[0168] This process may be performed in the management device 200. Specifically, the management device 200 may receive voice data from the support device 300 and send instructions to the support device 300 based on the results of voice recognition performed on the received voice data. The support device 300 may move the image of the virtual wall VM in accordance with the instructions sent from the management device 200.

[0169] Thus, in this embodiment, after displaying the image of the configured virtual wall VM, the position and other properties of the displayed virtual wall image can be corrected.

[0170] Furthermore, if the support device 300 is AR glasses with a transparent display device, the image of the virtual wall VM is displayed on screens 306, 307, and 308 as an AR image superimposed on the scenery seen by the worker P wearing the support device 300. In other words, the image of the virtual wall VM is seen by the worker P superimposed on the actual scenery, which is the utility pole 51b.

[0171] Furthermore, if the support device 300 is a typical smartphone, the images displayed on screens 306, 307, and 308 will be superimposed images of a landscape captured by the imaging device of the support device 300 and an AR image of the virtual wall VM.

[0172] In this embodiment, a virtual wall VM can be set at a position away from the plane identified based on the coordinate values ​​of three points. By setting the virtual wall VM in this way, for example, it is possible to prevent a part of the attachment of the shovel 100 from coming into contact with an object such as a utility pole 51b, thereby improving safety.

[0173] Next, we will explain the case where the virtual wall is set to the shovel 100. Figure 10 is a diagram showing an example of the display for the shovel. The image display unit 41 shown in Figure 10 is an example of the screen displayed on the display device 40 of the shovel 100 in step S614 of Figure 6.

[0174] The display device 40 shown in Figure 10 has an image display unit 41 and an input device 42. The image display unit 41 is a screen on which various images are displayed. The input device 42 includes various menu switches.

[0175] First, the image display unit 41 will be described. As shown in Figure 10, the image display unit 41 includes a date and time display area 41a, a driving mode display area 41b, an attachment display area 41c, a fuel consumption display area 41d, an engine control status display area 41e, an engine operating time display area 41f, a coolant temperature display area 41g, a fuel level display area 41h, a rotation speed mode display area 41i, a urea solution level display area 41j, a hydraulic oil temperature display area 41k, an air conditioner operation status display area 41m, an image display area 41n, and a menu display area 41p.

[0176] The driving mode display area 41b, attachment display area 41c, engine control status display area 41e, rotation speed mode display area 41i, and air conditioner operation status display area 41m are areas that display setting operation information, which is information related to the setting status of the shovel 100. The fuel consumption display area 41d, engine operating time display area 41f, coolant temperature display area 41g, fuel level display area 41h, urea solution level display area 41j, and hydraulic oil temperature display area 41k are areas that display operation information, which is information related to the operating status of the shovel 100.

[0177] Specifically, the date and time display area 41a is an area that displays the current date and time. The driving mode display area 41b is an area that displays the current driving mode. The attachment display area 41c is an area that displays an image representing the attachment currently installed. The fuel consumption display area 41d is an area that displays fuel consumption information calculated by the controller 30. The fuel consumption display area 41d includes an average fuel consumption display area 41d1 that displays lifetime average fuel consumption or section average fuel consumption, and an instantaneous fuel consumption display area 41d2 that displays instantaneous fuel consumption.

[0178] The engine control status display area 41e is an area that displays the control status of the engine 11. The engine operating time display area 41f is an area that displays the cumulative operating time of the engine 11. The coolant temperature display area 41g is an area that displays the current temperature status of the engine coolant. The fuel level display area 41h is an area that displays the remaining amount of fuel stored in the fuel tank.

[0179] The rotation speed mode display area 41i is an area that displays the current rotation speed mode set by the engine speed adjustment dial 75 as an image. The urea solution remaining amount display area 41j is an area that displays the remaining amount of urea solution stored in the urea solution tank as an image. The hydraulic oil temperature display area 41k is an area that displays the temperature of the hydraulic oil in the hydraulic oil tank.

[0180] The air conditioner operating status display area 41m includes an outlet display area 41m1 that displays the current outlet position, an operating mode display area 41m2 that displays the current operating mode, a temperature display area 41m3 that displays the current set temperature, and an airflow display area 41m4 that displays the current set airflow.

[0181] The image display area 41n is the area that displays the image captured by the imaging device S6. In the example in Figure 6, the image display area 41n displays image FV1, which includes the image of the configured virtual wall VM, and the rear image CBT.

[0182] Image FV1 is, for example, an image obtained by superimposing an image of a virtual wall VM, which is generated based on information indicating a virtual wall, onto an image captured by the front camera S6F. The image of the virtual wall VM is, for example, an image generated as an AR image by the display control unit 130 based on information indicating a virtual wall, position information indicating the current position of the shovel 100, and the orientation of the shovel 100.

[0183] Furthermore, the image display area 41n has a first image display area 41n1 located above and a second image display area 41n2 located below. In the example of Figure 10, image FV1, which includes the image of the virtual wall VM, is placed in the first image display area 41n1, and the rear image CBT is placed in the second image display area 41n2. However, the image display area 41n may also have image FV1, which includes the image of the virtual wall VM, placed in the second image display area 41n2, and the rear image CBT placed in the first image display area 41n1.

[0184] Furthermore, in the image display area 41n, the image displayed in the first image display area 41n1 can be switched from image FV1, which includes the image of the virtual wall VM, to an overhead view image, and from the overhead view image back to image FV1, which includes the image of the virtual wall VM. The switching operation may be performed using any switch on the input device 42.

[0185] The overhead view image in this embodiment is, for example, a virtual viewpoint image generated by the display control unit 130, and is generated based on images acquired by the rear camera S6B, the left camera S6L, and the right camera S6R, respectively.

[0186] Furthermore, in the example of Figure 10, the image FV1 or overhead image containing the image of the virtual wall VM and the rear view image CBT are arranged adjacent to each other vertically, but they may be arranged with a gap between them. Also, in the example of Figure 10, the image display area 41n is a vertically elongated area, but the image display area 41n may be a horizontally elongated area.

[0187] If the image display area 41n is a horizontally elongated area, the image display area 41n may have an image FV1 containing the image of the virtual wall VM or an overhead image placed on the left side as the first image display area 41n1, and a rear view image CBT placed on the right side as the second image display area 41n2. In this case, there may be a gap between the left and right sides, or the positions of the image FV1 containing the image of the virtual wall VM or the overhead image and the rear view image CBT may be swapped.

[0188] Furthermore, in this embodiment, icon images 41x are displayed in both the first image display area 41n1 and the second image display area 41n2. The icon image 41x is an image that represents the relative relationship between the position of the imaging device S6 and the orientation of the attachment of the upper rotating body 3.

[0189] The icon image 41x of this embodiment includes an image 41xM of the shovel 100, an image 41xF showing the front of the shovel 100, and an image 41xB showing the rear of the shovel 100. The icon image 41x also includes an image 41xL showing the left side of the shovel 100, an image 41xR showing the right side of the shovel 100, and an image 41xI showing the inside of the cabin 10.

[0190] Images 41xF, 41xB, 41xL, 41xR, and 41xI correspond to the front camera S6F, which images the area in front of the shovel 100; the rear camera S6B, which images the area behind the shovel 100; the left camera S6L, which images the area to the left of the shovel 100; and the right camera S6R, which images the area to the right of the shovel 100, respectively. Image 41xI also corresponds to the camera inside the cabin 10.

[0191] In this embodiment, when an image associated with each camera is selected in the icon image 41x, the image data captured by the camera corresponding to the selected image is displayed in the image display area 41n.

[0192] In the example shown in Figure 10, the display mode of image 41xF in the first image display area 41n1 is different from the display modes of images 41xB, 41xL, 41xR, and 41xI. Therefore, it can be seen that the first image display area 41n1 displays image FV1, which is based on image data captured by the front camera S6F corresponding to image 41xF.

[0193] Furthermore, in the second image display area 41n2, the display mode of image 41xB is different from the display modes of images 41xF, 41xL, 41xR, and 41xI. Therefore, it can be seen that the second image display area 41n2 displays the image represented by the image data captured by the rear camera S6B, which corresponds to image 41xB.

[0194] The menu display area 41p has tabs 41p1 to 41p7. In the example in Figure 10, tabs 41p1 to 41p7 are arranged horizontally at the bottom of the image display section 41, with space between them. Tabs 41p1 to 41p7 display icon images for displaying various information.

[0195] Tab 41p1 displays icon images for menu detail items. When tab 41p1 is selected by the operator, the icon images displayed in tabs 41p2 to 41p7 switch to the icon images associated with the menu detail items.

[0196] Tab 41p4 displays an icon image for displaying information about the digital level. When the operator selects tab 41p4, the rear-view CBT switches to a screen displaying information about the digital level. However, the screen displaying information about the digital level may be superimposed on the rear-view CBT or the rear-view CBT may be reduced in size.

[0197] Furthermore, in this embodiment, when an overhead view image is displayed, the overhead view image may switch to a screen showing information about the digital level, or the screen showing information about the digital level may be superimposed on the overhead view image, or the overhead view image may be reduced in size to display the screen showing information about the digital level.

[0198] Tab 41p5 displays an icon image for transitioning the main screen displayed on the image display unit 41 to the loading operation screen. When the operator selects the input device 42 corresponding to tab 41p5 (described later), the main screen displayed on the image display unit 41 transitions to the loading operation screen. At this time, the image display area 41n continues to be displayed, and the menu display area 41p switches to an area that displays information related to the loading operation.

[0199] Tab 41p6 displays icon images for displaying information related to information-based construction. When tab 41p6 is selected by the operator, the rear view CBT switches to a screen displaying information related to information-based construction. However, the screen displaying information related to information-based construction may be displayed by superimposing it on the rear view CBT or by shrinking the rear view CBT. Alternatively, image FV1, which includes an overhead view image or an image of a virtual wall VM, may switch to a screen displaying information related to information-based construction, or the screen displaying information related to a digital level may be displayed by superimposing it on image FV1 including an image of a virtual wall VM or the overhead view image, or by shrinking image FV1 including an image of a virtual wall VM or the overhead view image.

[0200] Tab 41p7 displays an icon image for displaying information about the crane mode. When the operator selects tab 41p7, the rear view CBT switches to a screen displaying information about the crane mode. However, the screen displaying information about the crane mode may be superimposed on the rear view CBT or the rear view CBT may be reduced in size. Alternatively, image FV1 containing the image of the virtual wall VM or the overhead view image may switch to a screen displaying information about the crane mode, or the screen displaying information about the crane mode may be superimposed on image FV1 containing the image of the virtual wall VM or the overhead view image, or image FV1 containing the image of the virtual wall VM or the overhead view image may be reduced in size.

[0201] No icon images are displayed on tabs 41p2 and 41p3. Therefore, even if tabs 41p2 and 41p3 are manipulated by the operator, the image displayed on the image display unit 41 will not change.

[0202] Note that the icon images displayed in tabs 41p1 to 41p7 are not limited to the examples described above; other icon images for displaying information may also be displayed.

[0203] Next, the input device 42 will be described. As shown in Figure 10, the input device 42 consists of one or more button-type switches on which the operator selects tabs 41p1 to 41p7, inputs settings, etc.

[0204] In the example shown in Figure 10, the input device 42 includes seven switches 42a1 to 42a7 arranged in the upper row and seven switches 42b1 to 42b7 arranged in the lower row. The switches 42b1 to 42b7 are located below each of the switches 42a1 to 42a7.

[0205] However, the number, form, and arrangement of switches in the input device 42 are not limited to the examples described above. For example, it may be a configuration in which the functions of multiple button-type switches are combined into one using a jog wheel, jog switch, etc., or the input device 42 may be separate from the display device 40. Alternatively, the image display unit 41 and the input device 42 may be integrated into a touch panel, allowing direct operation of tabs 41p1 to 41p7.

[0206] Switches 42a1 to 42a7 are located below tabs 41p1 to 41p7, corresponding to tabs 41p1 to 41p7 respectively, and function as switches to select tabs 41p1 to 41p7.

[0207] Since switches 42a1 to 42a7 are positioned below tabs 41p1 to 41p7, corresponding to tabs 41p1 to 41p7 respectively, operators can intuitively select tabs 41p1 to 41p7.

[0208] In Figure 10, for example, when switch 42a1 is operated, tab 41p1 is selected, the menu display area 41p changes from a single-line display to a two-line display, and icon images corresponding to the first menu are displayed on tabs 41p2 to 41p7. In addition, corresponding to the change from a single-line display to a two-line display in the menu display area 41p, the size of the rear view image CBT is reduced. At this time, the size of image FV1, which includes the image of the virtual wall VM, or the overhead view image is maintained without change, so the visibility for the operator when checking the area around the shovel 100 is not deteriorated.

[0209] Furthermore, when the switch 42a5 is operated, the display control unit 130 determines that tab 41p5 is selected and transitions the display on the image display unit 41 to the loading work screen.

[0210] Specifically, when the switch 42a5 is operated, the display control unit 130 maintains the image display area 41n while converting the menu display area 41p into a work information display area that displays information related to loading operations.

[0211] Thus, in this embodiment, the captured image is continuously displayed in the image display area 41n even on the loading operation screen, so the visibility for the operator when checking the area around the shovel 100 is not impaired.

[0212] Switch 42b1 is a switch for switching the captured image displayed in the image display area 41n. Each time switch 42b1 is operated, the captured image displayed in the first image display area 41n1 of the image display area 41n is configured to switch between, for example, a rear view image, a left view image, a right view image, and an overhead view image.

[0213] Furthermore, the system may be configured such that each time the switch 42b1 is operated, the captured image displayed in the second image display area 41n2 of the image display area 41n switches between, for example, a rear view image, a left view image, a right view image, and an overhead view image.

[0214] Furthermore, the display control unit 130 may change the display mode of images 41xF, 41xB, 41xL, 41xR, and 41xI in the icon image 41x in response to the operation of the switch 42b1.

[0215] Furthermore, the system may be configured such that each time the switch 42b1 is operated, the captured image displayed in the first image display area 41n1 of the image display area 41n and the captured image displayed in the second image display area 41n2 are swapped.

[0216] Thus, the switch 42b1, which functions as an input device 42, may switch between the screens displayed in the first image display area 41n1 or the second image display area 41n2, or it may switch between the screens displayed in the first image display area 41n1 and the second image display area 41n2. Alternatively, a separate switch may be provided for switching the screen displayed in the second image display area 41n2.

[0217] Switches 42b2 and 42b3 are switches that adjust the airflow of the air conditioner. In the example in Figure 10, when switch 42b2 is operated, the airflow of the air conditioner decreases, and when switch 42b3 is operated, the airflow of the air conditioner increases.

[0218] Switch 42b4 is a switch that turns the cooling and heating functions ON and OFF. In the example in Figure 10, the system is configured so that the cooling and heating functions are switched ON and OFF each time switch 42b4 is operated.

[0219] Switches 42b5 and 42b6 are switches that adjust the set temperature of the air conditioner. In the example in Figure 10, when switch 42b5 is operated, the set temperature is lowered, and when switch 42b6 is operated, the set temperature is raised.

[0220] Switch 42b7 is a switch that can toggle the display of the engine operating time display area 41f.

[0221] Furthermore, switches 42a2-42a6 and 42b2-42b6 are configured to allow input of the numbers displayed on or near the respective switches. Additionally, switches 42a3, 42a4, 42a5, and 42b4 are configured to allow movement of the cursor to the left, up, right, and down, respectively, when the cursor is displayed on the menu screen.

[0222] Note that the functions assigned to switches 42a1-42a7 and 42b1-42b7 are examples only, and they may be configured to perform other functions.

[0223] Thus, in this embodiment, when tab 41p1 is selected while image FV1 (or overhead image) and rear view image CBT, which include the image of the virtual wall VM, are displayed in the image display area 41n, the first menu detail items are displayed in tabs 41p2 to 41p7 while these images are displayed. Therefore, the operator can check the first menu detail items while checking image FV1 (or overhead image) and rear view image CBT, which include the image of the virtual wall VM.

[0224] Furthermore, the image display area 41n displays image FV1, which includes the image of the virtual wall VM, or an overhead image, without changing its size before or after tab 41p1 is selected. This ensures that the operator's visibility when checking the area around the shovel 100 is not impaired.

[0225] As described above, in this embodiment, by displaying image FV1, which includes an image of the virtual wall VM, on the display device 40 of the excavator 100, the operator of the excavator 100 can be made aware that the virtual wall VM has been set.

[0226] Furthermore, in this embodiment, for example, if the shovel 100 approaches the virtual wall VM and the operation of the shovel 100 is stopped, a notification indicating that the shovel 100 has approached the virtual wall VM may be displayed. At this time, the display device 40 may also display an image of the virtual wall VM.

[0227] In this embodiment, by notifying the operator of the shovel 100 of its approach to the virtual wall, the operator can understand the reason why the shovel 100 stopped working.

[0228] Thus, in this embodiment, an AR-enabled support device 300 is used to allow worker P to set multiple arbitrary points in space, and based on these multiple points, a virtual wall is set, which is a surface that the shovel 100 is prohibited from crossing.

[0229] Therefore, according to this embodiment, prior measurements and equipment installation for setting a virtual wall are unnecessary. Furthermore, in this embodiment, a worker P wearing the support device 300 can set any point in space as long as it is visible, thereby setting a virtual wall. Therefore, in this embodiment, a virtual wall can be easily set even in places where it is difficult to set up equipment such as road cones, or in places where it is difficult for worker P to enter.

[0230] Furthermore, in this embodiment, there is no need to permanently install road cones or the like to set up virtual walls; only temporary markers or the like are required. In addition, in this embodiment, existing objects can be used as landmarks instead of markers to set up virtual walls. Therefore, according to this embodiment, it is possible to eliminate the need to place objects in the work environment solely for the purpose of setting up virtual walls, thereby improving the work environment.

[0231] Furthermore, in this embodiment, the process of setting up a virtual wall and the operation of the shovel 100 can be performed independently. Therefore, according to this embodiment, a virtual wall can be set up in the work environment before performing work with the shovel 100.

[0232] Furthermore, in the work support system SYS of this embodiment, the coordinate values ​​of multiple points acquired by the support device 300 are transmitted to the management device 200, and a virtual wall is set on the shovel 100 via the management device 200, but the system is not limited to this.

[0233] In this embodiment, the support device 300 may have the functions of the management device 200. In this case, when the support device 300 obtains the coordinate values ​​of multiple points in space, it can identify a surface based on these coordinate values ​​as a virtual wall and issue an instruction to the shovel 100 to set the virtual wall. In other words, the support device 300 in this embodiment may have the surface identification unit 240 and the setting instruction unit 250 that the management device 200 has.

[0234] Furthermore, in this embodiment, the shovel 100 may also have the functions of the control device 200. In this case, when the support device 300 acquires the coordinate values ​​of multiple points in space, it transmits the acquired coordinate values ​​to the shovel 100. When the shovel 100 receives the coordinate values ​​of multiple points in space from the support device 300, it identifies a surface based on this coordinate information and sets the identified surface as a virtual wall for itself. In other words, the shovel 100 in this embodiment may have the surface identification unit 240 and the setting instruction unit 250 of the control device 200.

[0235] In this way, by enabling direct communication between the support device 300 and the shovel 100, the communication load can be reduced. Furthermore, a virtual wall can be easily set up even in areas where it is difficult to connect to the internet, for example.

[0236] As described above, according to this embodiment, virtual walls can be set freely and easily.

[0237] In this embodiment, a shovel 100 is used as an example of construction machinery, but this embodiment can also be applied to construction machinery other than a shovel 100. Specifically, for example, this embodiment can also be applied to construction machinery such as cranes.

[0238] The embodiments for carrying out the present invention have been described above, but the above description does not limit the scope of the invention, and various modifications and improvements are possible within the scope of the present invention. [Explanation of symbols]

[0239] 1. Lower running body 2. Swivel mechanism 3. Upper rotating body 30 controllers 40 Display device 100 Shovel 110 Virtual Wall Setting Section 120 Operation Control Unit 130 Display Control Unit 200 Management device 230 Coordinate value holding unit 240 surface identification part 250 Setting instruction unit 300 Support equipment 320 Display Control Unit 330 Coordinate value acquisition unit

Claims

1. A construction machine work support system including construction machinery, a management device, and a support device, The aforementioned control device is A construction machine work support system comprising: a support device that displays an AR image guiding the setting of a new virtual wall; and a setting instruction unit that causes the support device to set a surface identified based on the coordinate values ​​of a plurality of points in space set according to the AR image, which has been acquired by the support device, as a virtual wall for the construction machine.

2. A construction machine and a work support system for a construction machine, including support devices, The aforementioned support device is A work support system for construction machinery, comprising a setting instruction unit that displays an AR image guiding the setting of a new virtual wall, and instructs the construction machinery to set a surface identified based on the coordinate values ​​of a plurality of points in space set according to the AR image, acquired by the support device, as a virtual wall.

3. The aforementioned construction machine, A construction machine work support system according to claim 1 or 2, comprising an action control unit that restricts the movement of the construction machine based on the positional relationship between the virtual wall and the construction machine.

4. The coordinate values ​​of the multiple points in the aforementioned space are: A work support system for a construction machine according to claim 3, wherein the system is set based on position information indicating the position of the support device and the orientation of the support device when the support device receives an operation indicating that a guide image for setting a plurality of points in the space, displayed on the support device, and each of the plurality of markers corresponding to the plurality of points in the space, are overlapping.

5. The support device is carried by a worker in the vicinity of the construction machine. A work support system for a construction machine according to claim 3 or 4, which displays an image of the identified surface as an image of the virtual wall.

6. The aforementioned support device is The construction machine work support system according to claim 5, which, after the image of the virtual wall is displayed, accepts an operation to change the position of the virtual wall and changes the position of the image of the virtual wall.

7. The aforementioned construction machine, A work support system for construction machinery according to any one of claims 3 to 5, comprising an imaging device, which superimposes an image captured by the imaging device and an image of the set virtual wall onto a display device.

8. The work support system for construction machinery according to claim 1 or 2, wherein the point is the point where the direction the support device is facing at the first point intersects with the direction the support device is facing at the second point, among a plurality of points in different locations.

Images