Systems for Excavators

Abstract

To provide a system for a shovel capable of further appropriately supporting loading work.SOLUTION: A construction support system SYS is the system for a shovel 100 having a lower traveling body 1, an upper turning body 3 turnably loaded on the lower traveling body 1 and excavation attachments attached to the upper turning body 3. The system SYS is configured to acquire three-dimensional data related to a dump truck 200 on the basis of the output of a space recognition device S6 as an object detection device, and to estimate at least one of the position, direction and size of the dump truck 200, on the basis of information related to at least one face composing a platform of the dump truck 200.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present disclosure relates to a system for an excavator. [Background technology]

[0002] 2. Description of the Related Art Conventionally, there is known a shovel that recognizes the position of a rear corner of a loading platform of a dump truck in order to prevent contact between the digging attachment of the shovel and the dump truck (see Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2012-21290 A Summary of the Invention [Problem to be solved by the invention]

[0004] However, the above-mentioned excavator only locally recognizes the position of the rear corner of the bed of the dump truck, and does not recognize the size of the bed of the dump truck, so it cannot properly assist the loading operation.

[0005] It would therefore be desirable to provide a system for an excavator that can better assist in loading operations. [Means for solving the problem]

[0006] A system for a shovel according to an embodiment of the present invention is a system for a shovel having a lower traveling body, an upper rotating body rotatably mounted on the lower traveling body, and an attachment attached to the upper rotating body, the system acquiring point cloud data related to a dump truck based on an output of an object detection device, and calculating a distance from the point cloud data related to the dump truck. A single point of interest is randomly extracted, and a principal component analysis is performed on local point cloud data around the point of interest. A normal line at the point of interest and a plane perpendicular to the normal line and including the point of interest are detected as planes constituting the bed of the dump truck. Thus, a point of interest is extracted from the point cloud data relating to the dump truck that belongs to the plane constituting the bed of the dump truck. At least one of a position, an orientation, and a size of the dump truck is estimated. Effect of the Invention

[0007] The above-mentioned system for a shovel can better assist in loading operations. [Brief description of the drawings]

[0008] [Figure 1] FIG. 1 is a diagram illustrating an example of the configuration of a construction support system. [Diagram 2] FIG. [Diagram 3] FIG. 2 is a diagram illustrating a configuration example of a drive system mounted on a shovel. [Figure 4] 13 is a flowchart illustrating an example of an estimation process. [Diagram 5] FIG. 11 is a diagram illustrating an example of data used in the estimation process. [Figure 6] FIG. 2 is a top view of a shovel and a dump truck at a construction site. [Figure 7] 13 is a flowchart showing another example of the estimation process. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, an embodiment of the present invention will be described with reference to the drawings. In each drawing, the same or corresponding components are denoted by the same or corresponding reference numerals, and the description thereof will be omitted.

[0010] Fig. 1 is a diagram showing an example of the configuration of a construction support system SYS, which is an example of a system for a shovel according to an embodiment of the present invention. The construction support system SYS is configured to recognize the position, size, orientation, etc. of a dump truck 200 as a transport vehicle, and thus to support loading work by an excavator (shovel 100) as a construction machine. Fig. 2 is a top view of the shovel 100. Fig. 3 is a diagram showing an example of the configuration of a drive system mounted on the shovel 100. The drive system is a system for driving the shovel 100.

[0011] First, an overview of an excavator 100 will be described with reference to Figures 1 and 2. The excavator 100 includes a lower traveling body 1, a slewing mechanism 2, an upper rotating body 3, a boom 4, an arm 5, a bucket 6, and a cabin 10. The upper rotating body 3 is mounted on the lower traveling body 1 so as to be rotatable via the slewing mechanism 2. The boom 4, the arm 5, and the bucket 6 configure an excavation attachment, which is an example of an attachment. The attachment may be a lifting magnet attachment, a grapple attachment, or the like.

[0012] 2, the lower traveling body 1 has a crawler 1C including a left crawler 1CL and a right crawler 1CR. The crawler 1C is driven by a traveling hydraulic motor 2M. The traveling hydraulic motor 2M includes a left traveling hydraulic motor 2ML that drives the left crawler 1CL, and a right traveling hydraulic motor 2MR that drives the right crawler 1CR.

[0013] The upper rotating body 3 is driven by a swing hydraulic motor 2A. The swing hydraulic motor 2A can change the orientation of the upper rotating body 3 relative to the lower traveling body 1. The upper rotating body 3 may be electrically driven by a swing electric motor. The swing hydraulic motor 2A and the swing electric motor may be used in combination.

[0014] The boom 4 is rotatably attached to the front center of the upper rotating body 3. An arm 5 is rotatably attached to the tip of the boom 4, and a bucket 6 is rotatably attached to the tip of the arm 5 as an end attachment. The boom 4 is driven by a boom cylinder 7. The arm 5 is driven by an arm cylinder 8. The bucket 6 is driven by a bucket cylinder 9.

[0015] Depending on the work content, etc., another end attachment may be attached to the tip of the arm 5 instead of the bucket 6, which is an example of an end attachment. The other end attachment is, for example, a lifting magnet or a grapple.

[0016] The cabin 10 is a cab in which an operator sits. The cabin 10 is mounted on the front left side of the upper rotating body 3. When the excavator 100 is an unmanned excavator, the cabin 10 may be omitted.

[0017] Next, the functions of the excavator 100 will be described. As shown in Fig. 3, the drive system of the excavator 100 includes an engine 11, a regulator 13, a main pump 14, a control valve unit 17, etc. As shown in Fig. 2, the hydraulic drive system of the excavator 100 includes hydraulic actuators such as a traveling hydraulic motor 2M, a swing hydraulic motor 2A, a boom cylinder 7, an arm cylinder 8, and a bucket cylinder 9.

[0018] The engine 11 is a power source of the hydraulic drive system, and is mounted, for example, on the rear of the upper rotating body 3. Specifically, the engine 11 rotates at a preset target rotation speed under direct or indirect control by a shovel controller 30 (described later), and drives the main pump 14 and the pilot pump 15. The engine 11 is, for example, a diesel engine that uses light oil as fuel.

[0019] A dial 75 is provided in the cabin 10 of the excavator 100. The dial 75 is a dial for adjusting the engine speed, and is used, for example, to switch the engine speed in stages. In this embodiment, the dial 75 is provided so as to be able to switch the engine speed between four stages: SP mode, H mode, A mode, and IDLE mode. The dial 75 is configured to send data indicating the setting state of the engine speed to the excavator controller 30. Note that FIG. 3 shows a state in which the H mode is selected by the dial 75.

[0020] The SP mode is a rotation speed mode selected when priority is given to the amount of work, and uses the highest engine rotation speed. The H mode is a rotation speed mode selected when priority is given to both the amount of work and fuel efficiency, and uses the second highest engine rotation speed. The A mode is a rotation speed mode selected when priority is given to fuel efficiency while operating the excavator with low noise, and uses the third highest engine rotation speed. The IDLE mode is a rotation speed mode selected when the engine is to be put into an idling state, and uses the lowest engine rotation speed. The engine 11 is controlled to a constant engine rotation speed corresponding to the rotation speed mode set by the dial 75.

[0021] The main pump 14 is configured to discharge hydraulic oil. The main pump 14 is mounted on the rear of the upper rotating body 3, similar to the engine 11, and supplies hydraulic oil to the control valve unit 17 through a hydraulic oil line. The main pump 14 is driven by the engine 11. In this embodiment, the main pump 14 is a swash plate type variable displacement hydraulic pump, and the discharge flow rate (displacement volume per rotation) is controlled under the control of the excavator controller 30.

[0022] The regulator 13 is configured to control the discharge amount of the main pump 14. In this embodiment, the regulator 13 adjusts the angle (tilt angle) of the swash plate of the main pump 14 in response to a control command from the shovel controller 30.

[0023] The oil temperature sensor 14c, which is provided in the pipeline between the main pump 14 and a tank storing the hydraulic oil to be sucked in by the main pump 14, is configured to send data representing the temperature of the hydraulic oil flowing through the pipeline to the shovel controller 30.

[0024] The control valve unit 17 is a hydraulic control device including a plurality of control valves. The control valve unit 17 is mounted, for example, in the center of the upper rotating body 3, and is configured to control the hydraulic drive system in response to an operation of the operating device 26 by an operator. The control valve unit 17 is connected to the main pump 14 via a hydraulic oil line, and selectively supplies hydraulic oil supplied from the main pump 14 to one or more of the plurality of hydraulic actuators (the traveling hydraulic motor 2M, the swing hydraulic motor 2A, the boom cylinder 7, the arm cylinder 8, and the bucket cylinder 9) in response to the operating state of the operating device 26. Specifically, the control valve unit 17 includes a plurality of control valves (spool valves) that control the flow rate and flow direction of the hydraulic oil supplied from the main pump 14 to each of the plurality of hydraulic actuators.

[0025] The operation system of the shovel 100 includes the pilot pump 15 and an operation device 26. The operation system of the shovel 100 also includes a shuttle valve 32 as a component related to a machine control function (described later) by the shovel controller 30.

[0026] The pilot pump 15 is mounted, for example, on the rear of the upper rotating body 3 and is configured to be able to supply hydraulic oil to devices such as the operating device 26 via a pilot line. In this embodiment, the pilot pump 15 is a fixed displacement hydraulic pump and is driven by the engine 11.

[0027] The operating device 26 is provided near the cockpit in the cabin 10. The operating device 26 is an operation input means used to operate the operated parts (the lower traveling body 1, the upper rotating body 3, the boom 4, the arm 5, the bucket 6, etc.). That is, the operating device 26 is an operation input means used to operate the hydraulic actuators (the left traveling hydraulic motor 2ML, the right traveling hydraulic motor 2MR, the swing hydraulic motor 2A, the boom cylinder 7, the arm cylinder 8, the bucket cylinder 9, etc.) that drive the operated parts. The operating device 26 includes an operating lever and a traveling lever. Each of the operating devices 26 is connected to a corresponding control valve in the control valve unit 17 via a shuttle valve 32 provided in a secondary pilot line. As a result, a pilot pressure according to the operating state of the corresponding operating device 26 is input to each control valve. Therefore, each control valve operates according to the operating state of the corresponding operating device 26, and the hydraulic actuator is driven according to the operation of the corresponding control valve. The operating lever includes a left operating lever and a right operating lever. The left operating lever is used, for example, to operate the swing hydraulic motor 2A and the arm cylinder 8, and the right operating lever is used, for example, to operate the boom cylinder 7 and the bucket cylinder 9. The travel levers include a left travel lever and a right travel lever. The left travel lever is used, for example, to operate the left travel hydraulic motor 2ML, and the right travel lever is used, for example, to operate the right travel hydraulic motor 2MR. The left travel lever may be configured to operate in conjunction with a left travel pedal. The same applies to the right travel lever.

[0028] The operation device 26 is provided with a switch NS. In this embodiment, the switch NS is a push button switch. An operator can operate the operation device with his / her hand while pressing the switch NS with his / her finger. The switch NS is used, for example, when executing a machine control function.

[0029] The shuttle valve 32 has two inlet ports and one outlet port, and outputs (flows out) the hydraulic oil having a higher pressure from the outlet port out of the hydraulic oils inputted (flowing in) into each of the two inlet ports. One of the two inlet ports of the shuttle valve 32 is connected to the operating device 26, and the other is connected to the proportional valve 31. The outlet port of the shuttle valve 32 is connected to the pilot port of the corresponding control valve in the control valve unit 17 through a pilot line. Therefore, the shuttle valve 32 can apply the higher of the pilot pressure generated by the operating device 26 and the pilot pressure generated by the proportional valve 31 to the pilot port of the corresponding control valve. In other words, the shovel controller 30 can control the corresponding control valve to operate the corresponding operated part regardless of the operation of the operating device 26 by the operator by applying the pilot pressure generated by the proportional valve 31, which is higher than the pilot pressure generated by the operating device 26, to the pilot port of the corresponding control valve. In other words, the shovel controller 30 can operate the shovel 100 autonomously.

[0030] The operating device 26 may be an electric type that outputs an electric signal, instead of a hydraulic pilot type that generates a pilot pressure. In this case, the electric signal from the operating device 26 is input to the shovel controller 30. The shovel controller 30 can operate the hydraulic actuator according to the operation content on the operating device 26 by controlling the control valve in the control valve unit 17 according to the input electric signal. The control valve in the control valve unit 17 may be an electromagnetic solenoid type spool valve that is driven by a command from the shovel controller 30. Alternatively, a solenoid valve that operates according to an electric signal from the shovel controller 30 may be disposed between the pilot pump 15 and each pilot port of the control valve in the control valve unit 17. In this case, when a manual operation is performed using the electric operating device 26, the shovel controller 30 can operate the corresponding solenoid valve by an electric signal corresponding to the operation amount (e.g., lever operation amount). Specifically, the shovel controller 30 can operate the corresponding control valve according to the operation amount of the operating device 26 by increasing or decreasing the pilot pressure generated by using the hydraulic oil discharged by the pilot pump 15 with the solenoid valve.

[0031] When the shovel 100 is an unmanned shovel, the operation device 26 may be omitted. Alternatively, when the shovel 100 is a remote-controlled shovel, the operation device 26 may be installed outside the shovel 100 (cabin 10).

[0032] In this embodiment, the control system of the shovel 100 includes a discharge pressure sensor 28, an operating pressure sensor 29, a shovel controller 30, a proportional valve 31, a display device 40, an input device 42, a sound output device 43, a memory device 47, a boom angle sensor S1, an arm angle sensor S2, a bucket angle sensor S3, a machine body inclination sensor S4, a turning state sensor S5, a spatial recognition device S6, a positioning device P1, and a communication device T1.

[0033] The shovel controller 30, which is an example of a control device, is provided, for example, in the cabin 10 and configured to control the drive of the shovel 100. The shovel controller 30 may be realized by hardware, software, or a combination thereof. In this embodiment, the shovel controller 30 is configured by a computer including a CPU, a volatile storage device, a non-volatile storage device, and various input / output interfaces, etc. The shovel controller 30 realizes various functions by executing various programs stored in the non-volatile storage device.

[0034] For example, the shovel controller 30 sets a target rotation speed of the engine 11 based on a work mode or the like that is set in advance by a predetermined operation by an operator or the like, and controls the engine 11 to rotate at a constant speed. Also, for example, the shovel controller 30 outputs a control command to the regulator 13 as necessary, and changes the discharge amount of the main pump 14. Also, for example, the shovel controller 30 may execute a machine guidance function that guides (guides) the manual operation of the shovel 100 by the operator through the operation device 26. Also, the shovel controller 30 may execute a machine control function that automatically assists the manual operation of the shovel 100 by the operator through the operation device 26, or operates the shovel 100 independently of the manual operation. Alternatively, the machine control function may be a function for operating an unmanned shovel.

[0035] Note that some of the functions of the shovel controller 30 may be realized by other controllers (control devices). That is, the above-mentioned functions may be realized in a distributed processing manner by a plurality of controllers. For example, the machine guidance function and the machine control function may be realized by a dedicated controller (control device).

[0036] In this embodiment, the shovel controller 30 is configured to estimate at least one of the position, orientation, and size (hereinafter, referred to as "position, etc.") of the dump truck 200. The estimation of the position, etc. of the dump truck 200 by the shovel controller 30 will be described in detail later.

[0037] The discharge pressure sensor 28 is configured to detect the discharge pressure of the main pump 14. The discharge pressure sensor 28 transmits a detection signal related to the discharge pressure to the shovel controller 30.

[0038] The operating pressure sensor 29, which is an example of an operation state detection device, is configured to detect the operation state of the operating device 26. In this embodiment, the operating pressure sensor 29 detects the pilot pressure generated by the operating device 26. The operating pressure sensor 29 transmits a detection signal related to the operating pressure to the shovel controller 30.

[0039] The operation state detection device may be an encoder, a potentiometer, a tilt sensor, or the like, capable of detecting the tilt direction, tilt angle, etc. of the operating lever.

[0040] The proportional valve 31 is provided in a pilot line connecting the pilot pump 15 and the shuttle valve 32, and is configured so that its flow path area (cross-sectional area through which hydraulic oil can flow) can be changed. The proportional valve 31 operates in response to a control command input from the shovel controller 30. With this configuration, even when the operation lever is not being operated, for example, the shovel controller 30 can supply the hydraulic oil discharged by the pilot pump 15 to the pilot port of the corresponding control valve in the control valve unit 17 via the proportional valve 31 and the shuttle valve 32, thereby operating the hydraulic actuator.

[0041] The display device 40 is configured to be able to display various images. In this embodiment, the display device 40 is provided at a position that is easily visible to an operator seated in the cockpit in the cabin 10, and displays various images under the control of the shovel controller 30. The display device 40 may be connected to the shovel controller 30 via an on-board communication network such as a CAN, or may be connected to the shovel controller 30 via a one-to-one dedicated line.

[0042] In this embodiment, the display device 40 operates by receiving power from a storage battery 70. The storage battery 70 is charged with power generated by an alternator 11a of the engine 11. The power of the storage battery 70 is also supplied to the shovel controller 30, the display device 40, and other electrical components 72 of the shovel. In addition, the starter 11b of the engine 11 is driven by power from the storage battery 70 to start the engine 11.

[0043] The engine control device 74 is configured to control the engine 11. Various data indicating the state of the engine 11 (for example, data indicating the cooling water temperature detected by the water temperature sensor 11c, etc.) are transmitted from the engine control device 74 to the shovel controller 30. The shovel controller 30 can accumulate this data in an internal temporary storage unit (memory) and transmit it to the display device 40 as appropriate.

[0044] The input device 42 is a device for inputting information to the control device. In this embodiment, the input device 42 is provided within reach of an operator seated in a cockpit in the cabin 10, receives input by the operator, and outputs a signal corresponding to the input to the shovel controller 30. The input device 42 may be a touch panel, a knob switch provided at the tip of an operating lever, a button switch, a lever, a toggle, a rotary dial, or the like provided around the display device 40. The input device 42 transmits a signal related to the input to the shovel controller 30.

[0045] The sound output device 43 is configured to output sound. In this embodiment, the sound output device 43 is provided in the cabin 10 and connected to the shovel controller 30. The sound output device 43 is configured to output sound under the control of the shovel controller 30. Specifically, the sound output device 43 is a speaker, a buzzer, or the like. The sound output device 43 audibly outputs various pieces of information in response to an output command from the shovel controller 30.

[0046] The storage device 47 is configured to store information. In this embodiment, the storage device 47 is provided in the cabin 10 and configured to store various information under the control of the shovel controller 30. Specifically, the storage device 47 is a non-volatile storage medium such as a semiconductor memory. The storage device 47 may store information output by various devices during the operation of the shovel 100, or may store information acquired via various devices before the operation of the shovel 100 is started. The storage device 47 may store data related to a target construction surface acquired via the communication device T1 or the like, or set via the input device 42 or the like. The target construction surface may be set by the operator of the shovel 100, or may be set by a construction manager or the like.

[0047] The boom angle sensor S1 is attached to the boom 4 and configured to detect a rotation angle (hereinafter referred to as a “boom angle”) of the boom 4 relative to the upper rotating body 3. The boom angle sensor S1 transmits a detection signal related to the boom angle to the shovel controller 30.

[0048] The arm angle sensor S2 is attached to the arm 5, and is configured to detect a rotation angle (hereinafter referred to as the “arm angle”) of the arm 5 with respect to the boom 4. The arm angle sensor S2 transmits a detection signal related to the arm angle to the shovel controller 30.

[0049] The bucket angle sensor S3 is attached to the bucket 6, and is configured to detect the rotation angle of the bucket 6 with respect to the arm 5 (hereinafter referred to as the “bucket angle”). The bucket angle sensor S3 transmits a detection signal related to the bucket angle to the shovel controller 30.

[0050] The boom angle sensor S1 is composed of at least one of, for example, a rotary encoder, an acceleration sensor, a gyro sensor, an inertial measurement device, etc. The boom angle sensor S1 may be a potentiometer using a variable resistor, a cylinder stroke sensor that detects the stroke amount of a hydraulic cylinder (boom cylinder 7) corresponding to the boom angle, etc. The same applies to the arm angle sensor S2 and the bucket angle sensor S3.

[0051] The machine body tilt sensor S4 is configured to detect the tilt state of the machine body relative to a predetermined virtual plane. The predetermined virtual plane is, for example, a virtual horizontal plane. The machine body is, for example, the upper rotating body 3. The machine body tilt sensor S4 is attached to, for example, the upper rotating body 3, and detects a roll angle, which is the tilt angle around the front-rear axis of the upper rotating body 3, and a pitch angle, which is the tilt angle around the left-right axis of the upper rotating body 3. The machine body tilt sensor S4 may be, for example, at least one of an acceleration sensor, a gyro sensor, an inertial measurement unit, and the like. The machine body tilt sensor S4 then transmits a detection signal relating to the tilt state to the shovel controller 30.

[0052] The turning state sensor S5 is configured to detect the turning state of the upper rotating body 3. The turning state sensor S5 detects, for example, the turning angular velocity of the upper rotating body 3. The turning state sensor S5 may detect the turning angle of the upper rotating body 3. Specifically, the turning state sensor S5 is a gyro sensor, a resolver, a rotary encoder, or the like. The turning state sensor S5 transmits a detection signal related to the turning angle or the turning angular velocity of the upper rotating body 3 to the excavator controller 30.

[0053] The spatial recognition device S6 is configured to recognize the state of the space to be monitored. The state of the space includes the presence or absence of an object in the space. In this embodiment, the spatial recognition device S6 is configured to recognize the state of the space around the shovel 100. Specifically, the spatial recognition device S6 includes a sensor S6F that recognizes the state of the space in front of the shovel 100, a sensor S6L that recognizes the state of the space to the left of the shovel 100, a sensor S6R that recognizes the state of the space to the right of the shovel 100, and a sensor S6B that recognizes the state of the space behind the shovel 100.

[0054] Sensor S6F is attached to the front end of the upper surface of the cabin 10, sensor S6L is attached to the left end of the upper surface of the upper rotating body 3, sensor S6R is attached to the right end of the upper surface of the upper rotating body 3, and sensor S6B is attached to the rear end of the upper surface of the upper rotating body 3.

[0055] In this embodiment, the spatial recognition device S6 is a LIDAR. Specifically, the sensor S6F is configured by a combination of a plurality of LIDARs. The sensor S6F may be configured by one LIDAR. The same applies to the sensors S6B, S6L, and S6R. Also, each of the sensors S6B, S6F, S6L, and S6R may include a monocular camera. Also, the spatial recognition device S6 may be an RGB-D sensor, a stereo camera, a distance image camera, an infrared sensor, an ultrasonic sensor, a millimeter wave radar, or a laser radar, or may be a monocular camera having an imaging element such as a CCD or a CMOS and having a wide-angle lens attached thereto. The spatial recognition device S6 transmits the acquired information to the shovel controller 30. The spatial recognition device S6 may transmit the acquired information to the display device 40.

[0056] The spatial recognition device S6 may function as an object detection device. In this case, the spatial recognition device S6 may detect an object present around the shovel 100. The object to be detected may be, for example, a person, an animal, a vehicle, a construction machine, a building, or a hole. The spatial recognition device S6 may also calculate the distance between the spatial recognition device S6 or the shovel 100 and the detected object. In this case, the spatial recognition device S6 may also function as an object detection device. When the spatial recognition device S6 is an ultrasonic sensor, a millimeter wave radar, a laser radar, a LIDAR, an RGB-D sensor, an infrared sensor, or the like, the spatial recognition device S6 may transmit a number of signals (laser light, etc.) toward the surroundings and receive the reflected signals to detect the distance between the spatial recognition device S6 and the object and the direction in which the object exists.

[0057] In this embodiment, the spatial recognition device S6 is directly connected to the shovel controller 30. However, the spatial recognition device S6 may be connected to a display device 40 and indirectly connected to the shovel controller 30 via the display device 40.

[0058] The positioning device P1 is configured to measure the position of the upper rotating body 3. The positioning device P1 may be configured to detect the orientation of the upper rotating body 3. In this embodiment, the positioning device P1 is a GNSS compass, and derives the position and orientation of the upper rotating body 3. Then, the positioning device P1 transmits information regarding the position and orientation of the upper rotating body 3 to the excavator controller 30. The orientation of the upper rotating body 3 may be detected based on the output of a direction sensor attached to the upper rotating body 3.

[0059] The communication device T1 is configured to communicate with an external device through a communication network including at least one of a mobile communication network, a satellite communication network, a short-range wireless communication network, and an Internet network, etc. The communication device T1 is, for example, a mobile communication module corresponding to a mobile communication standard such as LTE, 4G, or 5G, a short-range wireless communication module corresponding to a short-range wireless communication standard such as Wi-Fi or Bluetooth (registered trademark), or a satellite communication module for connecting to a satellite communication network.

[0060] Next, the details of the construction support system SYS will be described with reference to Fig. 1. The construction support system SYS mainly includes a shovel 100, a dump truck 200, a management device 300, a support device 400, and a monitoring device 500. The shovel 100 (shovel controller 30), the dump truck 200 (dump controller 210), the management device 300 (control device 310), the support device 400 (control device 410), and the monitoring device 500 (control device 510) are configured to be able to communicate with each other via a communication network 900.

[0061] The dump truck 200 is an example of a transport vehicle. In this embodiment, the dump truck 200 is a vehicle for transporting a load loaded by the shovel 100, and mainly includes a dump controller 210, a display device 220, a communication device 230, and a positioning device 240. The load is, for example, soil, gravel, crushed stone, asphalt, concrete, or the like (hereinafter, referred to as "soil, sand, etc.").

[0062] The dump controller 210 is configured by a computer including a CPU, a volatile storage device, a non-volatile storage device, and various input / output interfaces. The dump controller 210 realizes various functions by executing various programs stored in the non-volatile storage device, for example. The dump controller 210 is configured to control the display device 220, the communication device 230, and the positioning device 240.

[0063] In this embodiment, the dump truck controller 210 has an image generation unit. The image generation unit is configured to generate an image including an image related to a target stop position (described later). Specifically, the image generation unit receives information related to the target stop position calculated by the shovel controller 30 via the communication device 230. Then, the image generation unit generates an image that enables the driver of the dump truck 200 to stop the dump truck 200 at the target stop position, and causes the display device 220 to display the image.

[0064] The display device 220 is configured to display an image. In this embodiment, the display device 220 is provided at a position that is easily visible to the driver seated in the driver's seat, and displays various images under the control of the dump controller 210. The display device 220 may be connected to the dump controller 210 via an in-vehicle communication network such as a CAN, or may be connected to the dump controller 210 via a one-to-one dedicated line.

[0065] The communication device 230 is configured to communicate with an external device through a communication network including at least one of a mobile communication network, a satellite communication network, a short-range wireless communication network, and an Internet network, etc. The communication device 230 is, for example, a mobile communication module corresponding to a mobile communication standard such as LTE, 4G, or 5G, a short-range wireless communication module corresponding to a short-range wireless communication standard such as Wi-Fi or Bluetooth, or a satellite communication module for connecting to a satellite communication network.

[0066] The positioning device 240 is configured to measure the position of the dump truck 200. The positioning device 240 may be configured to detect the orientation of the dump truck 200. In this embodiment, the positioning device 240 is a GNSS compass, and derives the position and orientation of the dump truck 200. The positioning device 240 then transmits information regarding the position and orientation of the dump truck 200 to the dump controller 210. The orientation of the dump truck 200 may be detected based on the output of a direction sensor attached to the dump truck 200.

[0067] The management device 300 is a terminal device installed outside the construction site. In this embodiment, the management device 300 is a stationary computer installed in a management center or the like outside the construction site, and mainly includes a control device 310, a display device 320, and a communication device 330. However, the management device 300 may be a portable terminal such as a notebook PC, a tablet PC, or a smartphone.

[0068] The control device 310 is configured by a computer including a CPU, a volatile storage device, a non-volatile storage device, and various input / output interfaces. The control device 310 realizes various functions by executing various programs stored in the non-volatile storage device, for example. The control device 310 is configured to control the display device 320 and the communication device 330.

[0069] The display device 320 is configured to display an image. In this embodiment, the display device 320 displays various images under the control of the control device 310.

[0070] The communication device 330 is configured to communicate with an external device through a communication network including at least one of a mobile communication network, a satellite communication network, a short-range wireless communication network, and an Internet network, etc. In this embodiment, the communication device 330 is a mobile communication module corresponding to a mobile communication standard such as LTE, 4G, or 5G, a short-range wireless communication module corresponding to a short-range wireless communication standard such as Wi-Fi or Bluetooth, or a satellite communication module for connecting to a satellite communication network.

[0071] The support device 400 is a mobile terminal, for example, a notebook PC, a tablet PC, or a smartphone carried by a worker at a construction site. In this embodiment, the support device 400 is a smartphone carried by the driver of the dump truck 200, and mainly includes a control device 410, a display device 420, a communication device 430, and a positioning device 440. The control device 410, the display device 420, and the communication device 430 are similar to the control device 310, the display device 320, and the communication device 330 in the management device 300.

[0072] In this embodiment, the control device 410 has an image generation unit, similar to the dump truck controller 210. The image generation unit is configured to generate an image including an image related to a target stop position (described later). Specifically, the image generation unit receives information related to the target stop position calculated by the shovel controller 30 via the communication device 430. Then, the image generation unit generates an image that enables the driver of the dump truck 200 to stop the dump truck 200 at the target stop position, and causes the display device 420 to display the image.

[0073] The positioning device 440 is configured to measure the position of the support device 400. The positioning device 440 may be configured to detect the orientation of the dump truck 200 when the support device 400 is attached to a cradle provided near the driver's seat of the dump truck 200. In this case, the positioning device 440 derives the orientation of the dump truck 200 based on the output of a direction sensor attached to the support device 400, for example. Then, the positioning device 440 transmits information regarding the position and orientation of the dump truck 200 to the control device 410.

[0074] The monitoring device 500 is configured to monitor a predetermined area. In this embodiment, the monitoring device 500 is mounted on an aircraft such as a multicopter (drone) or an airship, and is configured to be able to monitor a construction site from the sky. Specifically, the monitoring device 500 mainly includes a control device 510, a communication device 530, a positioning device 540, and a spatial recognition device 550. The control device 510 and the communication device 530 are similar to the control device 310 and the communication device 330 in the management device 300. The monitoring device 500 may be attached to the top of a tall structure such as a steel tower installed at a construction site, and may monitor the periphery of the structure from the sky.

[0075] The positioning device 540 is configured to measure the position of the monitoring device 500. The positioning device 540 may be configured to detect the orientation of the monitoring device 500. In this embodiment, the positioning device 540 is a GNSS compass, and derives the position and orientation of the monitoring device 500. The positioning device 540 then transmits information regarding the position and orientation of the monitoring device 500 to the control device 510. The orientation of the monitoring device 500 may be detected based on the output of a direction sensor attached to the monitoring device 500.

[0076] The spatial recognition device 550 is configured to recognize the state of the space to be monitored, similar to the spatial recognition device S6. In this embodiment, the spatial recognition device 550 is configured to capture an image of the construction site from the sky. Specifically, the spatial recognition device 550 is, for example, a monocular camera, a stereo camera, a distance image camera, an infrared camera, an ultrasonic sensor, a millimeter wave radar, a laser radar, a LIDAR, an RGB-D sensor, or an infrared sensor. In this embodiment, the spatial recognition device 550 is a monocular camera attached to a multicopter. The spatial recognition device 550 may function as an object detection device, similar to the spatial recognition device S6. In this case, the spatial recognition device 550 may detect an object present in the construction site.

[0077] Next, a process (hereinafter, referred to as "estimation process") in which the shovel controller 30 estimates the position and the like of the dump truck 200 will be described with reference to Fig. 4 and Fig. 5. Fig. 4 is a flowchart showing an example of the estimation process. The shovel controller 30 repeatedly executes this estimation process at a predetermined control period. Fig. 5 shows an example of data used in the estimation process.

[0078] First, the shovel controller 30 extracts a point cloud related to the dump truck 200 (step ST1). In this embodiment, the shovel controller 30 extracts the point cloud data related to the dump truck 200 from the point cloud data output by a LIDAR serving as the sensor S6F. The sensor S6F may be composed of a plurality of adjacent LIDARs.

[0079] Point cloud data is an example of three-dimensional data. In this embodiment, the point cloud data is represented by a distance image. A distance image is an image in which the distance from a specific point is used as the value of each element of a two-dimensional array.

[0080] For example, the shovel controller 30 extracts the point cloud data related to the dump truck 200 by removing point cloud data related to objects other than the dump truck 200, such as the ground or materials.

[0081] For example, the shovel controller 30 may extract a difference between a distance image acquired at a first time point and a distance image acquired at a second time point after the first time point as point cloud data related to the dump truck 200. That is, point cloud data related to an object that moved within the time between the first time point and the second time point may be extracted as point cloud data related to the dump truck 200. Alternatively, the shovel controller 30 may extract point cloud data whose height from the ground is equal to or greater than a predetermined height as point cloud data related to the dump truck 200. Alternatively, the shovel controller 30 may extract point cloud data related to the dump truck 200 using any other method.

[0082] Fig. 5(A) shows an image 200G of the dump truck 200 captured by a monocular camera attached to the LIDAR as the spatial recognition device S6. Fig. 5(B) shows point cloud data PG1 related to the dump truck 200 extracted by the shovel controller 30. The point cloud data PG1 shown in Fig. 5(B) corresponds to the image 200G of the dump truck 200 shown in Fig. 5(A). Note that the image 200G of the dump truck 200 captured by the monocular camera is not essential data for performing the estimation process. Therefore, the capture of the dump truck 200 by the monocular camera may be omitted, and the monocular camera itself may be omitted.

[0083] Thereafter, the shovel controller 30 performs a process of removing the bottom surface of the loading platform, and creates a histogram of the normals (step ST2). In this embodiment, the shovel controller 30 obtains a normal a at a point of interest from the point of interest and its neighboring points in the point cloud by principal component analysis. That is, the shovel controller 30 obtains a normal a at each of the points of interest constituting the point cloud relating to five surfaces constituting the loading platform of the dump truck 200. The five surfaces are the bottom surface, the front surface (front panel), the rear surface (rear gate), the left side surface (left side gate), and the right side surface (right side gate). Then, the shovel controller 30 obtains an angle θ formed between the normal a and the vertically upward normal b by using the following formula (1).

[0084]

number

[0085] When the dump truck 200 is located on a horizontal plane, the normal line of the bottom surface of the loading platform faces vertically upward. If the angle θ calculated by the formula (1) is equal to or smaller than a predetermined value, the shovel controller 30 can determine that the normal line a has substantially the same direction as the vertically upward normal line b, and can determine that the point of interest at that time is one of the point clouds related to the bottom surface of the loading platform. Therefore, the shovel controller 30 can remove the point cloud data related to the bottom surface of the loading platform from the point cloud data related to the dump truck 200 by calculating the angle θ. In this way, the shovel controller 30 can extract the surfaces for which the angle θ calculated by the formula (1) is greater than a predetermined value as the remaining four surfaces of the loading platform.

[0086] Fig. 5(C) shows point cloud data PG2 obtained by removing point cloud data related to the bottom surface of the bed from point cloud data PG1 related to the dump truck 200. Specifically, Fig. 5(C) shows point cloud data PG2 related to four surfaces constituting the bed of the dump truck 200. The four surfaces are the front surface (front panel), rear surface (rear gate), left side surface (left side gate), and right side surface (right side gate).

[0087] Then, based on the point cloud data relating to the remaining four faces of the platform, the shovel controller 30 creates a two-dimensional histogram in which the horizontal axis represents the z-component, which is the component in the depth direction of the normal (the front-to-rear direction of the shovel 100), and the vertical axis represents the angle of the normal with respect to the depth direction. In this embodiment, the point cloud data at this time excludes the point cloud data relating to the bottom face of the platform.

[0088] Point cloud data contains errors due to noise, and normals calculated from the point cloud data also contain errors due to noise. Therefore, in this embodiment, the shovel controller 30 assumes that the variation in error of the point cloud follows a Gaussian distribution, and assumes that a two-dimensional histogram created from the normals follows a mixed Gaussian distribution. The shovel controller 30 then estimates the normals of each surface by clustering the point clouds using an EM (Expectation-Maximization) algorithm. For example, the shovel controller 30 sets the normal corresponding to the class containing the largest number of point clouds as a representative normal, and performs clustering on the point clouds belonging to that class using the three-dimensional distance to a nearby point as a threshold value.

[0089] Thereafter, the shovel controller 30 detects a surface (step ST3). In this embodiment, the shovel controller 30 detects the point cloud related to the dump truck 200 after the point cloud related to the bottom surface of the loading platform is removed as a surface. Specifically, the shovel controller 30 detects at least one of the four surfaces constituting the loading platform of the dump truck 200.

[0090] More specifically, the shovel controller 30 detects, as a surface, a point cloud in which the number of points constituting the clustered point cloud as described above is equal to or greater than a predetermined value and which belongs to a class in which the three-dimensional distance is the smallest.

[0091] Furthermore, in order to determine whether the detected surface corresponds to a short side (the front or rear surface of the loading platform) or a long side (the left or right side of the loading platform) of the rectangle, the shovel controller 30 extracts a normal that forms an angle of 90 degrees with respect to the normal of the detected surface. Then, as described above, the shovel controller 30 detects other surfaces by clustering using the three-dimensional distance to the neighboring points. Then, based on the length of each detected surface, the shovel controller 30 determines whether each surface corresponds to a short side or a long side of the rectangle.

[0092] Thereafter, the shovel controller 30 estimates the normal (step ST4). In this embodiment, the shovel controller 30 executes the estimation of the normal by RANSAC (Random Sample Consensus). Specifically, the shovel controller 30 executes the estimation of the normal by RANSAC for the face corresponding to the short side (or long side) obtained from the representative normal, distinguishing the face corresponding to the short side (or long side) from the face corresponding to the long side (or short side).

[0093] The point cloud related to the detected surface typically contains a lot of noise, and therefore the direction of the derived normal line also contains errors. Therefore, in this embodiment, the shovel controller 30 is configured to reduce errors caused by the influence of noise by estimating the normal line by RANSAC for the representative surface.

[0094] Thereafter, the shovel controller 30 estimates the position and the like of the dump truck 200 (step ST5). In this embodiment, the shovel controller 30 detects the faces constituting the bed from the direction of the normal and the center of gravity of the faces obtained as described above, and estimates the position and the like of the dump truck 200 based on the position and orientation of the detected faces. Since the shape of the bed of the dump truck 200 is a substantially rectangular parallelepiped, the shovel controller 30 can estimate the position and orientation of the dump truck 200 by obtaining the shape of the faces constituting the substantially rectangular parallelepiped.

[0095] FIG. 5(D) shows a rectangle RS representing the right side of the bed of the dump truck 200, which is one of the surfaces detected by the shovel controller 30. Specifically, the rectangle RS corresponds to the inner surface of the right side gate. In this case, the shovel controller 30 can estimate the size (tonnage, vehicle type, etc.) of the dump truck 200 from the three-dimensional distance of at least one of the four sides of the rectangle RS, for example. In this case, the shovel controller 30 may derive the size of the dump truck 200 by referring to a table that systematically stores the relationship between the three-dimensional distance of at least one of the four sides of the rectangle RS and the size of the dump truck 200. For example, the shovel controller 30 may refer to the table using the three-dimensional distance of the long side of the rectangle RS representing the right side of the bed of the dump truck 200 (corresponding to the length of the long side of the right side gate) as a search key, and specify the vehicle type of the dump truck 200 equipped with a right side gate of such length from the vehicle type information input in advance. Then, the shovel controller 30 may derive the size of the dump truck 200 (for example, the maximum load capacity, the vehicle length, the vehicle width, or the vehicle height) stored in association with the identified vehicle type of the dump truck 200. Furthermore, the shovel controller 30 can estimate the orientation of the dump truck 200 from, for example, the orientation of the normal line of the rectangle RS. Furthermore, the shovel controller 30 can estimate the position of the dump truck 200 from, for example, the output of the positioning device P1 and the three-dimensional distance between at least one of the four vertices of the rectangle RS and the spatial recognition device S6.

[0096] The shovel controller 30 may be configured to estimate at least one of the position, orientation, and size of the dump truck 200 based on information on the two or more detected surfaces. For example, the shovel controller 30 may derive a rectangle representing the rear surface of the loading platform of the dump truck 200 in addition to the rectangle RS representing the right side of the loading platform of the dump truck 200. The rectangle representing the rear surface of the loading platform of the dump truck 200 corresponds to, for example, the outer surface of the rear gate. In this case, the shovel controller 30 may finally estimate the position, etc. of the dump truck 200 based on, for example, the position, etc. of the dump truck 200 provisionally estimated from the rectangle RS representing the right side of the loading platform and the position, etc. of the dump truck 200 provisionally estimated from the rectangle representing the rear surface of the loading platform. Alternatively, the shovel controller 30 may estimate a part of the position, etc. of the dump truck 200 (for example, only the orientation) from the rectangle RS representing the right side of the loading platform, and estimate the rest of the position, etc. of the dump truck 200 (for example, the position and size) from the rectangle representing the rear surface of the loading platform.

[0097] This estimation process allows the shovel controller 30 to accurately estimate the position of the dump truck 200, etc. Therefore, when the shovel controller 30 executes, for example, a machine control function, it can operate the hydraulic actuator of the shovel 100 appropriately and autonomously. Specifically, the shovel controller 30 can increase the reliability of the loading operation by the shovel 100 by appropriately and autonomously operating the hydraulic actuator mounted on the shovel 100. In the above embodiment, the shovel controller 30 detects the dump truck 200 from the rear, so it removes the bottom surface of the loading platform and extracts the right side surface. However, when the dump truck 200 is detected from above, extracting the bottom surface can increase the estimation accuracy of the position of the dump truck 200, etc. In this case, the shovel controller 30 removes the front surface, rear surface, left side surface, and right side surface, and can obtain the position and orientation of the dump truck 200 based on the extracted bottom surface. In this way, the surface to be removed and the surface to be extracted are changed depending on the orientation in which the dump truck 200 is detected.

[0098] Next, the effect of the estimation process will be described with reference to FIG. 6. FIG. 6 shows an example of a work site where the shovel 100 is loading earth and sand into the dump truck 200. Specifically, FIG. 6 is a top view of the shovel 100 and the dump truck 200 at the construction site. In FIG. 6, the shovel 100 drawn with a solid line represents the state of the shovel 100 when the excavation operation is completed, and the shovel 100 drawn with a dashed line represents the state of the shovel 100 before the soil discharge operation is started. In this embodiment, the soil discharge operation is a bucket opening operation or a combined operation including the bucket opening operation, and does not include a swing operation. Typically, the soil discharge operation is a combined operation including the bucket opening operation, the arm opening operation, and the boom lowering operation. In addition, the bucket 6A drawn with a solid line represents the state of the bucket 6 when the excavation operation is completed, and the bucket 6B drawn with a dashed line represents the state of the bucket 6 before the soil discharge operation is started. In addition, the thick dashed line in FIG. 6 represents a trajectory drawn by a predetermined point on the back surface of the bucket 6.

[0099] In loading work, the operator of the shovel 100 usually performs a boom-raising and swinging operation using the operating device 26 after the excavation operation is completed. In this embodiment, the operator performs a combined operation including a right swing operation. Specifically, the operator performs a combined operation including a boom-raising operation and a right swing operation until the posture of the shovel 100 becomes the posture shown by the dashed line, that is, until a predetermined point on the back surface of the bucket 6 reaches point P2. The combined operation may include an opening and closing operation of the bucket 6. This is to move the bucket 6 onto the bed of the dump truck 200 while preventing the bed of the dump truck 200 from contacting the bucket 6. By performing such a series of operations, the operator can load earth and sand into the bed of the dump truck 200.

[0100] A rectangle CB represented by a thick dotted line represents the position of the bed of the dump truck 200 estimated by the shovel controller 30 through the estimation process. In addition, a two-dot chain line represents the orientation of the dump truck 200 estimated by the shovel controller 30. Specifically, the two-dot chain line represents the front-rear axis L1 of the dump truck 200.

[0101] 6, by estimating the position of the dump truck 200, the shovel controller 30 can recognize that, for example, the rotation axis SA of the shovel 100 and the front-rear axis L1 of the dump truck 200 are in a positional relationship where they do not intersect. That is, the shovel controller 30 can recognize that no matter how the rotation position of the upper rotating body 3 is changed, the front-rear axis of the upper rotating body 3 and the front-rear axis L1 of the dump truck 200 cannot be made parallel to each other.

[0102] Then, when it is recognized that such a positional relationship exists, the shovel controller 30 can recognize that, for example, unless the upper rotating body 3 is rotated appropriately during soil discharge operation, the soil and sand taken into the bucket 6 cannot be discharged evenly onto the bed of the dump truck 200.

[0103] When starting the soil discharge operation, if the front-rear axis of the upper rotating body 3 and the front-rear axis L1 of the dump truck 200 are parallel, the excavator 100 can uniformly spread the soil and sand taken into the bucket 6 on the bed of the dump truck 200 simply by performing the soil discharge operation. This is because the bucket 6 can be moved along the front-rear axis L1 of the dump truck 200. However, if the front-rear axis of the upper rotating body 3 and the front-rear axis L1 of the dump truck 200 are not parallel, the excavator 100 cannot uniformly spread the soil and sand taken into the bucket 6 on the bed of the dump truck 200 simply by performing the soil discharge operation. This is because the bucket 6 cannot be moved along the front-rear axis L1 of the dump truck 200 simply by operating the excavation attachment AT. Therefore, when the shovel controller 30 recognizes that the rotation axis SA of the shovel 100 and the front-rear axis L1 of the dump truck 200 are in a positional relationship where they do not intersect, the shovel controller 30 can autonomously rotate the upper rotating body 3 during the earth-discharging operation so that the soil and sand taken into the bucket 6 can be spread evenly onto the bed of the dump truck 200. In other words, the operator of the shovel 100 can spread the soil and sand taken into the bucket 6 evenly onto the bed of the dump truck 200 without performing a rotation operation during the earth-discharging operation.

[0104] Alternatively, when the shovel controller 30 recognizes that the rotation axis SA of the shovel 100 and the front-rear axis L1 of the dump truck 200 are in a positional relationship where they do not intersect, the shovel controller 30 may drive the traveling hydraulic motor 2M to operate the lower traveling body 1. Specifically, the shovel controller 30 may move the shovel 100 so that the rotation axis SA of the shovel 100 and the front-rear axis L1 of the dump truck 200 intersect.

[0105] Alternatively, the shovel controller 30 may inform the operator of the shovel 100 that the rotation axis SA of the shovel 100 and the front-rear axis L1 of the dump truck 200 are in a positional relationship where they do not intersect. This is to prompt the operator to move the shovel 100 so that the rotation axis SA of the shovel 100 and the front-rear axis L1 of the dump truck 200 intersect.

[0106] Alternatively, the shovel controller 30 may inform the driver of the dump truck 200 that the rotation axis SA of the shovel 100 and the front-rear axis L1 of the dump truck 200 are in a positional relationship where they do not intersect. This is to prompt the driver to move the dump truck 200 so that the rotation axis SA of the shovel 100 and the front-rear axis L1 of the dump truck 200 intersect. In this case, the shovel controller 30 may derive a target stop position and present information about the target stop position to the driver of the dump truck 200. The target stop position is a position where the dump truck 200 should be stopped, and is represented by, for example, a rectangle corresponding to the occupation area of ​​the dump truck 200 when the dump truck 200 is viewed from above. The target stop position is typically set so that the front-rear axis L1 of the dump truck 200 stopped at the target stop position intersects with the rotation axis SA of the shovel 100.

[0107] In this way, being able to accurately estimate the position and the like of the dump truck 200 can improve the efficiency of the loading operation by the shovel 100. This is because the shovel controller 30 can act on at least one of the operator of the shovel 100 and the driver of the dump truck 200 so that the loading operation is performed in a state in which the rotation axis SA of the shovel 100 and the front-rear axis L1 of the dump truck 200 intersect with each other.

[0108] Next, another example of the estimation process will be described with reference to Fig. 7. Fig. 7 is a flowchart showing another example of the estimation process. The shovel controller 30 repeatedly executes this estimation process at a predetermined control period. The estimation process shown in Fig. 7 differs from the estimation process shown in Fig. 4 in that it includes step ST0, but is the same as the estimation process shown in Fig. 4 in that it includes steps ST1 to ST5. Therefore, in the following, a description of the common parts will be omitted, and the different parts will be described in detail.

[0109] In the estimation process shown in FIG. 7, the shovel controller 30 judges whether or not the dump truck 200 is located within a predetermined work range (step ST0). In this embodiment, the shovel controller 30 executes this judgment task before extracting a point cloud related to the dump truck 200 using the output of the spatial recognition device S6. That is, the shovel controller 30 does not extract a point cloud related to the dump truck 200 until it is determined that the dump truck 200 is located within the predetermined work range. This is to prevent unnecessary calculations from being performed, such as accurately estimating the position of the dump truck 200, even though the dump truck 200 is located away from the shovel 100 (a position unsuitable for loading work).

[0110] Specifically, the shovel controller 30 determines whether or not the dump truck 200 is located within a predetermined work range based on an image captured by a monocular camera attached to the spatial recognition device S6. The shovel controller 30 may determine whether or not the dump truck 200 is present based on the output of a device other than the monocular camera, such as an ultrasonic sensor or a millimeter wave radar. That is, the shovel controller 30 may determine whether or not the dump truck 200 is present based on the output of any device including a LIDAR, as long as the determination task of step ST0 can be executed with a lower computation cost than the computation cost for executing the tasks after step ST1. At this stage, the shovel controller 30 only needs to be able to determine whether or not the dump truck 200 is present around the shovel 100, and even if the dump truck 200 is present around the shovel 100, it is only necessary to roughly recognize the position of the dump truck 200. For example, the shovel controller 30 may determine the presence or absence of the dump truck 200 or recognize the position of the dump truck 200 by pattern matching that compares a preregistered image of the dump truck 200 with an image portion included in an image captured by the monocular camera. Alternatively, the shovel controller 30 may determine whether or not an image of the dump truck 200 exists in an image captured by the monocular camera or recognize the position of the dump truck 200 by utilizing deep learning. The shovel controller 30 recognizes, for example, the coordinates of the center of the loading platform of the dump truck 200 or the coordinates of the rear end of the loading platform of the dump truck 200 as the position of the dump truck 200. The coordinates are, for example, coordinates in a coordinate system centered on the position of the monocular camera.

[0111] Furthermore, the shovel controller 30 may recognize the position and orientation of the dump truck 200 based on an image captured by a monocular camera. In this case as well, it is sufficient for the shovel controller 30 to roughly recognize the position and orientation of the dump truck 200.

[0112] The predetermined work range is, for example, a range suitable for loading work. In other words, the dump truck 200 being located within the predetermined work range means that the dump truck 200 is located within a range suitable for loading work. In this embodiment, the predetermined work range is set as a coordinate range in a coordinate system centered on the position of the monocular camera.

[0113] The shovel controller 30 determines whether or not the dump truck 200 is located within the predetermined work range based on the recognized position of the dump truck 200 and the set predetermined work range. For example, when the coordinates of the center of the loading platform of the dump truck 200 are included within a coordinate range corresponding to the predetermined work range, the shovel controller 30 determines that the dump truck 200 is located within the predetermined work range.

[0114] When it is determined that the dump truck 200 is not located within the predetermined work range (NO in step ST0), the shovel controller 30 ends the current estimation process without executing the tasks in and after step ST1.

[0115] On the other hand, when it is determined that the dump truck 200 is located within the predetermined work range (YES in step ST0), the shovel controller 30 executes the tasks from step ST1 onwards and estimates the position of the dump truck 200 and the like.

[0116] Also, once it is determined that the dump truck 200 is located within the predetermined working range, the shovel controller 30 may omit the determination of step ST0 and execute the tasks of step ST1 and thereafter in the next estimation process. For example, during a predetermined period after it is determined that the dump truck 200 is located within the predetermined working range, or until the estimation process is repeated a predetermined number of times, the shovel controller 30 may omit the determination of step ST0 and execute the tasks of step ST1 and thereafter. This is to reduce the calculation load.

[0117] With this configuration, the shovel controller 30 can reduce the calculation load related to estimating the position and the like of the dump truck 200. This is because when there is no dump truck 200 around the shovel 100, or when the dump truck 200 is located in a place unsuitable for loading work, the shovel controller 30 does not execute tasks subsequent to step ST1, which is a task for estimating the position and the like of the dump truck 200.

[0118] As described above, the construction support system SYS, which is a system for the excavator 100 having the lower traveling body 1, the upper rotating body 3 rotatably mounted on the lower traveling body 1, and the attachment attached to the upper rotating body 3 according to one embodiment of the present invention, is configured to acquire three-dimensional data regarding the dump truck 200 based on the output of the object detection device, and estimate at least one of the position, orientation, and size of the dump truck 200 based on information regarding at least one surface constituting the bed of the dump truck 200. The object detection device is typically a spatial recognition device S6 attached to the excavator 100. However, the object detection device may be a spatial recognition device installed outside the excavator 100. For example, the object detection device may be attached to another construction machine, a building, a utility pole, an aircraft, or the like.

[0119] With this configuration, the construction support system SYS can more appropriately support the loading operation by the excavator 100. This is because the construction support system SYS can accurately estimate at least one of the position, orientation, and size of the dump truck 200. In addition, the construction support system SYS can estimate at least one of the position, orientation, and size of the dump truck 200 without performing image recognition processing using machine learning such as deep learning, and therefore can be easily introduced without the need to prepare a trained model or the like.

[0120] The construction support system SYS may be configured to extract information about at least one surface constituting the bed of the dump truck 200 from the three-dimensional data related to the dump truck 200. The at least one surface constituting the bed of the dump truck 200 is typically at least one of four side surfaces constituting the bed of the dump truck 200.

[0121] The construction support system SYS may be configured to extract three-dimensional data related to the bed of the dump truck 200 from the three-dimensional data related to the dump truck 200. For example, the construction support system SYS may be configured to extract three-dimensional data related to the bed of the dump truck 200 by removing three-dimensional data related to the soil and sand around the dump truck 200, the road surface, or the tires of the dump truck 200, from the three-dimensional data output by the spatial recognition device S6.

[0122] When extracting information about at least one surface constituting the bed of the dump truck 200, the construction support system SYS may be configured to preferentially extract information about a surface whose overall shape can be recognized. In this case, the surface whose overall shape can be recognized may be, for example, a surface whose three or more corners can be recognized. Alternatively, the surface whose overall shape can be recognized may be, for example, a surface whose two or more corners can be recognized. Typically, the surface whose overall shape can be recognized is at least one of the four side surfaces constituting the bed of the dump truck 200. When viewed from the spatial recognition device S6, the bottom surface of the bed of the dump truck 200 is often hidden in the shadow of the side surfaces constituting the bed of the dump truck 200, so in the above-mentioned embodiment, the construction support system SYS does not preferentially extract the bottom surface of the bed of the dump truck 200. However, in another embodiment, the construction support system SYS may preferentially extract the bottom surface of the bed of the dump truck 200.

[0123] With this configuration, the construction support system SYS can estimate at least one of the position, orientation, and size of the dump truck 200 based on information about the surface whose overall shape can be recognized, thereby improving the reliability of the estimation result.

[0124] The information on at least one surface constituting the bed of the dump truck 200 is preferably information on a surface other than the bottom surface of the bed of the dump truck 200. As described above, the bottom surface of the bed of the dump truck 200 is mostly hidden behind the side surfaces constituting the bed of the dump truck 200 when viewed from the spatial recognition device S6.

[0125] The construction support system SYS may be configured to derive the size or type of the dump truck 200 based on information on the shape of at least one surface constituting the bed of the dump truck 200, out of information on at least one surface constituting the bed of the dump truck 200. The information on the shape of the surface is, for example, the three-dimensional distance of the long side or short side of a rectangle.

[0126] The construction support system SYS may be configured to derive the position of the dump truck 200 based on information on the position of at least one surface constituting the bed of the dump truck 200, out of information on at least one surface constituting the bed of the dump truck 200. The information on the position of the surface is, for example, the coordinates of the center of gravity of the surface, or the coordinates of a point on the contour of the surface.

[0127] The construction support system SYS may be configured to derive the orientation of the dump truck 200 based on information on the orientation of at least one surface constituting the bed of the dump truck 200, out of information on at least one surface constituting the bed of the dump truck 200. The information on the orientation of the surface is, for example, the orientation of the normal to the surface, or the extension direction of the long side or short side of the surface.

[0128] The construction support system SYS may be configured to determine whether or not the dump truck 200 exists before acquiring the three-dimensional data related to the dump truck 200. Alternatively, the construction support system SYS may be configured to estimate at least one of the position, orientation, and type of the dump truck 200 before acquiring the three-dimensional data related to the dump truck 200. Alternatively, the construction support system SYS may be configured to determine whether or not the dump truck 200 has entered a predetermined working range before acquiring the three-dimensional data related to the dump truck 200. For example, the construction support system SYS may be configured to determine whether or not the dump truck 200 exists based on an image captured by a camera mounted on the shovel 100. In this case, the camera may be a monocular camera or a stereo camera. This is to prevent a task with a high computational cost for accurately estimating the position, etc. of the dump truck 200 from being executed even though the dump truck 200 is away from the shovel 100. With this configuration, the construction support system SYS can prevent the computational load from increasing unnecessarily.

[0129] The preferred embodiments of the present invention have been described above in detail. However, the present invention is not limited to the above-described embodiments. Various modifications or substitutions may be applied to the above-described embodiments without departing from the scope of the present invention. Furthermore, features described separately may be combined unless technical contradictions arise.

[0130] For example, in the above embodiment, the shovel controller 30 is configured to remove point cloud data related to the bottom surface of the bed of the dump truck 200 in the estimation process. However, the shovel controller 30 may be configured to remove point cloud data related to another surface other than the bottom surface of the bed of the dump truck 200. For example, in a situation where the spatial recognition device S6 looks down on the dump truck 200 from a high position, the bottom surface of the bed of the dump truck 200 tends to have a narrower range of blind spots. On the other hand, for example, the rear surface of the bed of the dump truck 200 is likely to have a change in the range of blind spots depending on changes in the distance between the spatial recognition device S6 and the dump truck 200. In this case, the shovel controller 30 may be configured to remove point cloud data related to the rear surface of the bed of the dump truck 200 in the estimation process. [Explanation of symbols]

[0131] 1···Lower traveling body 1C···Crawler 1CL··Left crawler 1CR··Right crawler 2···Slewing mechanism 2A···Slewing hydraulic motor 2M··Travel hydraulic motor 2ML··Left traveling hydraulic motor 2MR··Right traveling hydraulic motor 3···Upper rotating body 4···Boom 5··Arm 6··Bucket 7···Boom cylinder 8··Arm cylinder 9··Bucket cylinder 10··Cabin 11··Engine 11a··Alternator 11b··Starter 11c··Water temperature sensor 13··Regulator 14··Main pump 14c··Oil temperature sensor 15··Pilot pump 17··Control valve unit 26··Operating device 28··Discharge pressure sensor 29··Operating pressure sensor 30 shovel controller 31 proportional valve 32 shuttle valve 40 display device 42 input device 43 sound output device 47 storage device 70 storage battery 72 electrical equipment 74 engine control device 75 dial 100 shovel 200 dump truck 210 dump controller 220 display device 230 communication device 240 positioning device 300 management device 310 control device 320 display device 330 communication device 400 support device 410 control device 420 display device 430 communication device 440 positioning device 500 monitoring device 510 control device 530 communication device 540...Positioning device 550...Spatial recognition device 900...Communication network NS...Switch P1...Positioning device S1...Boom angle sensor S2...Arm angle sensor S3...Bucket angle sensor S4...Machine tilt sensor S5...Turning status sensor S6...Spatial recognition device S6B, S6F, S6L, S6R...Sensors SYS...Construction support system T1...Communication device

Claims

1. A lower running body; An upper rotating body rotatably mounted on the lower traveling body; and an attachment attached to the upper rotating body, acquiring point cloud data related to the dump truck based on the output of an object detection device, randomly extracting one point of interest from the point cloud data related to the dump truck, performing principal component analysis on local point cloud data around the point of interest, and detecting a normal line at the point of interest and a plane perpendicular to the normal line and including the point of interest as a plane constituting the bed of the dump truck, thereby estimating at least one of the position, orientation, and size of the dump truck from the point cloud data related to the dump truck belonging to the plane constituting the bed of the dump truck; System for excavators.

2. A lower running body; An upper rotating body rotatably mounted on the lower traveling body; and an attachment attached to the upper rotating body, Point cloud data relating to the dump truck is acquired based on the output of an object detection device, and before detecting at least one surface constituting the bed of the dump truck and after detecting at least one other surface constituting the bed of the dump truck, point cloud data belonging to at least one other surface constituting the bed of the dump truck is removed, thereby preventing the point cloud data belonging to at least one other surface constituting the bed of the dump truck from being used in detecting at least one surface constituting the bed of the dump truck. System for excavators.

3. At least one other surface constituting the bed of the dump truck from which the point cloud data is removed is changed depending on the orientation of the dump truck. A system for a shovel according to claim 2.

4. At least one other surface among the surfaces constituting the bed of the dump truck is at least one of a bottom surface, a front surface, a rear surface, a left side surface, and a right side surface. A system for a shovel according to claim 2.

5. Estimating at least one of a position, a direction, and a size of the dump truck based on information about a surface whose overall shape can be recognized among at least one surface constituting a bed of the dump truck; At least one surface constituting the bed of the dump truck is rectangular, The surface on which the overall shape can be recognized is a surface on which three or more corners can be recognized. A system for a shovel according to any one of claims 1 to 4.

6. The information regarding at least one surface constituting the bed of the dump truck is information regarding a surface other than the bottom surface of the bed of the dump truck. A system for a shovel according to any one of claims 1 to 5.

7. Deriving the size or type of the dump truck based on information about the shape of at least one surface that constitutes the bed of the dump truck, or Deriving a position of the dump truck based on information about the position of at least one surface constituting the bed of the dump truck, or Deriving an orientation of the dump truck based on information regarding an orientation of at least one surface constituting a bed of the dump truck; A system for a shovel according to any one of claims 1 to 6.

8. estimating a part of the position, orientation, and size of the dump truck based on information about at least one surface constituting the bed of the dump truck, and estimating the remaining part of the position, orientation, and size of the dump truck based on information about at least one other surface constituting the bed of the dump truck; A system for a shovel according to any one of claims 1 to 7.

9. At least one of the position, orientation, and size of the dump truck is estimated based on information about at least one surface constituting the bed of the dump truck, and at least one of the position, orientation, and size of the dump truck is estimated based on information about at least one other surface constituting the bed of the dump truck, estimating at least one of the position, orientation, and size of the dump truck; A system for a shovel according to any one of claims 1 to 8.

10. determining whether the dump truck is present or not before acquiring point cloud data relating to the dump truck; The position of the dump truck when it is determined that the dump truck is present is farther away from the shovel than the position of the dump truck when acquisition of point cloud data regarding the dump truck is started. A system for a shovel according to any one of claims 1 to 9.

11. Estimating at least one of a position, an orientation, and a type of the dump truck before acquiring point cloud data related to the dump truck; A position of the dump truck when at least one of the position, orientation, and type of the dump truck is estimated before acquiring point cloud data related to the dump truck is farther away from a shovel than a position of the dump truck when acquisition of the point cloud data related to the dump truck is started. A system for a shovel according to any one of claims 1 to 10.

12. determining whether the dump truck has entered a predetermined range; The position of the dump truck when the dump truck enters a predetermined range is farther from the shovel than the position of the dump truck when acquisition of point cloud data regarding the dump truck is started. A system for a shovel according to any one of claims 1 to 11.

13. determining whether the dump truck is present or not based on an image captured by a camera mounted on a shovel; The position of the dump truck when it is determined that the dump truck is present based on an image captured by a camera mounted on a shovel is farther away from the shovel than the position of the dump truck when acquisition of point cloud data regarding the dump truck is started. A system for a shovel according to any one of claims 1 to 12.

Images