Work machine

Abstract

To provide a work machine capable of accurately detecting the position of a tray even for a transport vehicle on which there is no data.SOLUTION: A work machine for loading objects to be transported onto a tray of a transport vehicle is provided with: a traveling body; a superstructure rotatably arranged on an upper part of the traveling body; a front work machine attached on the superstructure; a measurement device for measuring objects surrounding the work machine to output a group of points indicating the measurement results; and a processing device for carrying out a detection process of the tray from the group of points output by the measurement device, wherein the processing device samples two faces opposite to each other in a horizontal direction from the group of points output by the measurement device and determines whether or not a distance between the sampled two faces is longer than a predetermined set distance. When the distance between the sampled faces is longer than the predetermined set distance, it detects the tray having the two faces as right and left walls and outputs a position of the tray upon detection of the tray.SELECTED DRAWING: Figure 14

Description

[Technical field]

[0001] The present invention relates to a work machine such as a hydraulic excavator. [Background technology]

[0002] There are known working machines such as hydraulic excavators having an articulated front working machine. This type of working machine is often used for loading work in which excavated soil and other transported materials are loaded onto a transport vehicle such as a dump truck. The loading work includes a transport operation in which the rotating body is rotated relative to the traveling body to transport the material to the transport vehicle such as a dump truck, and a discharge operation (e.g., a soil dumping operation) in which the bucket is pushed up to discharge the transported material onto the transport vehicle.

[0003] During loading work, if the rotating body rotates while the height of the front working machine (for example, the height of the tip of the bucket) is lower than the tray of the transport vehicle, the front working machine may interfere with the transport vehicle during the transport operation. Therefore, there is a demand for a function to assist the operator in operating the work machine during loading work, and a technology that allows the work machine to automatically perform loading work. In order to perform loading work on the work machine automatically or semi-automatically, it is necessary to accurately detect the position and angle of the transport vehicle relative to the work machine.

[0004] A conventional technique for detecting a transport vehicle that is a loading target for a work machine's loading operation is, for example, the technique described in Patent Document 1. Patent Document 1 discloses an image processing system that identifies an area including the transport vehicle from an image showing the transport vehicle, and identifies at least one surface of the transport vehicle from the area including the transport vehicle. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] JP 2020-126363 A Summary of the Invention [Problem to be solved by the invention]

[0006] The image recognition system described in Patent Document 1 uses machine learning, and based on data obtained by machine learning, the position and angle of the transport vehicle are detected from images captured by a camera attached to a work machine. However, in a system using machine learning, if the site or transport vehicle is unlearned, the accuracy and validity of the detection results of the position and angle of the transport vehicle decreases. Therefore, when detecting a transport vehicle at a new site or when detecting a new transport vehicle, learning data for the new site and transport vehicle must be collected in order to ensure the validity and accuracy of the detection results.

[0007] An object of the present invention is to provide a work machine that can detect the position of the tray of a transport vehicle with high accuracy even if the transport vehicle does not have data. [Means for solving the problem]

[0008] In order to achieve the above-mentioned object, the present invention provides a work machine for loading transported goods onto the tray of a transport vehicle, comprising a running body, a rotating body rotatably mounted on the upper part of the running body, a front work machine attached to the rotating body, a measuring device that measures objects around the work machine and outputs a point cloud indicating the measurement results, and a processing device that executes a detection process for the tray from the point cloud output by the measuring device, wherein the processing device extracts two horizontally opposing surfaces from the point cloud output by the measuring device, determines whether the inter-surface distance between the two extracted surfaces is longer than a predetermined set distance, and if it determines that the inter-surface distance is longer than the set distance, detects the tray with the two surfaces as left and right wall surfaces, and outputs the position of the tray when the tray is detected. Effect of the Invention

[0009] According to the present invention, even if a transport vehicle has no data, the position of the tray can be detected with high accuracy. [Brief description of the drawings]

[0010] [Figure 1] FIG. 1 is a side view of a hydraulic excavator, which is an example of a work machine according to the present invention. [Diagram 2] A block diagram showing the main parts of the hydraulic system and the control system provided in the hydraulic excavator shown in FIG. 1 together with the related configuration. [Diagram 3] A plan view showing an example of hydraulic excavator operation. [Figure 4] A side view showing an example of a hydraulic excavator in operation. [Diagram 5] An example of a situation data table [Figure 6] FIG. 1 is a side view of a working machine according to a first embodiment of the present invention, together with a reference coordinate system; [Figure 7] FIG. 1 is a side view of a working machine according to a first embodiment of the present invention, together with a plan view showing a reference coordinate system. [Figure 8] A functional block diagram relating to a detection process of a tray of a transport vehicle by a processing device provided in a work machine according to a first embodiment of the present invention. [Figure 9] Side view showing the reference coordinate system of the transport vehicle [Figure 10] FIG. 1 is a plan view showing a reference coordinate system of a transport vehicle; [Figure 11] A flowchart showing an example of a series of operations including a tray detection process of a work machine according to a first embodiment of the present invention. [Figure 12] 8 is a flowchart showing an example of a procedure for a situation determination process (FIG. 8). [Figure 13] 9 is a flowchart showing an example of a procedure for a transport vehicle detection process (FIG. 8). [Figure 14] 11 is a flowchart illustrating an example of a procedure for a tray detection process. [Figure 15] Schematic diagram showing an example of a cross section of a tray of a transport vehicle [Figure 16] Schematic diagram showing an example of a cross section of a tray of a transport vehicle [Figure 17] Schematic diagram showing an example of a cross section of a tray of a transport vehicle [Figure 18] Schematic diagram showing an example of a cross section of a tray of a transport vehicle [Figure 19] 8 is a flowchart showing an example of a procedure for the detection result correction process (FIG. 8). [Figure 20] FIG. 13 is an explanatory diagram of an example of correction of tray detection data; [Figure 21] FIG. 13 is an explanatory diagram of an example of correction of tray detection data; [Figure 22] FIG. 13 is an explanatory diagram of an example of tray length correction; [Figure 23] Figure showing an example of the tray detection results [Figure 24] Functional block diagram relating to detection processing of a tray of a transport vehicle by a processing device provided in a work machine according to a second embodiment of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] Hereinafter, an embodiment of the present invention will be described with reference to the drawings.

[0012] (First embodiment) 1.Work Machine Fig. 1 is a side view of a hydraulic excavator, which is an example of a work machine according to the present invention, and Fig. 2 is a block diagram showing the main parts of the hydraulic system and the control system provided in the hydraulic excavator shown in Fig. 1 together with the related configuration. In this specification, the left direction in Fig. 1 is defined as the front of the hydraulic excavator (strictly speaking, the rotating body 7). Also, although a large hydraulic excavator is illustrated in Fig. 1 as an example, the present invention is also applicable to medium-sized or smaller hydraulic excavators, or other types of work machines having a front working implement.

[0013] In Fig. 1, a hydraulic excavator 1 (work machine) is used at a work site to perform excavation work for excavating a surface to be excavated, such as the ground, and loading work for loading excavated soil and other transported material onto a transport vehicle, such as a dump truck (described later). The loading work includes a transport operation for transporting the material scooped up into a bucket 10 (described later) to the transport vehicle by rotating a rotating body 7 (described later) relative to a traveling body 5 (described later), and a discharging operation for discharging the material from the bucket 10 to the transport vehicle. The hydraulic excavator 1 includes a vehicle body 3 and a multi-joint type front work machine 2 (work arm) mounted on the vehicle body 3.

[0014] The vehicle body 3 is configured to include a traveling body 5 and a rotating body 7. The traveling body 5 is a lower structure of the hydraulic excavator 1, and is equipped with left and right crawler-type traveling devices 5a driven by traveling motors 4 (hydraulic motors). The rotating body 7 is rotatably attached to the upper part of the frame of the traveling body 5 via a rotating device (not shown), and is driven by a rotating motor 6 (hydraulic motor) of the rotating device to rotate relative to the traveling body 5. The rotating body 7 is equipped with the drive system of the hydraulic excavator 1, including a prime mover, and a driver's cab 20.

[0015] The front work machine 2 is an articulated working device that is attached to the front of the rotating body 7 and rotates together with the rotating body 7 relative to the travelling body 5. The front work machine 2 includes a boom 8, an arm 9, and a bucket 10. The boom 8 is rotatably connected to the front of the rotating body 7 via a boom pin 8a (FIG. 6), and is driven in the vertical direction by a boom cylinder 11. The arm 9 is rotatably connected to the tip of the boom 8 via an arm pin 9a, and is driven in the clouding direction and the dumping direction by an arm cylinder 12. The bucket 10 is rotatably connected to the tip of the arm 9 via a bucket pin 10a and a bucket link 16, and is driven in the clouding direction and the dumping direction by a bucket cylinder 13.

[0016] A boom angle sensor 14 (FIG. 6) is attached to the boom pin 8a, detecting the angle of the boom 8 relative to the revolving structure 7. An arm angle sensor 15 is attached to the arm pin 9a, detecting the angle of the arm 9 relative to the boom 8. A bucket angle sensor 17 is attached to the bucket link 16, detecting the angle of the bucket 10 relative to the arm 9. The angles of the boom 8, arm 9, and bucket 10 may be obtained by detecting the angles of the boom 8, arm 9, and bucket 10 relative to a reference plane (for example, a horizontal plane) using an inertial measurement unit (IMU) and converting these detected values. Alternatively, the strokes of the boom cylinder 11, arm cylinder 12, and bucket cylinder 13 may be detected by a stroke sensor, and the angles of the boom 8, arm 9, and bucket 10 may be obtained by converting these detected values.

[0017] In addition, a tilt angle sensor (not shown) that detects the tilt angle of the vehicle body 3 with respect to a reference plane such as a horizontal plane is attached to the rotating body 7. A rotation device that connects the running body 5 and the rotating body 7 is attached with a rotation angle sensor 19 that detects the rotation angle, which is the relative angle of the rotating body 7 with respect to the running body 5. In addition, an angular velocity sensor (not shown) that detects the angular velocity of the rotating body 7 is attached to the rotating body 7.

[0018] In this embodiment, the boom angle sensor 14, the arm angle sensor 15, the bucket angle sensor 17, the inclination angle sensor (not shown), the turning angle sensor 19 and the angular velocity sensor (not shown) are collectively referred to as the attitude detection sensor 53 (FIG. 2).

[0019] In addition, inside the operator's cab 20 provided on the revolving body 7, there are arranged operation devices for instructing the operation of the traveling body 5, the revolving body 7, and the front work unit 2 (FIG. 2). The operation devices include, for example, a right travel lever 23a, a left travel lever 23b, a right operation lever 22a, and a left operation lever 22b. The right travel lever 23a is used to operate the traveling motor 4 of the right-side traveling device. The left travel lever 23b is used to operate the left-side traveling motor 4. The right operation lever 22a is also used to operate the boom cylinder 11 and the bucket cylinder 13. The left operation lever 22b is also used to operate the arm cylinder 12 and the swing motor 6. In this specification, when the subscript is omitted and it is written as the operation lever 22, it means the right operation lever 22a and the left operation lever 22b. Similarly, when it is written as the operation lever 23, it means the right travel lever 23a and the left travel lever 23b. The operating levers 22, 23 are of an electric lever type, and the amount and direction of operation of the lever by the operator are detected by sensors 52a to 52f (for example, rotary encoders, potentiometers) and an operation signal according to the amount and direction of operation is output.

[0020] Furthermore, a measuring device 70 is installed on the rotating body 7, which measures objects around the hydraulic excavator 1 and outputs a point cloud. This measuring device 70 is an external measuring device that measures the distance (depth) to objects present around the hydraulic excavator 1. The measuring device 70 may be any device that can output point cloud data, and may be, for example, a LiDAR (Light Detection And Ranging) or a stereo camera. One or more measuring devices 70 are attached to the hydraulic excavator 1. A suitable arrangement of the measuring device 70 will be described later.

[0021] 2. Control System As shown in FIG. 2, the drive system of the hydraulic excavator 1 includes a prime mover 103, a hydraulic pump 102, a pilot pump 104, a control device 40, solenoid proportional valves 47a to 47l, a flow control valve 101, a processing device (information processing device) 54, and the like.

[0022] The prime mover 103 is, for example, an engine (or an electric motor), and drives the hydraulic pump 102 and the pilot pump 104. The control device 40 controls the operations of the front working implement 2, the traveling body 5, and the revolving body 7 in response to operation signals input from the operation levers 22, 23. Specifically, the control device 40 calculates a command signal based on operation signals input from the sensors 52a-52f in response to the operation of the operation levers 22, 23 by the operator, and outputs the command signal to the electromagnetic proportional valves 47a-47l. The electromagnetic proportional valves 47a-47l operate in response to the command signal from the control device 40, reduce the pressure of the pressure oil supplied from the pilot pump 104 via the pilot line 100, and output the pilot pressure to the flow control valve 101.

[0023] The electromagnetic proportional valves 47a, 47b output pilot pressure related to the operation of the swing motor 6. The electromagnetic proportional valves 47c, 47d output pilot pressure related to the operation of the arm cylinder 12. The electromagnetic proportional valves 47e, 47f output pilot pressure related to the operation of the boom cylinder 11. The electromagnetic proportional valves 47g, 47h output pilot pressure related to the operation of the bucket cylinder 13. The electromagnetic proportional valves 47i, 47j output pilot pressure related to the operation of the right traveling motor 4. The electromagnetic proportional valves 47k, 47l output pilot pressure related to the operation of the left traveling motor 4.

[0024] The flow control valve 101 is provided with directional control valves (not shown) corresponding to the swing motor 6, the arm cylinder 12, the boom cylinder 11, the bucket cylinder 13, and the left and right traveling motors 4. Each directional control valve operates by receiving in its pressure-receiving chamber a pilot pressure input from a corresponding one of the solenoid proportional valves 47a to 47l, controls the direction and flow rate of the pressure oil supplied from the hydraulic pump 102, and supplies it to the corresponding hydraulic actuator. This causes the swing motor 6, the arm cylinder 12, the boom cylinder 11, the bucket cylinder 13, and the left and right traveling motors 4 to operate in response to the operation of the operation levers 22, 23 by the operator. This causes the position and angle of the bucket 10 to change, the swing body 7 to swing, and the traveling body 5 to travel.

[0025] The processing device 54 is a computer and has a function of executing a tray detection process (described later) based on the point cloud output by the measuring device 70. The processing device 54 has a ROM (Read Only Memory) 71, a RAM (Random Access Memory) 72, a CPU (Central Processing Unit) 73, an external I / F (Interface) 74, and the like. The ROM 71, the RAM 72, the CPU 73, and the external I / F 74 are connected via a bus 75. The processing device 54 is also connected to the control device 40, the monitor 55, the measuring device 70, the attitude detection sensor 53, and the storage device 57 (e.g., a hard disk drive or a large-capacity flash memory) via the external I / F 74. The monitor 55 is disposed in a position inside the cab 20 that is easily visible to the operator sitting in the driver's seat. The function of the processing device 54 will be described later.

[0026] 3.Working example 3 and 4 are a plan view and a side view showing an example of an operation of the hydraulic excavator 1. FIG. 3 and FIG. 4 show a loading operation in which the hydraulic excavator 1 excavates earth and sand, transports the excavated earth and sand, and loads it onto a tray T of a transport vehicle 200. For example, the transport vehicle 200 stops at a position where the hydraulic excavator 1 can load the transported goods (for example, a position within an area within the maximum turning radius from the turning center line 120 of the rotating body 7 and equal to or greater than the turning radius of the rear end of the rotating body 7). At that time, in the hydraulic excavator 1, a tray detection process is executed by the processing device 54 based on the output of the measuring device 70 to detect the position and angle of the tray T of the transport vehicle 200. Based on the result of this tray detection process, the control device 40 of the hydraulic excavator 1 controls the operation of the hydraulic excavator 1 (for example, the front working unit 2) so that the front working unit 2 and the like do not collide with the tray T during the loading operation of the transported goods onto the transport vehicle 200. The results of the tray detection process can be widely applied to interference prevention control between the tray T and the front work machine 2, not only during loading operations, and can also be applied to avoid interference between the tray T and the hydraulic excavator 1 when moving the hydraulic excavator 1 to the next excavation site after loading operations.

[0027] 4. Measurement Equipment Layout 3 and 4 are assumed, one suitable example is to install the measuring device 70 so that, for example, during excavation, a point cloud can be acquired for the area to the left of the rotating body 7. However, the installation position of the measuring device 70 is not limited to this example, and for example, the measuring device 70 can be attached to a position where the front of the rotating body 7 can be measured.

[0028] In addition, in the tray detection process, two surfaces on both the left and right sides of the tray T of the transport vehicle 200 are extracted based on the point cloud data obtained from the measuring device 70. Generally, the inner wall surfaces on both the left and right sides of the tray T of the transport vehicle 200 are formed as flat surfaces or smoothly curved surfaces, so the tray T can be detected by extracting these two surfaces. In this case, it is necessary to install the measuring device 70 so that the inner wall surfaces of the tray T of the transport vehicle 200 can be measured. In this embodiment, the measuring device 70 is positioned on an extension of the vehicle width of the transport vehicle 200 (on the projection of the tray T to the rear of the transport vehicle 200) with the measuring device 70 facing the transport vehicle 200 parked at a predetermined position to the side of the hydraulic excavator 1.

[0029] 3 and 4, when the transport vehicle 200 is parked at a low position relative to the hydraulic excavator 1, it is desirable to install the measuring device 70 so that it can measure the transport vehicle 200 from an oblique bird's-eye view. In this embodiment, the measuring device 70 is installed at the left end of the upper part of the cab 20 with its optical axis facing obliquely downward so as to measure the distance to an object present in the left area of ​​the rotating body 7. In Figs. 3 and 4, θsh represents the horizontal angle of the measuring range of the measuring device 70, θsv represents the vertical angle of the measuring range of the measuring device 70, and Lsr represents the distance from the hydraulic excavator 1 to the measuring range of the measuring device 70.

[0030] 5. Status Data Table FIG. 5 is an example of a situation data table. The situation data table 800 shown in the figure is a data table summarizing the current situation (transition state) related to the tray detection process and the processing contents for each situation, and is predefined and stored in, for example, the ROM 71. In this embodiment, the situation related to the tray detection process is classified into six types: "point cloud not acquired", "transport vehicle not detected", "transport vehicle detected", "tray detected", "turning", and "driving". For each of these states, "corresponding conditions", "detection process execution possibility", "detection result output possibility", and "detection result management method" are defined. The current situation in the situation data table 800 is determined by the transport vehicle detection process 83 and the situation determination process 84 by the processing device 54 as described later, and is temporarily stored in a predetermined storage area of, for example, the RAM 72. Each situation will be described.

[0031] 5-1. “Point cloud not acquired” The point cloud not being acquired refers to a situation where the point cloud data of the measuring device 70 is not input to the processing device 54 for a certain period of time or more due to communication conditions, etc. An example of a situation where the point cloud data cannot be acquired due to a network or equipment malfunction is assumed as an example of the point cloud not being acquired. Under this situation, the tray detection process is not executed by the processing device 54. During this time, the tray detection result is not output from the processing device 54 to the control device 40. Furthermore, the processing device 54 erases data stored in a buffer (e.g., RAM 72) related to the tray detection process so that the tray detection result is not output to the control device 40.

[0032] 5-2. "Transport vehicle not detected" The transport vehicle undetected state is a state in which point cloud data is input from the measuring device 70, but the processing device 54 does not detect anything that is estimated to be an object (excluding the ground) around the hydraulic excavator 1 from the point cloud. In this state, the tray detection process by the processing device 54 is continuously executed. However, since no object is detected, the processing device 54 does not output the tray detection result to the control device 40 (or the processing device 54 may output a detection result indicating that a tray is not detected). Also, as in the state in which point cloud data is undetected, the processing device 54 erases the data related to the tray detection process that is held in a buffer (e.g., RAM 72).

[0033] 5-3. "Transport vehicle detection" The transport vehicle detection is a situation where an object of a size equivalent to the transport vehicle 200 is detected around the hydraulic excavator 1 by the processing device 54 based on the point cloud data from the measuring device 70, but the tray T is not detected. A typical transport vehicle detection is a situation where the transport vehicle 200 is continuously detected, but the tray T is hidden by the transported goods as the loading of the transported goods onto the transport vehicle 200 progresses, and the shape of at least one of the left and right inner wall surfaces of the tray T is no longer detected. Under this situation (i.e., during the loading work of the transported goods), the processing device 54 continues to execute the tray detection process, and outputs the latest tray detection result to the control device 40 as still valid data, since the transport vehicle 200 remains stopped and the tray T is not moving. In other words, the tray T is not detected under the "transport vehicle detection" situation, but the latest tray detection result obtained and held under the "tray detection" situation immediately before the transition to the transport vehicle detection situation is output to the control device 40. Furthermore, in the state where a transport vehicle is being detected, the latest tray detection result is continuously held in a buffer (for example, the RAM 72).

[0034] 5-4. "Tray detection" Tray detection is a situation in which an object of a size equivalent to that of the transport vehicle 200 is detected around the hydraulic excavator 1 by the processing device 54 based on the point cloud data from the measuring device 70, and a tray T is also detected. In this situation, the processing device 54 continues to execute the tray detection process, and sequentially outputs the latest tray detection result obtained in real time to the control device 40. In addition, in a tray detection situation, the tray detection result is sequentially updated, and the latest tray detection result is always held in the RAM 72.

[0035] 5-5. "Turning" During the rotation, the revolving body 7 of the hydraulic excavator 1 rotates at a rotation speed equal to or higher than a preset speed (for example, 1 to 5 deg / sec). The rotation speed is measured, for example, by an angular velocity sensor provided on the revolving body 7. In this embodiment, since the measuring device 70 is installed on the revolving body 7, the accuracy of the tray detection result may decrease due to a decrease in the coordinate conversion accuracy of the point cloud data in the rotation direction during the rotation of the revolving body 7. Therefore, regardless of whether the transport vehicle 200 is detected or not, the tray detection process is not performed by the processing device 54. However, during the loading operation of the transport vehicle 200, the position of the transport vehicle 200 does not change before and after the rotation of the revolving body 7, so the processing device 54 outputs the latest tray detection result to the control device 40 as valid data, as in the case of the transport vehicle detection situation. If the transport vehicle is not being loaded, the data of the tray detection result in the RAM 72 is erased, and therefore is not output to the control device 40 during the rotation. Moreover, the latest tray detection result stored in the RAM 72 continues to be held even during rotation.

[0036] 5-6. "While driving" During traveling, the hydraulic excavator 1 is moving. In this embodiment, since no transported goods are loaded onto the transport vehicle 200 while the hydraulic excavator 1 is moving, the tray detection process is not executed by the processing device 54 in this embodiment. Furthermore, since the relative position between the hydraulic excavator 1 and the transport vehicle 200 changes when the hydraulic excavator 1 moves, the processing device 54 does not notify the control device 40 of the most recent tray detection result. Furthermore, as in the case where the point cloud has not been detected, the processing device 54 erases the data related to the tray detection process stored in the RAM 72.

[0037] The traveling operation of the hydraulic excavator 1 is determined based on whether the amount of operation of the right traveling lever 23a and the left traveling lever 23b detected by the sensors 52e, 52f is equal to or greater than a preset value. However, the method of detecting the traveling operation of the hydraulic excavator 1 is not limited to this, and various other methods can be used, and for example, the traveling operation can be detected by measuring the vehicle speed of the hydraulic excavator 1 with a speedometer or measuring the rotation speed of the traveling motor 4 with a rotation speed sensor.

[0038] 6. Processing Equipment Overview The processing device 54 executes a tray detection process to detect the tray T of the transport vehicle 200 from the point cloud output by the measuring device 70. In the tray detection process, the processing device 54 first extracts two surfaces that face each other in the horizontal direction from the point cloud output by the measuring device 70, and judges whether the distance between the two extracted surfaces is longer than a preset distance. If the distance between the two surfaces is longer than the preset distance, the processing device 54 judges that the two extracted surfaces are the left and right walls of the tray T, detects the tray T having the two extracted surfaces as the left and right walls, and outputs the current position of the tray T to the control device 40. The preset distance to be compared with the distance between the two extracted surfaces is set to be at least larger than the left-right width of the bucket 10 of the front working machine 2, and is set to, for example, a general vehicle width of a vehicle assumed as the transport vehicle 200 or slightly smaller than the width.

[0039] This tray detection process can be preferably performed on the premise that the transport vehicle 200 has been detected. In this case, the processing device 54 performs a detection process of an object having a size equal to or larger than a preset transport vehicle size, or an object detected including a circle corresponding to a running wheel, from the point cloud output by the measuring device 70 prior to the tray detection process, and detects the object as the transport vehicle 200 when the corresponding object is detected. For example, when the site is a mine, the width of a dump truck used as the transport vehicle 200 is generally small, about 7 m, so that a value slightly smaller than that (for example, 5 to 6 m) is assumed as the size of the transport vehicle to be compared with the extracted object. After detecting the transport vehicle 200 in this way, the processing device 54 extracts the above two faces facing each other in the horizontal direction from the point cloud of the object detected as the transport vehicle 200 as part of the tray detection process.

[0040] When the point cloud is not input from the measuring device 70 and when the transport vehicle 200 is not detected, the tray detection process by the processing device 54 is interrupted (not executed). In addition, it is preferable that the tray detection process by the processing device 54 is interrupted (not executed) in a predetermined case regardless of whether the transport vehicle 200 is detected or not. In the case of this embodiment, as described in FIG. 5 on the left, the tray detection process by the processing device 54 is not executed during turning or traveling. For example, the processing device 54 is programmed not to execute the tray detection process when it is determined that the rotating body 7 is turning based on the output of the angular velocity sensor (for example, when the turning speed is 1 deg / sec or more). Also, the processing device 54 is programmed not to execute the tray detection process when it is determined that the hydraulic excavator 1 is traveling based on the output of the traveling sensor (for example, the sensors 52e, 52f).

[0041] The processing device 54 holds the latest detection data of the tray T obtained by the tray detection process in a predetermined case (for example, the transport vehicle detection, tray detection, and each of the turning situations in FIG. 5). Even if the tray T is no longer detected, the processing device 54 outputs the held latest detection data to the control device 40 as being equivalent to the data detected in real time while the transport vehicle 200 is detected (for example, the transport vehicle detection and each of the turning situations in FIG. 5).

[0042] Also, preferably, the processing device 54 corrects the tray detection result in a predetermined case during the tray detection process. As an example, the processing device 54 can be programmed to compare the longitudinal lengths of the two faces extracted during the tray detection process, and when the longitudinal lengths of the two faces are different, correct the detection data (length) of the shorter face to match the detection data (length) of the longer face. As another example, the processing device 54 is programmed to compare the longitudinal length of the tray T detected during the tray detection process with a threshold value, and when the detected length of the tray T is shorter than the threshold value, correct the length of the tray T to a predetermined set tray length (for example, a value set slightly longer than the threshold value). In this case, the threshold value is a value generally assumed as the front-rear length of the tray T of the transport vehicle 200 or a value previously set to be slightly shorter than the threshold value, and can be stored in the ROM 71, for example. When adjusting the front-rear length of the tray T to the set tray length, it is desirable to program the processing device 54 to correct the detected length of the tray T by extending it forward of the transport vehicle 200 based on the detection result of the transport vehicle 200.

[0043] Furthermore, the processing device 54 displays the tray detection data on the monitor 55, superimposed on the point cloud from the measuring device 70. Furthermore, while the processing device 54 stops outputting the current position of the tray T to the control device 40, the processing device 54 displays on the monitor 55 a message indicating that the current position of the tray T has not been detected.

[0044] 6-2.Specific examples The function of the processing device 54 will now be described in detail.

[0045] FIG. 6 is a side view showing the reference coordinate system together with the hydraulic excavator, and FIG. 7 is a plan view thereof. First, a preset vehicle body coordinate system 400 (FIGS. 6 and 7) which is a local coordinate system of the vehicle body 3 with the traveling body 5 as a reference, is stored in, for example, the ROM 71 of the processing device 54 as a reference coordinate system for specifying the positions and angles of the components of the hydraulic excavator 1. In this embodiment, the vehicle body coordinate system 400 is defined as an XYZ right-handed orthogonal coordinate system having an origin at the intersection of, for example, the turning center line 120 which is the rotation axis of the revolving body 7 and the ground contact surface (bottom surface in contact with the ground G) of the traveling body 5. In the vehicle body coordinate system 400, the forward direction of the traveling body 5 is defined as the positive direction of the X axis, the direction from the traveling body 5 toward the revolving body 7 along the turning center line 120 is defined as the positive direction of the Z axis, and the left side of the traveling body 5 is defined as the positive direction of the Y axis. In the vehicle body coordinate system 400, the rotation angle θsw (Figure 7) of the rotating body 7 is defined as 0 degrees when the front working implement 2 faces the positive X-axis direction and the center line of the front working implement 2 (the dashed dotted line in Figure 7) is parallel to the X-axis of the vehicle body coordinate system 400.

[0046] Also, a preset xyz right-handed orthogonal coordinate system with the measuring device 70 as the reference is stored as sensor coordinate system 300, and a preset X'Y'Z' right-handed orthogonal coordinate system serving as the reference for the work site is stored as site coordinate system 500, in ROM 71. Note that in Fig. 6, Lbm is the length of the boom 8 (the center-to-center distance between the boom pin 8a and the arm pin 9a), Lam is the length of the arm 9 (the center-to-center distance between the arm pin 9a and the bucket pin 10a), and Lbk is the length of the bucket 10 (the distance between the center of the bucket pin 10a and the tip of the bucket).

[0047] Fig. 8 is a functional block diagram relating to the tray detection process by the processing device 54. As shown in Fig. 8, the processing device 54 executes, in relation to the tray detection process, an attitude calculation process 81, a coordinate conversion process 82, a transport vehicle detection process 83, a situation determination process 84, and a detection result correction process 85. The contents of each process will be described below in order.

[0048] -Posture calculation- The attitude calculation process 81 is a process of the processing device 54 that calculates the positions and angles of the components of the hydraulic excavator 1 in the vehicle body coordinate system 400 based on the detection signal of the attitude detection sensor 53 .

[0049] For example, a rotation angle θbm (FIG. 6) of the boom 8 with respect to the X-axis is calculated based on a detection signal of the rotation angle of the boom 8 output from boom angle sensor 14. Also, a rotation angle θam (FIG. 6) of the arm 9 with respect to the boom 8 is calculated based on a detection signal of the rotation angle of the arm 9 output from arm angle sensor 15. A rotation angle θbk (FIG. 6) of the bucket 10 with respect to the arm 9 is calculated based on a detection signal of the rotation angle of the bucket 10 output from bucket angle sensor 17. A rotation angle θsw (FIG. 7) of the rotating unit 7 with respect to the traveling body 5 is calculated based on a detection signal of the rotation angle of the rotating unit 7 output from swing angle sensor 19.

[0050] Furthermore, the inclination angle θg (FIG. 6) of the vehicle body 3 (traveling body 5) with respect to a reference plane DP (FIG. 6) is calculated based on a detection signal of the inclination angle of the vehicle body 3 output from an inclination angle sensor (not shown). The reference plane DP is, for example, a horizontal plane perpendicular to the direction of gravity. The inclination angle θg includes a rotation angle θp (not shown) about the Y axis and a rotation angle θr (not shown) about the X axis as components. In addition, the rotation angular velocity ωsw of the rotating body 7 is calculated based on the detection signal of the attitude detection sensor 53.

[0051] - Coordinate conversion - Coordinate conversion process 82 is a process for converting the output point cloud of measurement device 70 from values ​​in sensor coordinate system 300 to values ​​in vehicle body coordinate system 400, using data on the positions and angles of various vehicle body parts calculated in attitude calculation process 81. The point cloud output by measurement device 70 is a collection of three-dimensional point data Ps (Xps, Yps, Zps) represented in sensor coordinate system 300. The point data (Xps, Yps, Zps) in sensor coordinate system 300 is converted into point data Pv (Xpv, Ypv, Zpv) in vehicle body coordinate system 400 using the following (Equations 1) to (Equations 3).

[0052]

number

[0053]

number

[0054]

number

[0055] Here, Rsv is a rotation matrix that converts coordinates in the sensor coordinate system 300 into coordinates in the vehicle body coordinate system 400, and αs, βs, and γs in the rotation matrix Rsv are inclination angles of the measuring device 70 with respect to the x, y, and z axes in the vehicle body coordinate system 400. When the measuring device 70 is fixed to the hydraulic excavator 1, αs, βs, and γs can be obtained, for example, by measuring the position and angle of the measuring device 70 in the vehicle body coordinate system 400 and stored in advance in the ROM 71 or the storage device 57.

[0056] θsw is the rotation angle of the rotating body 7 and is calculated by the attitude calculation process 81.

[0057] Tsv is a translation vector whose starting point is the origin of the vehicle body coordinate system 400 and whose ending point is the origin of the sensor coordinate system 300. The components Lsx, Lsy, and Lsz of Tsv are equal to the origin coordinates of the sensor coordinate system 300 in the vehicle body coordinate system 400. When the measuring device 70 is fixed, the origin position of the sensor coordinate system 300 in the vehicle body coordinate system 400 is immovable, so Tsv(Lsx, Lsy, Lsz) can be measured in advance and stored in the ROM 71 or the storage device 57 in advance.

[0058] -Transport vehicle detection- The transport vehicle detection process 83 is a process for calculating Pdump, which is the position and angle of the tray T of the transport vehicle 200, from the output point cloud of the measuring device 70 converted into values ​​in the vehicle body coordinate system 400 in the coordinate conversion process 82. This transport vehicle detection process 83 also determines whether the situation corresponds to any of "point cloud not obtained", "driving vehicle not detected", "transport vehicle detected", and "tray detected". The flow of the transport vehicle detection process 83 will be described later with reference to Figs. 1 and 13. In the transport vehicle detection process 83, the position and angle of the transport vehicle 200 are specifically acquired according to the following principle.

[0059] FIG. 9 is a side view showing the reference coordinate system of the transport vehicle, and FIG. 10 is a plan view. As shown in FIG. 9 and FIG. 10, the X″Y″Z″ right-handed Cartesian coordinate system, which has the origin at the center of the vehicle width on the central axis of the rear wheels of the transport vehicle 200, is set as the vehicle body coordinate system 600 of the transport vehicle 200. The positive direction of the X″ axis is the direction from the origin toward the front of the vehicle, the positive direction of the Y″ axis is the direction from the origin toward the left along the wheel axis, and the positive direction of the Z″ axis is perpendicular to the X″Y″ axis and is the direction from the origin toward the top of the vehicle. In this embodiment, the position and angle of the tray T of the transport vehicle 200 are specified based on four points Pd1, Pd2, Pd3, and Pd4 located at the four corners of the tray T of the transport vehicle 200 in a plan view. In the transport vehicle detection process 83, points Pd1 to Pd4 are extracted from the output point cloud of the measuring device 70 transformed into the vehicle body coordinate system 400 of the hydraulic excavator 1, and the coordinates of these points Pd1 to Pd4 are obtained as data (Pdump) of the position and angle of the tray T of the transport vehicle 200. The transport vehicle detection process 83 is executed at a predetermined cycle (for example, about 100 ms to 1 sec).

[0060] -Situation assessment- The situation determination process 84 is a process for determining whether the situation corresponds to "traveling" or "turning" based on the lever operation signal acquired via the control device 40, the positions and angles of each part calculated in the attitude calculation process 81, and the turning speed of the turning body 7. The flow of the situation determination process 84 will be described later with reference to FIG.

[0061] -Detection result correction- The detection result correction process 85 is a process for correcting the data of the detection result of the tray T acquired in the transport vehicle detection process 83 as necessary depending on the situation determined in the situation determination process 84. The final detection result data of the tray T acquired through the detection result correction process 85 is output to the control device 40 and the monitor 55, used as basic data for the machine control by the control device 40, and also displayed on the monitor 55 to notify the operator. A specific example of the detection result correction process 85 will be described later.

[0062] 7. Operation 7-1. Series of actions Fig. 11 is a flow chart showing an example of a series of operations of the hydraulic excavator including the tray detection process, illustrating an example of the operation flow of the hydraulic excavator 1 during loading work.

[0063] (Step S101) First, the hydraulic excavator 1 checks whether the transport vehicle 200 is stopped at a position where the transported goods can be loaded. The stop of the transport vehicle 200 can be checked by, for example, the processing device 54 of the hydraulic excavator 1 by receiving a transport vehicle stop notification signal from a vehicle dispatch management system for the transport vehicle 200, such as an FMS (Fleet Management System) used at the work site. In this case, a communication device (not shown) for wireless communication with a server of the vehicle dispatch management system is mounted on the hydraulic excavator 1. Alternatively, the operator may visually check that the transport vehicle 200 is stopped at a position where the transported goods can be loaded, and manually input the fact that the transport vehicle 200 is stopped into the processing device 54. In other words, the processing device 54 can check whether the transport vehicle 200 is stopped at a position where the transported goods can be loaded, based on an input signal input by the operator.

[0064] (Step S102) When it is confirmed that the transport vehicle 200 has stopped at a position where the transported goods can be loaded, the hydraulic excavator 1 detects the start of the loading operation. The start of the loading operation can be detected by detecting the operation of the operation levers 22, 23 by the processing device 54 based on the output of the sensors 52e, 52f, for example. The start of the loading operation is not limited to this method, and can also be detected by receiving an excavation start command from a control such as an FMS. In addition, for example, at the start of the loading operation, the revolving body 7 may be revolved toward the transport vehicle 200, and the start of the loading operation may be detected from the operation of the operation levers 22, 23 based on this data obtained in advance by deep learning of the operation of the hydraulic excavator 1 accompanying the start of the loading operation.

[0065] (Step S103) When the start of the loading operation is detected, the hydraulic excavator 1 acquires, by the processing device 54, current data on the position and angle of the tray T of the transport vehicle 200 onto which the transported goods are to be loaded, and outputs the data to the control device 40. The procedure for acquiring the data on the position and angle of the tray T in step S103 will be described later.

[0066] (Step S104) After acquiring the data on the position and angle of the tray T, the hydraulic excavator 1 executes loading control according to the data on the tray T using the control device 40. In the loading control, the operation of at least one of the rotating body 7 and the front work implement 2 (the boom 8 or the arm 9) is controlled based on the data on the position and angle of the tray T acquired in step S103. As an example, when a swing operation signal is input, even if the boom raising operation is not performed (or even if the operation amount is smaller than appropriate), not only swing but also boom raising according to the swing speed is automatically or semi-automatically commanded. This loading control allows the front work implement 2 (particularly the bucket 10) to move between the excavation position and the discharge position (soil dumping position) without colliding with the tray T.

[0067] (Step S105) The hydraulic excavator 1 performs a determination as to whether or not the loading operation for the transport vehicle 200 currently parked nearby has ended, separately from the loading control. For example, the operator visually checks the loading state of the transported materials on the tray T, and when the operator wishes to end the loading operation on the tray T, he or she performs a specific operation on the operation levers 22, 23 or the monitor 55, etc., to input the end of the loading operation to the processing device 54. Based on the signal input in response to this, the processing device 54 can determine the end of the loading operation. Note that the configuration may also be such that the bucket load of the front working implement 2 is measured, for example, by the pressure of the boom cylinder 11, and the processing device 54 determines the end of the loading operation when the cumulative load amount on the transport vehicle 200 reaches a specified value. Alternatively, the configuration may also be such that the processing device 54 receives measurement data from a load weight measuring device (for example, a load cell) mounted on the transport vehicle 200 from the transport vehicle 200, and determines the end of the loading operation when the measurement data reaches a specified value. If the processing device 54 determines that the loading operation on the parked transport vehicle 200 has not been completed, the hydraulic excavator 1 returns to step S102 from step S105 and repeats the series of processes related to the loading operation on the currently parked transport vehicle 200.

[0068] (Step S106) When the processing device 54 determines that the loading operation on the parked transport vehicle 200 has been completed, the hydraulic excavator 1 determines whether to end operation. The processing device 54 determines whether to end operation, for example, based on a signal that is input (or stopped being input) when an operator (or an administrator in the case of an unmanned vehicle) performs an operation to stop the prime mover 103 of the hydraulic excavator 1. If it is determined that operation is continuing without an operation to end operation, the hydraulic excavator 1 returns to the procedure at step S101 and waits until the next transport vehicle 200 stops at a position where the transported goods can be loaded. If an operation to end operation is performed, the hydraulic excavator 1 executes a termination process (delay process) of the processing device 54 and the control device 40, and ends the flow of FIG. 11.

[0069] 7-2. Tray position acquisition The procedure for acquiring data on the position and angle of the tray T (S103) described above will be described. In the procedure for acquiring data on the position and angle of the tray T, the processing device 54 executes the flows of Figs. 12 to 14 and 19. Figs. 12 and 13 are flows for determining the situation, Fig. 14 is a flow for the tray detection process, and Fig. 19 is a flow for the detection result correction process. Specifically, the flow of Fig. 12 shows the procedure for determining the situation (Fig. 8), in which it is determined whether the tray detection process can be executed based on the displacement of the measuring device 70 (i.e., whether the situation corresponds to "turning" or "traveling" in Fig. 5). The flow of Fig. 13 shows the procedure for the transport vehicle detection process 83 (Fig. 8), in which the processing device 54 determines whether the situation corresponds to "point cloud not acquired", "transport vehicle not detected", "transport vehicle detected", or "tray detected". The tray detection process is executed in the process of determining whether the situation is "tray detected" or "transport vehicle detected", and the procedure is described separately in Fig. 14. 19 shows the procedure of the detection result correction process 85 (FIG. 8). These processes executed in the procedure for obtaining data on the position and angle of the tray T (S103) will be described below in order.

[0070] 7-2. Procedure of the situation judgment process 84 (Figure 8) FIG. 12 is a flowchart showing an example of the procedure of the situation determination process (FIG. 8).

[0071] (Step S201) In the situation determination process 84, the processor 54 first acquires operation signals input from the sensors 52a to 52f (FIG. 2) in response to the operation of the control levers 22, 23 by the operator.

[0072] (Step S202) The processing device 54 determines whether the hydraulic excavator 1 is traveling based on the input operation signal. In this embodiment, whether the hydraulic excavator 1 is traveling is determined based on whether the travel operation signal input from the sensors 52e, 52f is equal to or higher than a preset threshold value Vth.

[0073] (Step S203) If it is determined in step S202 that the travel operation signal is equal to or greater than the threshold value Vth, the processing device 54 determines that the hydraulic excavator 1 is in a "traveling" state, stores data indicative of the current state being "traveling" in the RAM 72, appropriately executes the processing defined in the state data table 800, and terminates the flow of the state determination processing 84 in Figure 12.

[0074] (Step S204) On the other hand, if it is determined in step S202 that the traveling operation signal is less than the threshold value Vth, the processing device 54 determines that the hydraulic excavator 1 is not traveling (the traveling body 5 is stationary relative to the ground), and reads from the RAM 72 the current rotation speed calculated by the posture calculation processing 81.

[0075] (Step S205) After acquiring the current swing speed in step S204, the processing device 54 judges whether the hydraulic excavator 1 (swing body 7) is swinging or not based on the current swing speed. In this embodiment, whether or not the hydraulic excavator 1 is swinging or not is judged based on whether or not the current swing speed is equal to or greater than a preset threshold value Ωth.

[0076] (Step S206) If it is determined in step S205 that the rotation speed is equal to or greater than the threshold value Ωth, the processing device 54 determines that the hydraulic excavator 1 is in a state where the rotating body 7 is “rotating”, stores data indicative of the current state being “rotating” in the RAM 72, appropriately executes the processing defined in the state data table 800, and terminates the flow of the state determination processing 84 in FIG. 12.

[0077] (Step S207) On the other hand, if it is determined in step S205 that the rotation speed is less than the threshold value Ωth, the processing device 54 determines that the hydraulic excavator 1 is not rotating (the rotating body 7 is stationary relative to the running body 5), and acquires the current detection result obtained in the transport vehicle detection process 83 (Figure 13).

[0078] (Step S208) When the detection result by the transport vehicle detection process 83 (FIG. 13) is acquired in step S207, the processing device 54 updates the situation determination data stored in the RAM 72 with the acquired detection result and terminates the situation determination process 84 in FIG. 12.

[0079] 7-3. Procedure of transport vehicle detection process 83 (Figure 8) Fig. 13 is a flow chart showing an example of the procedure of the transport vehicle detection process (Fig. 8). This flow is executed by the processing device 54 separately from (for example, in parallel with) the transport vehicle detection process 83 in Fig. 12. Alternatively, when the vehicle is not determined to be "driving" or "turning" in the situation determination process 84 in Fig. 12, this flow may be executed as a detection result acquisition process (S207). In this case, the processes of steps S304 and S305 in Fig. 13 (described later) are not necessary.

[0080] (Step S301) In the transportation vehicle detection process 83, the processing device 54 first acquires the latest point cloud data measured by the measuring device 70. For example, the processing device 54 converts the point cloud data output by the measuring device 70 from values ​​in the sensor coordinate system 300 to values ​​in the vehicle body coordinate system 400 in the coordinate conversion process 82 (FIG. 8), temporarily stores the converted data in the RAM 72, and reads out the latest data at the start of the process in step S103 in FIG.

[0081] (Step S302) Next, the processing device 54 compares the output time of the latest point cloud data acquired in step S301 (the time output from the measuring device 70) with the current time, and determines whether the elapsed time from the output time of the point cloud data to the current time is less than or equal to a preset time ΔT (e.g., 1 to 3 sec). For the output time of the point cloud data, for example, a time stamp given by the processing device 54 when the point cloud data is input from the measuring device 70 can be used. Also, for the current time, the real-time time measured by the processing device 54 is used.

[0082] (Step S303) If it is determined in step S302 that the elapsed time is equal to or longer than the set time ΔT, the processing device 54 determines that the situation is "point cloud not acquired", which means that no point cloud data has been input from the measuring device 70 for a certain period of time (i.e., equal to or longer than the set time ΔT). Then, the processing device 54 stores data indicating that the current situation is "point cloud not acquired" in the RAM 72, and appropriately executes the processing defined in the situation data table 800 to end the transport vehicle detection processing 83 in FIG.

[0083] (Step S304) When it is determined in step S302 that the elapsed time is less than the set time ΔT, that is, when the point cloud data is being acquired in real time, the processing device 54 acquires data on the current situation determined in the situation determination process 84 (FIG. 12). Specifically, data on whether the vehicle is "driving" or "turning" is read from the RAM 72 by the processing device 54.

[0084] (Step S305) After reading the determination of the situation determination process 84 (Fig. 12), the processing device 54 determines whether or not to execute detection processing for the transport vehicle 200 and the tray T based on the situation data table 800 (Fig. 5). If the current situation is "driving" or "turning", execution of the detection process is not possible, and the processing device 54 ends the transport vehicle detection process 83 in Fig. 13. On the other hand, the determination in step S305 is executed on the condition that the situation is not "point cloud not yet acquired", so if the situation is neither "driving" nor "turning", the processing device 54 moves the procedure to the detection process.

[0085] (Step S306) When it is determined in step S305 that the excavator 1 is not in the "traveling" state or the "turning" state and the procedure is shifted to the detection process, the processing device 54 detects objects around the excavator 1 from the point cloud data acquired in step S301. For example, the processing device 54 calculates the distance between each point in the point cloud data, extracts a set (cluster) of points whose distance between adjacent points is equal to or less than a predetermined distance, and detects a cluster having a number of points greater than a predetermined value as a surrounding object. At that time, points estimated to be points whose distance to the ground is measured are removed from the point cloud data acquired in step S301, and the remaining data is used for detecting the surrounding object. For example, a plane corresponding to the ground can be extracted from the point cloud data, and a point cloud included in the extracted plane can be estimated to be points whose distance to the ground is measured and removed. Then, the distance between adjacent points (two-point distance) is calculated for each point remaining after removing the points whose distance to the ground is measured, and a set of points whose two-point distance is smaller than a predetermined value is extracted as a point cloud cluster capturing the same object. For example, the Euclidean Cluster Extraction algorithm can be applied to extract point cloud clusters to be detected as objects, and for example, the RANSAC (RANdom SAmple Consensus) algorithm can be applied to extract the ground surface.

[0086] (Step S307) After completing the detection process of the surrounding objects, the processing device 54 judges whether the detected surrounding objects include the transportation vehicle 200. For example, the smallest cube that encloses the cluster of the point cloud detected in the process of step S306 is calculated, and if one side of the cube is a preset size (for example, 5 m) or more, it can be judged that the point cloud cluster enclosed by the cube is data for detecting the transportation vehicle 200. It is also desirable to prevent the point cloud grater that measures sand and dust from being erroneously detected as the transportation vehicle 200. In that case, for example, if one side of the cube that encloses the cluster of the point cloud is larger than the maximum value of the assumed size of the transportation vehicle or a value estimated larger than that (for example, 10 m), the processing device 54 can be programmed so that even a point cloud cluster of a set size or more is not detected as the transportation vehicle 200.

[0087] As described above, the determination of the transport vehicle 200 is not limited to the determination based on the size of the point cloud cluster, but can also be determined based on whether or not a characteristic geometric shape of the transport vehicle 200 (for example, the circular shape of the tires) is extracted.

[0088] (Step S308) If it is determined in step S307 that the transport vehicle 200 has not been detected, the processing device 54 determines the situation to be "transport vehicle not detected", stores data indicating that the current situation is "transport vehicle not detected" in the RAM 72, and appropriately executes the processing specified in the situation data table 800 to terminate the transport vehicle detection processing 83 in Figure 13.

[0089] (Step S309) When it is determined in step S307 that the transport vehicle 200 is detected, the processing device 54 executes a tray detection process. In this tray detection process, for the point cloud cluster estimated to be the transport vehicle 200, Pdump (Pd1 to Pd4) described above in Fig. 9 and Fig. 10 is calculated. The tray detection process (step S309) will be described later with reference to Fig. 14.

[0090] (Step S310) After executing the tray detection process, the processing device 54 judges whether the tray detection result is successful. The validity of the tray detection result can be judged by calculating the length in the width direction (i.e., the distance between the two left and right faces) for Pdump obtained in the tray detection process and checking whether it is equal to or larger than a predetermined size. The predetermined size in this case is, for example, equivalent to the width of the transport vehicle 200. Furthermore, if there is a possibility that the bucket of the hydraulic excavator 1 may enter the measurement range of the measuring device 70, there is a possibility that the side of the bucket may be erroneously detected as the side of the transport vehicle, so it is desirable to make it at least equal to or larger than the bucket width.

[0091] (Step S311) If it is determined in step S310 that a tray T has been detected, the processing device 54 determines that the situation is "tray detected", stores data indicating that the current situation is "tray detected" and data on the detection results of the transport vehicle 200 and the tray T in the RAM 72, and appropriately executes the processing specified in the situation data table 800 to terminate the transport vehicle detection processing 83 in Figure 13.

[0092] (Step S312) If it is determined in step S310 that the tray T is not detected, the processing device 54 determines that the situation is "transport vehicle detected", stores data indicating that the current situation is "transport vehicle detected" in the RAM 72, and appropriately executes the processing specified in the situation data table 800 to terminate the transport vehicle detection processing 83 in Figure 13.

[0093] 7-4. Tray detection process 14 is a flowchart showing an example of a procedure for the tray detection process. In the tray detection process, two surfaces (for example, inner wall surfaces on the left and right sides) of the tray T of the transport vehicle 200 are extracted as follows, and Pdump is output as a feature point of the tray T.

[0094] (Step S401) When the tray detection process is started, the processing device 54 first acquires data of the point cloud cluster extracted as the transport vehicle 200.

[0095] (Step S402) After acquiring the data of the point cloud cluster extracted as the transport vehicle 200, the processing device 54 calculates the normal vector of each point of the point cloud cluster. The normal vector of each point can be obtained by a general method, for example, by fitting a plane to the local point cloud using the least squares method, and a vector perpendicular to the plane can be obtained as the normal vector.

[0096] (Step S403) After calculating the normal vector of each point, the processing device 54 extracts point cloud clusters whose distance from adjacent points is equal to or less than a predetermined value and whose normal vectors are equal to or less than a predetermined value (grouping process). For example, the Region Growing algorithm can be applied to the grouping process using normals.

[0097] (Step S404) After the grouping process is performed, the processing device 54 calculates an approximation plane for each of the grouped point cloud clusters, and calculates a normal vector of the approximation plane for each point cloud cluster. For example, the RANSAC algorithm can be applied to calculate the approximation plane.

[0098] (Step S405) After calculating the approximate plane, the processing device 54 removes unnecessary point cloud clusters, specifically, point cloud clusters whose normal vectors calculated in step S404 are parallel to the gravity direction vector. For example, the processing device 54 calculates the inner product of the normal vector and the Z-axis direction vector of the vehicle body coordinate system 400, and determines whether the normal vector is parallel to the gravity direction vector based on whether the calculated inner product is equal to or larger than a preset threshold. When the pitch angle θpitch (FIG. 9), roll angle θroll (FIG. 9), and yaw angle θyaw (FIG. 10) of the transport vehicle 200 can be obtained, the inner product may be calculated taking into account the pitch angle and roll angle.

[0099] (Step S406) After removing unnecessary point cloud clusters, the processing device 54 determines whether or not there are point cloud clusters that form a pair (e.g., whose approximate planes are parallel) as the left and right inner wall surfaces of the tray T, from the point cloud clusters that constitute the approximate planes calculated in step S404. For example, for each point cloud cluster extracted in step S404, the combination in which the absolute value of the inner product of the normal vectors of each approximate plane is equal to or greater than a preset threshold and the inner product is maximum is calculated as the two left and right surfaces of the tray T.

[0100] The tray of the transport vehicle 200 has various shapes. FIGS. 15 to 18 are schematic diagrams of cross sections of the tray T of the transport vehicle cut by a vertical plane extending in the vehicle width direction. The tray T in FIG. 16 is an example in which the left and right side surfaces are flat and parallel, but depending on the tray T, as shown in FIG. 15, the left and right side surfaces are flat, but the distance between the surfaces increases toward the top, and the left and right side surfaces may not be parallel. In addition, as shown in FIG. 17 and FIG. 18, some or all of the left and right side surfaces of the tray T may be formed by curved surfaces. In this case, instead of comparing the normals of the approximation plane calculated from the point cloud cluster, for example, the projection vectors of the normal vectors of the approximation plane onto the XY plane in the vehicle body coordinate system 400 can be compared to extract pairs constituting the left and right two surfaces of the tray T.

[0101] (Step S407) If it is determined in step S406 that the two surfaces forming a pair have not been extracted (for example, if there is no pair of approximate planes whose absolute value of the inner product is greater than or equal to a threshold value), the processing device 54 proceeds to step S402 (Figure 13) (terminating tray detection in Figure 14).

[0102] (Step S408) If it is determined in step S406 that two surfaces forming a pair have been extracted, the processing device 54 calculates the vertices of the four corners of the tray T (points Pd1 to Pd4 in FIG. 10). For example, two point groups (point groups arranged in a line) that form the upper end faces of each point group cluster that constitutes the point group are extracted, and both ends (start point and end point) of these two point groups are extracted as points Pd1 to Pd4. At this time, since the transport vehicle 200 has already been detected, the processing device 54 can detect the front and rear of the tray T.

[0103] (Step S409) After calculating the four corner points Pd1 to Pd4 of the tray T, the processor 54 moves the procedure to step S401 (FIG. 13) (tray detection in FIG. 14 is completed).

[0104] 7-5. Processing procedure for correcting detection results 19 is a flow chart showing an example of the procedure of the detection result correction process (FIG. 8). The detection result of the tray T by the transport vehicle detection process 83 is corrected in a predetermined case as described below.

[0105] (Step S501) After completing the transport vehicle detection process 83 and the situation determination process 84 , the processing device 54 reads data on the situation determination and tray detection results obtained in the transport vehicle detection process 83 and the situation determination process 84 from the RAM 72 .

[0106] (Step S502) Next, the processing device 54 determines whether or not to output the detection result based on the situation data table 800. That is, it determines whether the current situation is "transport vehicle detected", "tray detected", "turning", or another situation.

[0107] (Step S503) When the current situation is “transport vehicle detected”, “tray detected”, or “turning”, the processing device 54 corrects the coordinates of the points Pd1 to Pd4 of Pdump detected in the transport vehicle detection process 83.

[0108] 20 and 21 are explanatory diagrams of an example of correction of tray detection data. Tray T is generally rectangular when viewed from above. However, for example, when transported objects such as soil and sand loaded on tray T interfere with the distance to the four corners of tray T by the measuring device 70, or when the point cloud output from the measuring device 70 is biased, the detected tray T may become a trapezoid as shown in FIG. 20 and FIG. 21. Therefore, assuming that the actual tray T is rectangular, the points Pd1 to Pd4 of the tray T detected as a trapezoid are corrected to become the vertices of a rectangle. In FIG. 20 and FIG. 21, the vertices of tray T before the correction in step S503 are displayed as points Pd1, Pd2, Pd3, and Pd4, and the vertices of tray T after the correction are displayed as points Pd1', Pd2', Pd3', and Pd4'.

[0109] The correction of points Pd1 to Pd4 is performed by the processing device 54 using, for example, the following (Equation 4) to (Equation 8). In the example of Fig. 20 and Fig. 21, the line segment Pd3-Pd4 and the line segment Pd1-Pd2 are parallel but have different lengths. In this example, the length of the shorter line segment Pd1-Pd2 is adjusted to the length of the longer line segment Pd3-Pd4, and points Pd1 and Pd2 are corrected to points Pd1' and Pd2' so that points Pd1' to Pd4' form a rectangle. In Fig. 20, point Pd1 is corrected to point Pd1', and further, in Fig. 21, point Pd2 is corrected to point Pd2'.

[0110]

number

[0111]

number

[0112]

number

[0113]

number

[0114]

number

[0115] (Step S504) After correcting the vertices to a rectangle, the processor 54 calculates the detected tray length Lrecg as the front-rear length of the tray T assumed by the corrected points Pd1' to Pd4'. Lrecg is equal to the lengths of the vector A (FIG. 20) and vector D (FIG. 21) in the corrected Pdump.

[0116] (Step S505) After calculating the detected tray length Lrecg, the processor 54 determines whether Lrecg is less than a preset threshold value Lth.

[0117] (Step S506) If Ldump is less than Lth, the processor 54 corrects the coordinates of the points Pd1 to Pd4 so that Ldump becomes equal to the preset tray length Ldump.

[0118] Fig. 22 is an explanatory diagram of an example of tray length correction. As shown in the figure, the detected tray length Lrecg is extended in the traveling direction (forward) of the transport vehicle 200. Specifically, the coordinates of points Pd2' and Pd3' are corrected to points Pd2", Pd3" using (Equation 9).

[0119]

number

[0120] (Step S507) Once the tray length has been corrected, or if Ldump is greater than or equal to Lth in step S505, the processing device 54 outputs the corrected position and angle data (Pdump) of the tray T to the control device 40 and monitor 55, and terminates the procedure of Figure 19 relating to the detection result correction process 85.

[0121] 8. Display examples 23 is a diagram showing an example of the display of the detection results of the tray T. In the example of the figure, as data (Pdump) of the position and angle of the transport vehicle 200 detected by the processing device 54, line segments Pd1-Pd2 and Pd3-Pd4 are displayed superimposed on the point cloud measured by the measuring device 70. This makes it possible to inform the operator that the upper end surface of the side surface of the tray T has been detected from the point cloud. The operator on board the hydraulic excavator 1 can visually confirm the detection state of the tray T by the processing device 54 from this display, and can use it as a reference for, for example, the operation timing of the loading control.

[0122] Preferably, among the points measured by the measuring device 70, the points extracted as the left and right side surfaces of the tray T in the transport vehicle detection process 83 are displayed in a manner that distinguishes them from other parts by changing the color or density as shown in the figure. It is possible to visually check whether the side surface plane of the tray has been properly detected.

[0123] In addition, in the example of the figure, a frame F is displayed with a display color changed according to the situation defined in the situation data table 800. For example, while the processing device 54 is detecting the tray T, the operator can intuitively confirm that the tray T has been detected by displaying the frame F in blue or green, for example. In addition, when the transport vehicle 200 is not detected due to dust or the like, the operator can intuitively confirm that the transport vehicle 200 has not been detected by displaying the frame F in red, for example. By notifying the operator of the tray detection situation in this way, the operator can appropriately determine whether or not automatic control may be applied to the operation of the hydraulic excavator 1. In addition, the situation can be specifically notified by text information I such as "tray detection in progress" as shown in the figure.

[0124] In addition, when displaying the detection status by the processing device 54 according to the status data table 800, while the output of the current position of the tray T has stopped (for example, "point cloud not acquired," "transport vehicle not detected," "moving"), it is also possible to display on the monitor 55 that the current position of the tray T has not been detected.

[0125] 9. Effects (1) Two surfaces facing each other in the horizontal direction are extracted from the point cloud output by the measuring device 70, and it is determined whether the distance between the two extracted surfaces is longer than a preset distance. If the distance between the surfaces is longer than the preset distance, the tray T having the two surfaces as left and right wall surfaces is detected. By detecting the tray T based on the characteristics common to the transport vehicle 200 in this way, the tray T can be detected without relying on machine learning. Even if there is no data for the transport vehicle, the position of the tray T can be detected with high accuracy. Of course, the tray T of the transport vehicle 200 can be detected at a new work site. Since new machine learning is not required, the adjustment time of the processing device 54 at the work site can also be shortened. By applying the detection data of the tray T to the control of the loading operation by the hydraulic excavator 1, the productivity of the earth and sand transport work is also improved.

[0126] (2) By setting the set distance to be compared with the inter-surface distance between the two extracted surfaces to be at least greater than the width of the bucket 10 of the front work implement 2, it is possible to prevent the bucket 10 from being measured by the measuring device 70 and erroneously detected as a surrounding object.

[0127] (3) In this embodiment, an object that is equal to or larger than a preset size, or an object that is detected from the point cloud output by the measuring device 70 and includes a circle corresponding to a running wheel, is detected as the transport vehicle 200, and two horizontally opposing surfaces are extracted from the point cloud of the transport vehicle 200. By detecting the tray T on the premise that the transport vehicle 200 is detected in this manner, the tray T, which is a component of the transport vehicle 200, can be detected rationally.

[0128] (4) If a point cloud is not input from the measuring device 70, if the transport vehicle 200 is not detected, or if the effectiveness of the tray detection process is not ensured, the tray detection process can be interrupted, thereby reducing the load on the processing device 54.

[0129] (5) Although the tray detection process itself can be executed even during turning or traveling, the measuring device 70 is displaced during turning or traveling, and therefore the accuracy of coordinate calculation of the point cloud during that time decreases. Therefore, by interrupting the tray detection process when the hydraulic excavator 1 is turning or traveling as in this embodiment, it is possible to suppress the output of detection results that lack accuracy and to reduce the calculation load on the processing device 54.

[0130] (6) Since an algorithm is defined for each situation in the situation data table 800, for example, even if the tray T is no longer detected due to loading of transported goods, the latest detection data is output to the control device 40 and the monitor 55 while the transport vehicle 200 is detected. In this way, by predefining whether or not to execute the tray detection process and the method of managing the detection results, it is possible to accurately detect the tray T according to the situation, reduce the load on the processing device 54, and take flexible measures. In addition, in the "transport vehicle detected" and "turning" situations, although the tray T is not detected, valid tray detection data can be held, and the opportunities for valid tray detection data to be output increase, improving productivity.

[0131] (7) If the longitudinal lengths of the two surfaces extracted during the tray detection process are different, the detection data of the shorter surface is corrected to match the longer surface. For example, even if a transported object gets in the way and causes a difference in the proportion of the area detected by the measuring device 70 between the left and right surfaces of the tray T, the two left and right surfaces of the tray T are actually symmetrical, so the detection area of ​​the tray T can be rationally expanded.

[0132] (8) Furthermore, in many cases, the tray T of the transport vehicle 200 at a mine site has a bottom that slopes downward toward the front, as shown in the schematic diagram of Fig. 9, so that the loaded soil and sand will be concentrated toward the front. In this case, as the loading of soil and sand into the tray T progresses, the front portions of the left and right inner wall surfaces of the tray T cannot be measured by the measuring device 70, and in step S504, Lrecg may be calculated to be much shorter than the actual length of the tray T.

[0133] Therefore, in this embodiment, when the length of the detected tray T, i.e., the detected tray length Lrecg, is shorter than the threshold value Lth, the detected tray length Lrecg is corrected to the set tray length Ldump. At that time, by correcting the detected tray length Lrecg by extending it forward, it is possible to detect the entire tray T rationally, taking into account the fact that the front part of the tray T is difficult to detect as described above.

[0134] (9) Since the measuring device 70 is installed on the rotating body 7, particularly on the upper part of the driver's cab 20 in this embodiment, the transport vehicle 200 can be measured efficiently by the measuring device 70.

[0135] (10) By displaying the detection data of the tray T on the monitor 55 superimposed on the point cloud from the measuring device 70, it is possible to inform the operator that the upper end surface of the side of the tray T has been detected from the point cloud. The operator on board the hydraulic excavator 1 can visually confirm the detection state of the tray T by the processing device 54 from this display, and can use it as a reference for, for example, the operation timing of the loading control.

[0136] (11) Of the point clouds measured by the measuring device 70, the point clouds extracted as the left and right side surfaces of the tray T in the transport vehicle detection process 83 are highlighted and displayed using color, density, etc. This allows the operator to visually confirm whether the processing device 54 has properly detected the tray T from the shape of the highlighted point clouds, etc. The display of a frame F on the monitor 55 that changes color depending on the situation and text information I that specifically explains the situation can also effectively inform the operator of the situation.

[0137] Second embodiment Fig. 24 is a functional block diagram relating to tray detection processing by a processing device provided in a work machine according to a second embodiment of the present invention. In this figure, elements that are the same as or correspond to those in the first embodiment are given the same reference numerals as those in the previously mentioned drawings, and descriptions thereof will be omitted.

[0138] The work machine according to this embodiment is equipped with a communication device 60 that receives position data of the transport vehicle 200. The processing device 54 receives position data of the transport vehicle 200 around the hydraulic excavator 1 from a vehicle dispatch management system (e.g., FMS) via the communication device 60. The processing device 54 of this embodiment executes the tray detection process only when it is determined that the transport vehicle 200 is located in a preset area based on the position data of the transport vehicle 200 received by the communication device 60. In the first embodiment, the tray detection process (the procedure of acquiring the tray position in step S103 in FIG. 11) is executed periodically regardless of the presence or absence of the transport vehicle 200. In contrast, the processing device 54 of this embodiment executes the tray detection process (the procedure of acquiring the tray position in step S103 in FIG. 11) based on the data received via the communication device 60 only during the period from when the transport vehicle 200 enters the set area until when it leaves the set area. In other respects, this embodiment is similar to the first embodiment.

[0139] According to this embodiment, in addition to the same effects as those of the first embodiment, the effect of reducing the processing load of the processing device 54 compared to the first embodiment can be obtained by limiting the opportunity for executing the tray detection process to only when the transport vehicle 200 is located around the hydraulic excavator 1. Furthermore, since the tray detection process is executed when the transport vehicle 200 is actually located near the hydraulic excavator 1 based on the dispatch data received via the communication device 60, the detection accuracy of the tray T is also improved.

[0140] (Modification) The embodiments of the present invention are not limited to the above, and include various modified examples and combinations within the scope of the invention. For example, except for the configuration (including the algorithm) for obtaining the essential effect (1) described above, other configurations may be omitted or changed as appropriate. To give some examples, the hydraulic excavator 1 is configured to be operated by an operator in the cab 20, but the hydraulic excavator 1 may be a manned vehicle or an unmanned vehicle as long as the effect (1) is obtained. In the case of an unmanned vehicle, the design is changed so that the processing device 54 receives an operation signal received by wireless communication from a control station or the like, instead of an operation signal associated with the operation of the operator on board. In addition, a type that scans an object by swinging an optical axis can also be applied as the measuring device 70. In this case, a matrix that dynamically changes according to the movement of the optical axis is applied as a rotation matrix Rsv for converting coordinates from the sensor coordinate system 300 to the vehicle body coordinate system 400. Furthermore, the threshold value Lth that is compared with the detected tray length Lrecg when correcting the tray length, and the set tray length Ldump that is the correction target, do not have to be fixed values, but may be values ​​that are set based on, for example, log data of the tray detection results of past tray detection processes. [Explanation of symbols]

[0141] Reference Signs List 1... hydraulic excavator (working machine), 2... front working machine, 5... running body, 7... rotating body, 10... bucket, 19... rotation angle sensor, 20... operator's cab, 52e, 52f... sensor (running sensor), 54... processing device, 55... monitor, 60... communication device, 70... measuring device, 200... transport vehicle, Ldump... set tray length, Lth... threshold, T... tray

Claims

1. In a work machine that loads transported goods onto a tray of a transport vehicle, A running body, A rotating body rotatably provided on an upper portion of the traveling body; A front working machine attached to the rotating body; a measurement device that measures objects around the work machine and outputs a point cloud indicative of the measurement results; a processing device that executes a detection process for the tray from the point cloud output by the measuring device; a monitor connected to the processing device and displaying the point cloud output from the measuring device; The processing device includes: Extracting two horizontally opposing surfaces from the point cloud output by the measurement device; determining whether the distance between the two extracted surfaces is longer than a preset distance; When it is determined that the inter-surface distance is longer than the set distance, the tray having the two surfaces as left and right wall surfaces is detected; When the tray is detected, two line segments forming the upper end faces of each of the two opposing sides and the start points and end points of each of the line segments are superimposed on the point cloud output from the measuring device and displayed on the monitor, thereby outputting the position of the tray, and displaying information indicating that the tray is being detected on the monitor while the tray is being detected. A working machine characterized by:

2. 2. The work machine according to claim 1, A work machine characterized in that the set distance is set to be greater than the width of a bucket of the front work machine.

3. 2. The work machine according to claim 1, The processing device includes: Detecting an object having a size equal to or larger than a preset size or an object including a circular shape corresponding to a running wheel as the transport vehicle from the point cloud output by the measurement device; Extract two horizontally opposing surfaces from the point cloud of the transport vehicle. A working machine characterized by:

4. 2. The work machine according to claim 1, A work machine characterized in that the processing device interrupts the tray detection process when a point cloud is not input from the measuring device and when the transport vehicle is not detected.

5. 2. The work machine according to claim 1, a rotation angle sensor for detecting an angle of the rotating body relative to the traveling body, The measuring device is installed on the rotating body, The processing device interrupts the detection process for the tray when it is determined that the rotating body is rotating based on the output of the rotation angle sensor. A working machine characterized by:

6. 2. The work machine according to claim 1, A travel sensor is provided to detect travel of the travel object, The processing device interrupts the tray detection process when it is determined that the work machine is traveling based on the output of the travel sensor. A working machine characterized by:

7. 2. The work machine according to claim 1, The processing device includes: The latest detection data of the tray obtained in the detection process of the tray is stored; Even if the tray is no longer detected, the latest detection data is output as long as the transport vehicle is detected. A working machine characterized by:

8. 2. The work machine according to claim 1, The processing device includes: The lengths of the extracted two surfaces in the longitudinal direction are compared; If the longitudinal lengths of the two faces are different, the detection data of the shorter face is corrected to match the detection data of the longer face. A working machine characterized by:

9. 2. The work machine according to claim 1, The processing device includes: A preset threshold value is stored as the length of the tray in the longitudinal direction; The detected length of the tray in the longitudinal direction is compared with the threshold value; If the detected length of the tray is shorter than the threshold, the length of the tray is corrected to a predetermined set tray length. A working machine characterized by:

10. 10. The work machine according to claim 9, The processing device corrects the detected length of the tray by extending it forward of the transport vehicle.

11. 2. The work machine according to claim 1, A work machine characterized in that the measuring device is installed on the rotating body.

12. 2. The work machine according to claim 1, a communication device for receiving position data of the transport vehicle; The processing device executes the tray detection process only when it is determined that the transport vehicle is located in a preset area based on the location data of the transport vehicle received by the communication device. A working machine characterized by:

Images