Work machine

Abstract

The purpose of the present invention is to improve the recognition accuracy of an object other than terrain, such as a transport vehicle, by generating a map in which the latest terrain of a work site is recorded while preventing the object from being recorded in the map. An information processing device for a hydraulic shovel includes: a work region estimation unit that estimates, from among regions in a terrain map, a work region in which the terrain will change as a result of the operation of the hydraulic shovel; an update region determination unit that classifies the regions in the terrain map into update regions in which the terrain height is to be updated and non-update regions in which the terrain height is not to be updated; a point group separation unit that performs separation into a point group representing the terrain and a point group representing objects other than the terrain on the basis of the terrain heights recorded in the non-update regions and point group data corresponding to the non-update regions; and a map update unit that updates the terrain heights recorded in the update regions using point group data from which the point group representing objects other than the terrain has been excluded.

Description

Working machinery

[0001] This invention relates to work machinery such as hydraulic excavators.

[0002] A type of work machine known as a hydraulic excavator is equipped with a self-propelled vehicle, a slewing body rotatably mounted to the vehicle, and a multi-jointed work device rotatably mounted to the slewing body. The work machine is used for loading excavated material such as soil or minerals (hereinafter referred to as soil, etc.) onto transport vehicles such as dump trucks. The loading operation includes a transport operation in which the slewing body is rotated relative to the vehicle to transport the soil, etc., and a discharge operation in which the work device is operated to discharge the transported soil, etc., onto the transport vehicle.

[0003] When loading, if the working device (for example, the height of the bucket tip) is lower than the vessel of the transport vehicle when the rotating body is rotated, the working device may interfere with the transport vehicle during the transport process, potentially damaging either the working device or the transport vehicle. Therefore, working devices are required to have functions that assist the operator in operating the device during loading, or functions that automate the loading process. In order to perform loading operations semi-automatically or automatically, it is necessary to accurately recognize the position and orientation (angle) of the transport vehicle relative to the working device.

[0004] To recognize the position and orientation of transport vehicles, it is necessary to extract moving objects such as transport vehicles from measurement data of the work site measured by external environment recognition sensors such as LiDAR mounted on the work machinery. One technique for extracting moving objects from measurement data of the work site measured by external environment recognition sensors is the background difference method, which calculates the difference between map data of the target environment such as the work site measured in advance and data measured by LiDAR mounted on the work machinery.

[0005] Patent Document 1 describes a technique for detecting changes in the work environment by calculating the difference between a 3D map of the work site measured in advance and the latest measurement data measured at the work site, and for resolving the discrepancy between the work site and the 3D map by updating the 3D map of the changed area with the measurement data.

[0006] Japanese Patent Publication No. 2021-196489

[0007] However, in mining work sites, the terrain changes moment by moment due to excavation or soil removal, making it difficult to extract only moving objects using a map of the target environment measured in advance. Therefore, in order to accurately extract moving objects using background subtraction at a work site, it is necessary to generate a map that reflects only the changing terrain of the work site (for example, the shape of the ground surface such as the ground, slopes, or excavation surfaces, as well as features such as buildings).

[0008] In the technology described in Patent Document 1, all areas that have changed in the field environment are updated based on the difference between the map and the latest measurement data of the work site, so even stationary moving objects are recorded on the map as part of the terrain. Therefore, with the technology described in Patent Document 1, it is difficult to accurately extract moving objects using the background subtraction method.

[0009] This invention has been made in view of the circumstances described above, and aims to improve the recognition accuracy of objects by generating a map that records the latest topography of the work site while preventing objects other than terrain, such as transport vehicles, from being recorded on the map.

[0010] To solve the aforementioned problems, the present invention provides a work machine comprising: a traveling body; a slewing body rotatably attached to the traveling body; and a work device attached to the slewing body having a boom, arm, and bucket, wherein the work machine comprises: an attitude detection device for detecting the attitude of each part of the work machine; a measuring device for measuring objects around the work machine and acquiring point cloud data; and a storage unit for storing a topographic map that records the terrain height around the work machine, and an information processing device for performing recognition processing of objects around the work machine based on the point cloud data acquired by the measuring device and the topographic map stored in the storage unit, wherein the information processing device performs recognition processing of objects around the work machine from within the area of ​​the topographic map based on the position information of each part of the work machine calculated from the detection results of the attitude detection device, The system is characterized by comprising: a work area estimation unit that estimates a work area which is an area in which the terrain changes; an update area determination unit that, based on the estimation results of the work area estimation unit, classifies the areas in the topographic map into update areas which the terrain height is updated and non-update areas which are not updated; a point cloud separation unit that, based on the terrain height recorded in the non-update areas of the topographic map classified by the update area determination unit and the point cloud data corresponding to the non-update areas, separates the point cloud data acquired by the measuring device into a point cloud representing the terrain and a point cloud representing objects other than the terrain; and a map update unit that updates the terrain height recorded in the update area using the point cloud data from which the point cloud representing objects other than the terrain separated by the point cloud separation unit has been excluded, and stores the updated topographic map in the storage unit.

[0011] According to the present invention, it is possible to generate a map that records the latest topography of the work site while preventing objects other than terrain, such as transport vehicles, from being recorded on the map, thereby improving the recognition accuracy of such objects. Other problems, components, and effects will be clarified by the following description of embodiments.

[0012] A side view of a hydraulic excavator, an example of a work machine. A diagram showing the hydraulic system and control system mounted on the hydraulic excavator. A plan view showing an example of a hydraulic excavator operation. A side view showing an example of a hydraulic excavator operation. A side view showing the reference coordinate system together with the hydraulic excavator. A plan view showing the reference coordinate system together with the hydraulic excavator. A flowchart showing the loading control process executed by the control device. A functional block diagram of the information processing device according to the first embodiment. A diagram showing a grid map representing a topographic map. A flowchart showing the topographic map update process and point cloud separation process executed by the information processing device. A flowchart showing the details of step S204 in Figure 10. A flowchart showing the details of step S205 in Figure 10. A diagram showing a specific example of the topographic map update process during excavation and loading onto a transport vehicle. A diagram showing a specific example of the topographic map update process during excavation and soil discharge onto the ground. A diagram showing a specific example of the topographic map update process during ground leveling. A diagram showing a specific example of the topographic map update process during spotting. A diagram showing a specific example of the topographic map update process during jack-up turns. A diagram showing a specific example of the topographic map update process when leveling soil loaded onto a transport vehicle. A diagram showing a specific example of the point cloud separation process that separates the point cloud representing the transport vehicle from the point cloud data acquired by the measuring device. A functional block diagram of the information processing device according to the second embodiment. A flowchart showing the details of step S205 according to the second embodiment. A diagram showing a specific example of the topographic map update process when the transport vehicle passes through the update area. A functional block diagram of the information processing device according to the third embodiment. A flowchart showing the stop position calculation process executed by the information processing device. A diagram showing a specific example of the topographic map update process when spotting using a topographic map.

[0013] Embodiments of the present invention will be described below with reference to the drawings. In each embodiment, components denoted by the same reference numerals are similar in each embodiment unless otherwise specified, and their descriptions will be omitted.

[0014] [First Embodiment] -Work Machine- Figure 1 is a side view of a hydraulic excavator 1, which is an example of a work machine. In this specification, the left direction in Figure 1 is considered the front of the hydraulic excavator 1. Although Figure 1 illustrates a relatively large hydraulic excavator 1 as an example, the work machine of the present invention may be a hydraulic excavator 1 of various sizes, for example, a medium-sized or smaller hydraulic excavator 1. Alternatively, the work machine of the present invention is applicable to other types of work machines other than hydraulic excavators that have a work device 2.

[0015] The hydraulic excavator 1 (working machine) shown in Figure 1 performs excavation work to excavate objects such as mines at the work site, and loading work to load the excavated soil and sand onto a transport vehicle 200 (see Figure 3), such as a dump truck. Furthermore, the hydraulic excavator 1 also performs work to discharge the excavated soil and sand onto the ground around the hydraulic excavator 1, and leveling work using the bucket 10 or blade 16. Loading work includes transport and discharge operations. Transport operation is, for example, the operation of transporting soil and sand scooped up in the bucket 10 to above the transport vehicle 200 by rotating the slewing body 7. Discharge operation is, for example, the operation of discharging soil and sand from the bucket 10 onto the vessel 201 of the transport vehicle 200.

[0016] The hydraulic excavator 1 comprises a body 3 and a multi-jointed work device (front work device) 2 mounted on the body 3. The body 3 includes a traveling body 5 and a slewing body 7. The traveling body 5 is the lower structure of the hydraulic excavator 1 and is equipped with crawler-type traveling devices 5a on both sides of the traveling body 5. The traveling devices 5a are driven by a traveling motor 4 (hydraulic motor). The traveling body 5 moves driven by the traveling motor 4 of the traveling devices 5a. The slewing body 7 is rotatably mounted to the traveling body 5 via a slewing device (not shown). The slewing body 7 rotates relative to the traveling body 5 driven by a slewing motor 6 (hydraulic motor) of the slewing device. The slewing body 7 is provided with an operator's cab 21. The slewing body 7 is also equipped with the hydraulic system and control system of the hydraulic excavator 1, which will be described later.

[0017] Some medium-sized or smaller hydraulic excavators 1 are equipped with a blade 16. The present invention can be applied to medium-sized or smaller hydraulic excavators 1, or to various types of work machines having a work device 2. When the blade 16 is mentioned in the description of this embodiment, the blade 16 is assumed to be located in front of the traveling body 5 as shown in Figure 1.

[0018] The working device 2 is attached to the front of the slewing body 7 and rotates with the slewing body 7 relative to the traveling body 5. The working device 2 has a boom 8, an arm 9, and a bucket 10. The boom 8 is rotatably connected to the front of the slewing body 7 via a boom pin 8a (see Figure 5). The boom 8 is moved vertically by the extension and retraction of the boom cylinder 11. The arm 9 is rotatably connected to the tip of the boom 8 via an arm pin 9a. The arm 9 is moved in the cloud direction and dump direction by the extension and retraction of the arm cylinder 12. The bucket 10 is rotatably connected to the tip of the arm 9 via a bucket pin 10a. The bucket 10 is moved in the cloud direction and dump direction by the extension and retraction of the bucket cylinder 13. The boom 8, arm 9, and bucket 10 are driven members (front members) driven by hydraulic cylinders 11 to 13.

[0019] A boom angle sensor 14 (see Figure 5) is attached to the boom pin 8a to detect the angle of the boom 8 relative to the slewing body 7. An arm angle sensor 15 is attached to the arm pin 9a to detect the angle of the arm 9 relative to the boom 8. A bucket angle sensor 17 is attached to the bucket pin 10a to detect the angle of the bucket 10 relative to the arm 9. These angle sensors are configured, for example, by potentiometers.

[0020] The angles of the boom 8, arm 9, and bucket 10 may also be obtained by detecting the acceleration acting on the boom 8, arm 9, and bucket 10 using an inertial measurement unit (IMU) and converting these detected values. Alternatively, the angles of the boom 8, arm 9, and bucket 10 may also be obtained by detecting the strokes of the boom cylinder 11, arm cylinder 12, and bucket cylinder 13 using stroke sensors and converting these detected values.

[0021] The rotating body 7 is equipped with an inclination angle sensor 18 that detects the inclination angle of the rotating body 7 with respect to a reference plane DP such as a horizontal plane. The rotating device connecting the traveling body 5 and the rotating body 7 is equipped with a rotation angle sensor 19 that detects the rotation angle, which is the relative angle of the rotating body 7 with respect to the traveling body 5. The rotating body 7 is equipped with an angular velocity sensor 20 that detects the angular velocity of the rotating body 7.

[0022] The rotating body 7 is equipped with a measuring device 70 that measures objects around the hydraulic excavator 1. The measuring device 70 acquires point cloud data representing the measurement results. The measuring device 70 is a distance measuring sensor that measures the distance (depth) to objects present around the hydraulic excavator 1. The measuring device 70 can be any device capable of acquiring point cloud data, and can be configured, for example, as a LiDAR (Light Detection And Ranging) or a stereo camera. One or more measuring devices 70 are attached to the hydraulic excavator 1. Preferred arrangements of the measuring devices 70 will be described later.

[0023] - Hydraulic System and Control System - Figure 2 shows the hydraulic system and control system mounted on the hydraulic excavator 1. As shown in Figure 2, the hydraulic excavator 1 comprises a prime mover 103, a hydraulic system, and a control system. The hydraulic system includes a main pump 102, a pilot pump 104, electromagnetic proportional valves 47a to 47l, and a flow control valve 101. The control system includes a control device 40, an information processing device 54, a posture detection device 53, a touch sensor 56, a transport volume calculation device 80, a measuring device 70, and sensors 52a to 52f of the operating devices 22 and 23.

[0024] The prime mover 103 is, for example, an engine (it may also be an electric motor) and drives the main pump 102 and pilot pump 104, which are hydraulic pumps. The control device 40 controls the operation of the work device 2, the slewing body 7, and the traveling body 5 in accordance with the operation signals input from the operating devices 22 and 23. In other words, the operating devices 22 and 23 are instruction devices that instruct the operation of the work device 2, the slewing body 7, and the traveling body 5.

[0025] The operating devices 22 and 23 are located inside the operator's cab 21 of the slewing body 7. Operating device 22 has a right operating lever 22a and a left operating lever 22b. Operating device 23 has a right travel lever 23a and a left travel lever 23b. The right travel lever 23a is used to operate the travel motor 4 of the right travel device 5a. The left travel lever 23b is used to operate the travel motor 4 of the left travel device 5a. The right operating lever 22a is used to operate the boom cylinder 11 and the bucket cylinder 13. The left operating lever 22b is used to operate the arm cylinder 12 and the slewing motor 6. The operating devices 22 and 23 are electric lever type, and their sensors 52a to 52f (e.g., rotary encoder, potentiometer) detect the amount and direction of lever operation by the operator, and output an operation signal corresponding to the amount and direction of operation.

[0026] The control device 40 calculates a command signal based on the operation signals input from sensors 52a to 52f in response to the operation of the operating devices 22 and 23 by the operator, and outputs it to the electromagnetic proportional valves 47a to 47l. The electromagnetic proportional valves 47a to 47l operate in response to the command signal from the control device 40, reducing the pressure of the pressurized oil supplied from the pilot pump 104 via the pilot line 105 and outputting pilot pressure to the flow control valve 101.

[0027] Solenoid proportional valves 47a and 47b output pilot pressure related to the operation of the swing motor 6. Solenoid proportional valves 47c and 47d output pilot pressure related to the operation of the arm cylinder 12. Solenoid proportional valves 47e and 47f output pilot pressure related to the operation of the boom cylinder 11. Solenoid proportional valves 47g and 47h output pilot pressure related to the operation of the bucket cylinder 13. Solenoid proportional valves 47i and 47j output pilot pressure related to the operation of the right-side travel motor 4. Solenoid proportional valves 47k and 47l output pilot pressure related to the operation of the left-side travel motor 4.

[0028] The flow control valve 101 has directional control valves (not shown) corresponding to the slewing motor 6, arm cylinder 12, boom cylinder 11, bucket cylinder 13, and left and right travel motors 4. Each directional control valve operates by receiving pilot pressure input from the corresponding valve among the electromagnetic proportional valves 47a to 47l in a pressure receiving chamber. Each directional control valve controls the direction and flow rate of pressurized oil supplied from the main pump 102 and supplies it to the corresponding hydraulic actuator. As a result, the slewing motor 6, arm cylinder 12, boom cylinder 11, bucket cylinder 13, and left and right travel motors 4 operate in response to the operation of the control devices 22 and 23 by the operator. Therefore, when the control devices 22 and 23 are operated, the position and angle of the bucket 10 changes, the slewing body 7 rotates, and the travel body 5 moves.

[0029] The information processing device 54 is a computer and has the function of performing recognition processing (described later) of the transport vehicle 200 based on point cloud data acquired by the measuring device 70. The information processing device 54 has a ROM (Read Only Memory) 71, a RAM (Random Access Memory) 72, a CPU (Central Processing Unit) 73, an I / F (Interface) 74, and a storage device 57, etc. The storage device 57 is, for example, a hard disk drive or a large-capacity flash memory. These hardware components work together to operate the software and realize multiple functions. The information processing device 54 may be composed of one computer or multiple computers.

[0030] ROM 71, RAM 72, CPU 73, and I / F 74 are connected to each other via bus 75. ROM 71 stores a program capable of performing various calculations. That is, ROM 71 is a storage medium (memory device) that can read the program that realizes the functions of this embodiment. RAM 72 is a storage medium (memory device) that temporarily stores the calculation results from CPU 73 and signals input from I / F 74. CPU 73 is a processor that loads the program stored in ROM 71 into RAM 72 and executes the program. CPU 73 performs predetermined calculation processing on data acquired from I / F 74, ROM 71, or RAM 72 according to the program. The input section of I / F 74 converts signals input from various devices (measuring device 70, attitude detection device 53, carrying amount calculation device 80, touch sensor 56, and control device 40, etc.) into data that can be calculated by CPU 73. The output section of the I / F 74 generates an output signal corresponding to the calculation result of the CPU 73 and outputs this signal to various devices (control device 40 and monitor 55, etc.). Note that the hardware configuration of the control device 40 is the same as that of the information processing device 54, so its explanation is omitted.

[0031] The information processing device 54 is connected to the control device 40, monitor 55, touch sensor 56, measuring device 70, attitude detection device 53, and transport volume calculation device 80 via an I / F 74. The monitor 55 is a display device that displays an image representing the operation information of the hydraulic excavator 1 based on display control signals from the information processing device 54. The monitor 55 is, for example, a liquid crystal display monitor and is positioned inside the operator's cab 21 in a location easily visible to the operator sitting in the driver's seat. The monitor 55 is a touch panel monitor with a touch sensor 56 provided on its display screen. The touch sensor 56 is an input device that inputs an input signal to the information processing device 54 in response to the operator's operation. The attitude detection device 53 is composed of a boom angle sensor 14, an arm angle sensor 15, a bucket angle sensor 17, a tilt angle sensor 18, a slewing angle sensor 19, and an angular velocity sensor 20. The posture detection device 53 detects the posture of the work device 2 (for example, the angle of the boom 8, the angle of the arm 9, and the angle of the bucket 10) and the posture of the slewing body 7 (for example, the slewing angle of the slewing body 7 relative to the traveling body 5, and the inclination angle of the slewing body 7 from the reference plane DP), and outputs a signal representing the detection result to the information processing device 54.

[0032] The transport volume calculation device 80 is a device that calculates the weight of soil and other materials in the bucket 10 of the hydraulic excavator 1. The transport volume calculation device 80 includes a pressure sensor that detects the pressure of the arm cylinder 12, a pressure sensor that detects the pressure of the boom cylinder 11, a pressure sensor that detects the pressure of the bucket cylinder 13, and a weight calculation unit. The weight calculation unit is mounted on the slewing body 7. The weight calculation unit calculates the weight of soil and other materials held in the bucket 10 based on the cylinder pressures detected by the multiple pressure sensors and the posture of the work device 2 detected by the posture detection device 53. The weight calculation unit calculates the load capacity of the transport vehicle 200 by adding up the calculated weight of soil and other materials in the bucket 10 each time the bucket 10 is transported from the excavation site to the discharge point on the transport vehicle 200. The load capacity is the total weight of soil and other materials loaded onto the transport vehicle 200 as the loading operation is repeatedly performed on the transport vehicle 200. A signal indicating the timing for adding up the weights of soil and other materials in bucket 10 is input from the information processing device 54. When the loaded weight exceeds a specified amount and the loading operation onto the transport vehicle 200 is completed, the calculated value of the loaded weight is reset (to 0). A signal indicating the timing for resetting the loaded weight is input from the information processing device 54. Note that the weight calculation unit function may also be provided in the information processing device 54.

[0033] -Example of Operation- Figure 3 is a plan view showing an example of operation of the hydraulic excavator 1. Figure 4 is a side view showing an example of operation of the hydraulic excavator 1. Figures 3 and 4 show the loading operation in which the hydraulic excavator 1 excavates soil and other materials, transports the excavated soil and other materials, and loads them onto the vessel 201 of the transport vehicle 200. The transport vehicle 200 stops at the loading position prior to the loading operation. The loading position is a position in which the hydraulic excavator 1 can load soil and other materials. The loading position is set to a position within a range that is within the maximum rotation radius from the rotation centerline 120 (see Figure 5) of the slewing body 7, and is greater than or equal to the rear end rotation radius of the slewing body 7. The loading position is one of the stopping positions (stopping positions of the moving body) of the transport vehicle 200, which is the position in which the transport vehicle 200 is stopped.

[0034] When the transport vehicle 200 is stopped at the loading position, the information processing device 54 of the hydraulic excavator 1 performs recognition processing to recognize the position and orientation of the transport vehicle 200 based on the measurement results (point cloud data) of the measuring device 70. Based on the results of the recognition processing of the information processing device 54, the control device 40 of the hydraulic excavator 1 controls the operation of the hydraulic excavator 1 (for example, the working device 2) so that, for example, the working device 2 does not collide with the vessel 201 during loading operations. The results of the recognition processing are applied not only to loading operations but also to interference prevention control between the transport vehicle 200 and the hydraulic excavator 1 in various operations. For example, the results of the recognition processing are applied to interference avoidance control between the vessel 201 and the working device 2 during reaching, when the bucket 10 of the hydraulic excavator 1 is moved to the next excavation site after loading operations.

[0035] In this embodiment, the information processing device 54 creates and stores a map (hereinafter referred to as a topographic map) in advance, which records the terrain around the hydraulic excavator 1, based on point cloud data acquired in advance by the measuring device 70. The information processing device 54 performs recognition processing of transport vehicles 200 located around the hydraulic excavator 1 based on the point cloud data acquired by the measuring device 70 and the topographic map created in advance. Specifically, the information processing device 54 extracts the point cloud of the transport vehicles 200 by calculating the difference between the topographic map created in advance and the latest point cloud data acquired by the measuring device 70. Then, it performs recognition processing of the transport vehicles 200 on the extracted point cloud. In the recognition processing of the transport vehicles 200, two surfaces, the left and right sides of the vessel 201 of the transport vehicles 200, are extracted. Generally, the inner wall surfaces on both the left and right sides of the vessel 201 of the transport vehicles 200 are formed by flat or smoothly curved surfaces. Therefore, in the recognition process of the transport vehicle 200, the position and orientation of the vessel 201 (transport vehicle 200) can be recognized by extracting the two sides of the vessel 201, both the left and right sides.

[0036] - Coordinate System - FIG. 5 is a side view showing a reference coordinate system together with the hydraulic excavator 1. FIG. 6 is a plan view showing the reference coordinate system together with the hydraulic excavator 1. In the ROM 71 of the information processing device 54, the vehicle body coordinate system 400, the sensor coordinate system 300, and the site coordinate system 500 shown in FIGS. 5 and 6 are stored. The vehicle body coordinate system 400 is a reference coordinate system for specifying the position and orientation (angle) of each part (for example, the bucket 10) which is a component of the hydraulic excavator 1. The vehicle body coordinate system 400 is a local coordinate system of the vehicle body 3 based on the traveling body 5 and is preset.

[0037] In the present embodiment, the vehicle body coordinate system 400 is defined as an XYZ right-handed orthogonal coordinate system with the intersection point of the turning center line 120 which is the rotation axis of the revolving body 7 and the ground contact surface of the traveling body 5 (the bottom surface of the traveling body 5 in contact with the ground GL) as the origin. In the vehicle body coordinate system 400, the forward direction of the traveling body 5 is the positive direction of the X axis, the direction from the traveling body 5 toward the revolving body 7 along the turning center line 120 (upward direction) is the positive direction of the Z axis, and the left side of the traveling body 5 is the positive direction of the Y axis. In the vehicle body coordinate system 400, the turning angle θsw of the revolving body 7 (see FIG. 6) is set to 0 degrees when the working device 2 faces the positive direction of the X axis and the center line of the working device 2 (the dashed-dotted line in FIG. 6) is parallel to the X axis of the vehicle body coordinate system 400.

[0038] The sensor coordinate system 300 is an xyz right-handed orthogonal coordinate system based on the measuring device 70. In the sensor coordinate system 300, the front of the revolving body 7 is the positive direction of the x axis, the left direction of the revolving body 7 is the positive direction of the y axis, and the direction from the traveling body 5 toward the revolving body 7 (upward direction) is the positive direction of the z axis. The site coordinate system 500 is an X'Y'Z' right-handed orthogonal coordinate system based on the work site. In the site coordinate system 500, the vertically upward direction is the positive direction of the Z' axis, an arbitrary direction parallel to the horizontal direction is the positive direction of the X' axis, and one of the directions orthogonal to the X' axis and the Z' axis is the positive direction of the Y' axis.

[0039] - Position and Orientation Calculation - Geometric information used for calculating the position and orientation (angle) of each part (e.g., bucket 10) of the hydraulic excavator 1 is stored in the ROM 71 of the information processing device 54. This geometric information includes sizes such as boom length Lbm, arm length Lam, and bucket length Lbk. The boom length Lbm is the length of the boom 8 and corresponds to the center-to-center distance between the boom pin 8a and the arm pin 9a. The arm length Lam is the length of the arm 9 and corresponds to the center-to-center distance between the arm pin 9a and the bucket pin 10a. The bucket length Lbk is the length of the bucket 10 and corresponds to the distance from the center of the bucket pin 10a to the tip (end) of the bucket 10.

[0040] Based on the detection results of the attitude detection device 53, the information processing device 54 calculates the position and orientation (angle) of each part of the hydraulic excavator 1 in the vehicle body coordinate system 400. For example, based on the detection results of the boom angle sensor 14, the information processing device 54 calculates the angle θbm of the boom 8 with respect to the X-axis. Based on the detection results of the arm angle sensor 15, the information processing device 54 calculates the angle θam of the arm 9 with respect to the boom 8. Based on the detection results of the bucket angle sensor 17, the information processing device 54 calculates the angle θbk of the bucket 10 with respect to the arm 9. Based on the detection results of the swing angle sensor 19, the information processing device 54 calculates the swing angle θsw of the swing body 7 with respect to the traveling body 5. Based on the detection results of the inclination angle sensor 18, the information processing device 54 calculates the inclination angle θg of the swing body with respect to the reference plane DP. The reference plane DP is, for example, a horizontal plane perpendicular to the vertical direction. The information processing device 54 calculates the position of the tip of the bucket 10 (end position) based on the boom angle θbm, arm angle θam, bucket angle θbk, and geometric information (including sizes such as Lbm, Lam, Lbk, etc.). The calculation results of the information processing device 54 are output to the control device 40 and used for control such as loading control by the control device 40.

[0041] -Coordinate Transformation- The information processing device 54 uses the position and orientation data of each part of the hydraulic excavator 1 obtained by position and orientation calculation to transform the point cloud data acquired by the measuring device 70 from data in the sensor coordinate system 300 to data in the vehicle body coordinate system 400. The point cloud data acquired by the measuring device 70 is a set of three-dimensional point data Ps(Xps, Yps, Zps) represented by the sensor coordinate system 300. The point data Ps(Xps, Yps, Zps) in the sensor coordinate system 300 is transformed into point data Pv(Xpv, Ypv, Zpv) in the vehicle body coordinate system 400 using the following equations 1 to 3.

[0042]

[0043]

[0044]

[0045] Here, Rsv is a rotation matrix that transforms the coordinates of the sensor coordinate system 300 to the coordinates of the vehicle body coordinate system 400, and αs, βs, and γs in the rotation matrix Rsv are the inclination angles of the measuring device 70 with respect to the x, y, and z axes in the vehicle body coordinate system 400. When the measuring device 70 is fixed to the hydraulic excavator 1, αs, βs, and γs can be obtained, for example, by measuring the position and orientation of the measuring device 70 in the vehicle body coordinate system 400, and can be stored in advance in the ROM 71 or storage device 57. θsw is the rotation angle of the rotating body 7, and is obtained by position and orientation calculations.

[0046] Tsv is a translation vector that starts at the origin of the vehicle body coordinate system 400 and ends at the origin of the sensor coordinate system 300. The components Lsx, Lsy, and Lsz of Tsv are equal to the coordinates of the origin of the sensor coordinate system 300 in the vehicle body coordinate system 400. When the measuring device 70 is fixed, the coordinates of the origin of the sensor coordinate system 300 in the vehicle body coordinate system 400 are immovable, so Tsv (Lsx, Lsy, Lsz) can be measured in advance and stored in the ROM 71 or storage device 57.

[0047] Figure 7 is a flowchart showing the loading control process performed by the control device 40. The process shown in Figure 7 is started, for example, when the ignition switch is turned on, and is repeatedly executed at a predetermined control cycle.

[0048] In step S101, the control device 40 performs a process to confirm that the transport vehicle 200 is stopped. In the process to confirm that the transport vehicle 200 is stopped, the control device 40 determines whether or not the transport vehicle 200 is stopped at a position where soil and sand can be loaded (loading position). For example, if the control device 40 receives a stop notification signal for the transport vehicle 200 from the control system of the transport vehicle 200, it determines that the transport vehicle 200 is stopped at the loading position. If the control device 40 does not receive a stop notification signal, it determines that the transport vehicle 200 is not stopped at the loading position. The hydraulic excavator 1 is equipped with a communication device (not shown) for wireless communication with the control system server. An example of a control system is an FMS (Fleet Management System) used at the work site.

[0049] Furthermore, the control device 40 may determine that the transport vehicle 200 is stopped at the loading position if a stop notification signal is input from the stop confirmation switch (not shown) in response to the operator's operation of the stop confirmation switch. In this case, the operator visually confirms that the transport vehicle 200 is stopped at the loading position and operates the stop confirmation switch. If the control device 40 determines in step S101 that the transport vehicle 200 is stopped at the loading position, that is, if the operator confirms that the transport vehicle 200 is stopped at the loading position, the control device 40 notifies the information processing device 54 of this fact and proceeds to step S102, which is the process shown in Figure 7.

[0050] In step S102, the control device 40 executes a loading operation start detection process. In the loading operation start detection process, the control device 40 determines whether or not the loading operation has been instructed to start. When loading control is performed automatically, that is, when the operator does not operate the work device 2 and the rotating body 7 for the loading operation, if the control device 40 receives a loading start signal from the control system of the transport vehicle 200, it determines that the loading operation has been instructed to start. If the loading start signal has not been received, the control device 40 determines that the loading operation has not been instructed to start. The control device 40 may also determine that the loading operation has been instructed if a loading start signal is input from the loading start switch (not shown) in response to the operator's operation of the loading start switch.

[0051] When loading control is performed semi-automatically, that is, when the control device 40 assists the operator in operating the work device 2 and the slewing body 7, for example, when the operation of the operating devices 22 and 23 is detected by sensors 52a to 52f, the control device 40 determines that the start of loading work has been instructed. For example, at the start of loading work, characteristic actions are performed, such as the slewing body 7 rotating toward the transport vehicle 200 or the boom 8 beginning to rise. Therefore, it is also possible to configure the system to detect the start of loading work from the operation of the operating devices 22 and 23 based on a trained model acquired by deep learning in advance of the operation of the hydraulic excavator 1 associated with the start of loading work. If it is determined in step S102 that the start of loading has been instructed, that is, if the start of loading work is detected, the control device 40 proceeds to step S103 with the processing shown in Figure 7.

[0052] In step S103, the control device 40 sends a request command to the information processing device 54 for a process to extract point cloud data representing objects other than terrain (for example, moving objects such as the transport vehicle 200) from the point cloud data acquired by the measuring device 70. After receiving the request command from the control device 40, the information processing device 54 calculates the difference between the topographic map stored in the storage device 57 and the point cloud data acquired by the measuring device 70, and extracts point cloud data representing objects other than terrain. The extracted point cloud data is stored in the storage device 57 so that it can be used for other processes, such as the recognition process of the transport vehicle 200. The topographic map used here is updated at regular intervals based on point cloud data previously acquired by the measuring device 70. After extracting the point cloud data of objects other than terrain and storing it in the storage device 57, the control device 40 proceeds to step S104 with the process shown in Figure 7.

[0053] In step S104, the control device 40 obtains the result of the recognition process of the transport vehicle 200. In this process, the control device 40 obtains from the information processing device 54 the result of the recognition process for the current position and orientation of the vessel 201, as well as the currently set recognition processing mode, as a result of the recognition process of the transport vehicle 200. Specifically, the control device 40 outputs a request command to the information processing device 54 requesting the execution of the recognition process of the transport vehicle 200. When the information processing device 54 receives the request command, it executes the recognition process of the transport vehicle 200. The information processing device 54 recognizes the current position and orientation of the vessel 201 through the recognition process of the transport vehicle 200. The information processing device 54 outputs the recognized position and orientation of the vessel 201 to the control device 40 as a result of the recognition process of the transport vehicle 200.

[0054] The result of the recognition process of the transport vehicle 200 output from the information processing device 54 to the control device 40 is identified by the body coordinate system 400 of the hydraulic excavator 1. For example, the information processing device 54 outputs to the control device 40 the coordinates in the body coordinate system 400 that represent the four corners of the upper end of the vessel 201. The information processing device 54 also outputs to the control device 40 the coordinates of the origin of the vehicle coordinate system of the transport vehicle 200 in the body coordinate system 400 as the position of the transport vehicle 200 (vessel 201). Furthermore, the information processing device 54 outputs to the control device 40 the angle (azimuth angle) between the X'' axis of the vehicle coordinate system of the transport vehicle 200 and the X axis of the body coordinate system 400 of the hydraulic excavator 1 as the attitude of the transport vehicle 200 (vessel 201). If the result of the recognition process of the transport vehicle 200 is obtained in step S104, the control device 40 proceeds to step S105 with the processing shown in Figure 7.

[0055] In step S105, the control device 40 starts loading control. Loading control causes the hydraulic excavator 1 to repeatedly perform excavation and loading operations. In step S105, the control device 40 controls at least one operation of the slewing body 7 and the working device 2 (boom 8, arm 9, and bucket 10) based on the position and orientation of the transport vehicle 200 acquired in step S104. For example, when a slewing operation signal is input, the control device 40 semi-automatically or automatically controls the working device 2 so that not only slewing but also boom raising operation according to the slewing speed is performed even if the boom raising operation is not performed (or if the operation amount is smaller than appropriate). This loading control allows the working device 2 (especially the bucket 10) to move between the excavation position and the soil discharge position without colliding with the vessel 201. If loading control is started in step S105, the control device 40 proceeds to step S106 with the process shown in Figure 7.

[0056] In step S106, the control device 40 executes a loading completion determination process. In the loading completion determination process, the control device 40 determines whether or not the loading completion conditions have been met. When loading control is performed automatically, if the control device 40 receives a loading completion signal from the control system of the transport vehicle 200, it determines that the loading completion conditions have been met. If the loading completion signal has not been received, the control device 40 determines that the loading completion conditions have not been met. The control device 40 may also determine that the loading completion conditions have been met if a loading completion signal is input from the loading completion switch (not shown) in response to the operator's operation of the loading completion switch. For example, the operator visually checks the loading status of soil, etc., onto the vessel 201 and operates the loading completion switch when the loading operation onto the vessel 201 is complete. Furthermore, the hydraulic excavator 1 may be configured such that when an operator performs a specific operation using an input device such as an operating switch provided on the operating devices 22, 23 or a touch sensor 56 provided on the monitor 55, a loading completion signal is input from the input device to the control device 40.

[0057] The control device 40 may determine that the loading operation has been completed when the total weight of the soil and sand loaded onto the transport vehicle 200 (the load capacity of the transport vehicle 200) reaches a specified value. In this case, the control device 40 obtains the total weight of the soil and sand calculated by the transport volume calculation device 80 from the information processing device 54. The transport vehicle 200 may be equipped with a load capacity measuring instrument (for example, a load cell). In this case, the control device 40 may receive the load capacity measurement result from the transport vehicle 200 using the measuring instrument and determine that the loading operation has been completed when the load capacity reaches a specified value.

[0058] In step S106, if the control device 40 determines that the conditions for completing the loading operation onto the stationary transport vehicle 200 have not been met, it returns to step S102 with the process shown in Figure 7. In other words, the excavation and loading operations by the hydraulic excavator 1 are repeated. If it is determined that the conditions for completing the loading operation have been met, the control device 40 proceeds to step S107 with the process shown in Figure 7.

[0059] In step S107, the control device 40 executes a work completion determination process. In the work completion determination process, the control device 40 determines whether the work by the hydraulic excavator 1 has been completed based on a signal input (or input stop) when, for example, an operator (or manager in the case of an unmanned machine) performs an operation to stop the prime mover 103 of the hydraulic excavator 1 (hereinafter referred to as the stop operation). If it is determined in step S107 that there has been no stop operation and the operation is continuing, the control device 40 returns to step S101 with the process shown in Figure 7 and waits until the next transport vehicle 200 stops at the loading position. If a stop operation is performed, the control device 40 executes a predetermined termination process, including the process of stopping the prime mover 103, and terminates the process shown in Figure 7.

[0060] As described above, in step S104, in order to obtain the result of the recognition process of the transport vehicle 200, in step S103, point clouds representing objects other than terrain are extracted from the point cloud data acquired by the measuring device 70. In order to extract point clouds representing objects other than terrain, the information processing device 54 performs a point cloud separation process to separate the point clouds representing terrain from the point clouds representing objects other than terrain in the point cloud data acquired by the measuring device 70. In order to perform the point cloud separation process, the information processing device 54 performs a topographic map update process that records the terrain around the hydraulic excavator 1. That is, the result of the recognition process of the transport vehicle 200 acquired by the control device 40 in step S104 is the result of the recognition process performed by the information processing device 54 based on the point clouds representing objects other than terrain that were separated by the information processing device 54. The topographic map update function and point cloud separation function of the information processing device 54 will be described in detail below.

[0061] Figure 8 is a functional block diagram of the information processing device 54 according to the first embodiment. The information processing device 54 generates a topographic map in advance using point cloud data acquired by measuring the area around the hydraulic excavator 1 with the measuring device 70. The information processing device 54 has a storage unit (map storage unit 86) that stores the pre-generated topographic map and updates the topographic map in accordance with changes in the terrain around the hydraulic excavator 1. In order to update the topographic map so that the latest terrain of the work site is recorded, while preventing stationary moving objects other than terrain from being recorded on the topographic map, the information processing device 54 separates the point cloud representing terrain from the point cloud representing objects other than terrain in the point cloud data acquired by the measuring device 70, based on the pre-generated topographic map.

[0062] As shown in Figure 8, the information processing device 54 implements its various functions by having the CPU 73 execute a program stored in the ROM 71 using various data calculated or acquired by the transport volume calculation device 80, the attitude detection device 53, and the measurement device 70. These functions include a work area estimation unit 81, an update area determination unit 82, a point cloud separation unit 83, and a map update unit 84. Furthermore, the information processing device 54 implements the functions of the map storage unit 86, the mobile object storage unit 87, and the model storage unit 88, primarily through the storage device 57, the ROM 71, and the RAM 72.

[0063] The work area estimation unit 81 estimates the work area, which is the area in the topographic map where the terrain changes due to the operation of the hydraulic excavator 1, based on the positions of each part of the hydraulic excavator 1 calculated based on the detection results of the posture detection device 53. Specifically, the work area estimation unit 81 estimates the work area where the terrain changes due to the operation and work performed by the hydraulic excavator 1, based on the calculation results of the transport volume calculation device 80, the detection results of the posture detection device 53, and the 3D model of the hydraulic excavator 1 stored in the model storage unit 88. The 3D model stored in the model storage unit 88 reflects the size of each part of the hydraulic excavator 1.

[0064] For example, the work area estimation unit 81 performs a contact determination to determine whether the bucket 10 is in contact with the terrain based on the positional relationship between the bucket 10's position and the terrain in the topographic map, and estimates the work area based on the result of the contact determination. Alternatively, for example, the work area estimation unit 81 estimates the work area based on the weight of the load in the bucket 10 and the position the bucket 10 passes through relative to the terrain in the topographic map.

[0065] Here, the operations performed by the hydraulic excavator 1 to change the terrain include not only excavation and leveling of the terrain and discharge of excavated soil, but also operations in which the bucket 10 of the hydraulic excavator 1 comes into contact with the terrain, such as jack-up turns. The work area estimated by the work area estimation unit 81 is output to the update area determination unit 82.

[0066] The update area determination unit 82 classifies the areas within the topographic map stored in the map storage unit 86 into update areas and non-update areas, based on the estimation results of the work area estimation unit 81. In other words, the update area determination unit 82 determines which areas to update from among the areas included in the topographic map stored in the map storage unit 86, using point cloud data acquired by the measuring device 70. The areas within the topographic map are defined by dividing the work site around the hydraulic excavator 1 into a grid. The work area estimated by the work area estimation unit 81 and the variance of topographic height recorded in association with the grid of the topographic map are used to determine the update area. The variance of topographic height is a value obtained by calculating the variance of multiple topographic heights acquired by measuring the same grid at different times.

[0067] The point cloud separation unit 83 separates the point cloud data acquired by the measuring device 70 into a point cloud representing terrain and a point cloud representing objects other than terrain, based on the terrain height and the distribution of terrain height recorded in the non-updated area of ​​the topographic map classified by the update area determination unit 82, and the point cloud data from the point cloud data acquired by the measuring device 70 that corresponds to the non-updated area. For example, the point cloud separation unit 83 compares the point cloud measured in the non-updated area classified by the update area determination unit 82 with the terrain height recorded in the non-updated area of ​​the topographic map, and if there is a difference between the two, the point cloud measured in the non-updated area can be separated as a point cloud representing objects other than terrain. In this embodiment, the point cloud separation unit 83 separates the point cloud data acquired by the measuring device 70 into a point cloud representing terrain and a point cloud representing objects other than terrain by extracting outliers present in the point cloud data corresponding to the non-updated area of ​​the topographic map classified by the update area determination unit 82 using the terrain height and the distribution of terrain height recorded in the said non-updated area. In this embodiment, objects other than terrain are represented as moving objects such as transport vehicles 200. The point clouds representing the moving objects separated by the point cloud separation unit 83 are stored in the moving object storage unit 87.

[0068] The map update unit 84 updates the update area of ​​the topographic map, which has been classified by the update area determination unit 82, based on the point cloud data acquired by the measuring device 70. The updated topographic map is stored in the map storage unit 86. Specifically, the map update unit 84 updates the topographic height recorded in the update area of ​​the topographic map using the point cloud data from which point clouds representing objects other than topography that have been separated by the update area determination unit 82 have been excluded, and stores the updated topographic map in the map storage unit 86.

[0069] The map storage unit 86 stores the topographic map. For each region within the topographic map, the topographic height, the distribution of topographic height, and the final measurement time, which indicates the time when the measuring device 70 last measured the region, are recorded. The topographic map is represented by a grid map as shown in Figure 9. The grids within the grid map correspond to regions within the topographic map. The origin of the grid map is represented by the origin of the field coordinate system 500. The length of one side of the grid map is expressed as Ld for the length in the X' direction and Wd for the length in the Y' direction. The length of one side of a grid is the value obtained by dividing the length of one side of the grid map by the number of grids N and M. The topographic height, the distribution of topographic height, and the final measurement time are recorded in the grid. The origin and the length of one side of the grid map are stored in the storage device 57. The topographic map is initialized when the control device 40 is started. Initialization here means, for example, setting the origin of the field coordinate system 500 stored in the storage device 57 as the origin of the grid map, setting the length of one side of the grid based on the length of one side of the grid map and the number of grids stored in the storage device 57, and setting initial values ​​for each recording area, such as terrain height, distribution of terrain height, and final measurement time, which are recorded in the grid. In this embodiment, it is assumed that each initial value is the maximum value that can be recorded in the variable of each recording area.

[0070] The mobile object storage unit 87 stores the point cloud representing the mobile object separated by the point cloud separation unit 83 (hereinafter referred to as the mobile object point cloud), and the results of the recognition process for the object, such as the transport vehicle 200, which is the mobile object.

[0071] The model storage unit 88 stores 3D models of each part of the hydraulic excavator 1, such as the boom 8, arm 9, and bucket 10. The 3D models stored in the model storage unit 88 include the dimensions of each part of the hydraulic excavator 1.

[0072] Next, the topographic map update process and point cloud separation process performed by the information processing device 54 will be described. Figure 10 is a flowchart showing the topographic map update process and point cloud separation process performed by the information processing device 54.

[0073] In the explanations from Figure 10 onward, the topographic map is represented by a grid map, and the areas within the topographic map are defined by the grid. Furthermore, as an example of a moving object other than terrain, a transport vehicle 200 will be used.

[0074] In step S201, the information processing device 54 acquires the point cloud data obtained by the measuring device 70. Subsequently, the information processing device 54 proceeds to step S202 with the processing shown in Figure 10.

[0075] In step S202, the information processing device 54 acquires the position and orientation of each part of the hydraulic excavator 1 calculated based on the detection results of the orientation detection device 53. In particular, the information processing device 54 acquires the position and orientation of the boom 8, arm 9, bucket 10, and slewing body 7 in the vehicle coordinate system 400.

[0076] In step S203, the information processing device 54 converts the point cloud data acquired by the measuring device 70 into an arbitrary coordinate system. For example, when converting the point cloud data acquired by the measuring device 70 into the field coordinate system 500, the information processing device 54 uses the position and orientation of each measuring device 70 relative to the origin of the vehicle coordinate system 400 to convert from the coordinates in the sensor coordinate system 300 to the coordinates in the vehicle coordinate system 400. After that, the information processing device 54 uses the position and orientation of the hydraulic excavator 1 in the field coordinate system 500 to convert from the coordinates in the vehicle coordinate system 400 to the coordinates in the field coordinate system 500. The parameters used for coordinate conversion, namely the x, y, and z positions of each measuring device 70 relative to the rotating body 7 and the inclination angles for each of the x, y, and z axes, are stored in advance in the storage device 57 or ROM 71. Furthermore, the position and orientation of the hydraulic excavator 1 relative to the site coordinate system 500 are calculated using GNSS by integrating the amount of change obtained when the point cloud data acquired by the measuring device 70 is aligned using ICP (Iterative Closest Point), etc. After converting to an arbitrary coordinate system, the information processing device 54 proceeds to step S204 with the processing shown in Figure 10.

[0077] In step S204, the information processing device 54 estimates the work area from the grid of the topographic map. This process will be described later using the flowchart shown in Figure 11.

[0078] In step S205, the information processing device 54 classifies the grid of the topographic map into updated areas and non-updated areas. This process will be described later using the flowchart shown in Figure 12. In this embodiment, the process in step S205 is performed on all grids (all areas) within the topographic map.

[0079] In step S206, the information processing device 54 assigns the point cloud data acquired by the measuring device 70 to the grid of the topographic map. To assign the point cloud data to the grid of the topographic map, the point cloud data is coordinate-transformed to the same coordinate system as the topographic map, and the X and Y coordinate values ​​are divided by the grid size. After assigning the point cloud data to the grid, the information processing device 54 proceeds to step S207 with the processing shown in Figure 10.

[0080] In step S207, the information processing device 54 determines whether the grid cell in the topographic map is classified as an updated area or a non-updated area. If the grid cell in the topographic map is classified as an updated area, the information processing device 54 proceeds to step S208 with the process shown in Figure 10. If the grid cell in the topographic map is classified as a non-updated area, the information processing device 54 proceeds to step S209 with the process shown in Figure 10.

[0081] In step S208, the information processing device 54 updates the terrain height, terrain height variance, and last measurement time recorded in the grid using the point cloud data acquired by the measuring device 70 and assigned to the grid classified as the update area. The terrain height is calculated using the following formula 4. The terrain height variance is calculated using the following formula 5. In this embodiment, it is assumed that the average value of the point cloud assigned to the grid is input to formulas 4 and 5 to calculate the terrain height and terrain height variance. Alternatively, the point cloud may be repeatedly input to formulas 4 and 5 for the number of point clouds assigned to the grid to calculate the terrain height and terrain height variance. Here, μ １：ｔ μ indicates the updated terrain elevation. １：ｔ－1represents the terrain height before update. σ ２ １：ｔ represents the variance of the terrain height after update. σ ２ １：ｔ－1 represents the variance of the terrain height before update. σ ２ ｔ represents the variance generated in the measurement value of the sensor (measurement device 70). z ｔ represents the height of the assigned point cloud.

[0082] Update formula for terrain height

[0083] Update formula for variance of terrain height

[0084] In this embodiment, although the method of calculating the terrain height and the variance of the terrain height by Formula 4 and Formula 5 has been described, the information processing device 54 may, for example, store the height of the point cloud assigned for each grid, and calculate the terrain height and the variance of the terrain height from the stored point cloud each time new point cloud data is assigned.

[0085] In step S209, the information processing device 54 performs a rejection test on the point cloud assigned to the grid classified as the non-update area among the point cloud data acquired by the measurement device 70. The Mahalanobis distance used for the rejection test is calculated using the following Formula 6. Here, Dm represents the Mahalanobis distance. p ｚ ｉ represents the height of the i-th point assigned to the grid. μ １：ｔ represents the terrain height recorded in the grid. σ １：ｔ represents the standard deviation calculated from the variance of the terrain height recorded in the grid.

[0086] Calculation formula for Mahalanobis distance (one-dimensional)

[0087] The information processing device 54 performs a rejection test based on the chi-square distribution using the calculated Mahalanobis distance. For example, when the value of Dm exceeds 3.84, the information processing device 54 classifies the point p ｚ ｉ assigned to the grid as an outlier.

[0088] In step S210, the information processing device 54 determines whether or not outliers exist in the point cloud assigned to the grid classified as a non-updated area. If outliers exist, the information processing device 54 proceeds to step S211 with the process shown in Figure 10. If no outliers exist, the information processing device 54 executes the process in step S207 for the next grid. In this embodiment, a method for determining outliers has been described using terrain height and the variance of terrain height, but the information processing device 54 may, for example, consider points that have a difference exceeding a preset threshold from the terrain height recorded for each grid as outliers.

[0089] In step S211, the information processing device 54 extracts the points determined to be outliers and stores them as a moving point cloud. Then, the information processing device 54 performs the process in step S207 for the next grid cell. After performing the processes in steps S207 to S211 for all grid cells of the topographic map, the information processing device 54 proceeds to step S212, which is the process shown in Figure 10.

[0090] In step S212, the information processing device 54 groups the accumulated moving point clouds. The grouping of the moving point clouds is performed using a known method such as k-means clustering.

[0091] In step S213, the information processing device 54 stores the moving object point cloud grouped in step S212 in the moving object storage unit 87. After that, the information processing device 54 proceeds to step S214 with the processing shown in Figure 10.

[0092] In step S214, the information processing device 54 stores in the map storage unit 86 a topographic map that records the topographic elevation and the distribution of topographic elevation updated in step S208. After that, the information processing device 54 terminates the process shown in Figure 10.

[0093] Figure 11 is a flowchart detailing step S204 of Figure 10. In step S301, the information processing device 54 determines whether or not soil or other material is present in the bucket 10. This determination uses a value calculated by the transport volume calculation device 80. If soil or other material is present in the bucket 10, the information processing device 54 proceeds to step S302 with the process shown in Figure 11. If soil or other material is not present in the bucket 10, the information processing device 54 proceeds to step S305 with the process shown in Figure 11.

[0094] In step S302, if the information processing device 54 has already recognized the transport vehicle 200 that is present around the hydraulic excavator 1, it reads the position and orientation of the transport vehicle 200 (vessel 201) from the mobile object storage unit 87, which are included in the results of the transport vehicle 200 recognition process. After that, the information processing device 54 proceeds to step S303 with the process shown in Figure 11.

[0095] In step S303, the information processing device 54 determines whether or not the vessel 201 of the transport vehicle 200 is located directly beneath the bucket 10. If the vessel 201 is located directly beneath the bucket 10, the information processing device 54 determines that no change in terrain will occur due to the falling of soil and other materials from the bucket 10, and therefore terminates the process shown in Figure 11. After that, the information processing device 54 executes step S205 in Figure 10. On the other hand, if the vessel 201 is not located directly beneath the bucket 10, the soil and other materials from the bucket 10 will fall, causing a change in terrain, and therefore the information processing device 54 proceeds to step S304 of the process shown in Figure 11.

[0096] In step S305, the information processing device 54 calculates the positional relationship between the bucket 10 or blade 16 and the terrain. These positional relationships are calculated, for example, as follows: The information processing device 54 calculates an approximate surface that approximates the surface of the terrain by connecting the terrain heights recorded in the grid of the terrain map. Then, the information processing device 54 calculates how close each surface of the bucket 10 or blade 16 is to the calculated approximate surface of the terrain.

[0097] In step S306, the information processing device 54 calculates the intersection position between each surface of the bucket 10 or blade 16 and the approximate surface of the terrain, and determines whether each surface of the bucket 10 or blade 16 is in contact with the approximate surface of the terrain. The method for calculating this intersection position is well known, so its explanation is omitted. In this embodiment, the contact determination between the bucket 10 or blade 16 and the terrain is performed by the method described above, but the contact determination may also be performed, for example, by utilizing the reaction force generated when the bucket 10 or blade 16 comes into contact with the terrain. If it is determined that each surface of the bucket 10 or blade 16 is in contact with the approximate surface of the terrain, the information processing device 54 proceeds to step S304 with the process shown in Figure 11. If it is determined that each surface of the bucket 10 or blade 16 is not in contact with the approximate surface of the terrain, the information processing device 54 terminates the process shown in Figure 11. After that, the information processing device 54 executes step S205 in Figure 10.

[0098] In step S304, if the user has transitioned from step S303, the information processing device 54 estimates the grids on the topographic map that are directly below and around the bucket 10 as the work area. The grids estimated as the work area may be only the grids directly below the bucket 10, or they may include grids within a certain range from the grids directly below the bucket 10. Also, in step S304, if the user has transitioned from step S306, the information processing device 54 estimates the grids on the topographic map that record the topographic height of the vertex of the approximate topographic surface that is determined to be in contact with the bucket 10 or blade 16, and the grids around this grid, as the work area. Similarly, in this case as well, the grids estimated as the work area may include grids within a certain range from the grids that record the topographic height of the vertex of the approximate topographic surface that is determined to be in contact with the bucket 10 or blade 16, and the grids around this grid, based on factors such as the susceptibility to collapse based on the soil type of the topography. After estimating the work area, the information processing device 54 terminates the process shown in Figure 11. Subsequently, the information processing device 54 executes step S205 in Figure 10.

[0099] Figure 12 is a flowchart detailing step S205 of Figure 10. The information processing device 54 executes steps S401 to S404 for all grids of the topographic map.

[0100] In step S401, the information processing device 54 determines whether the distribution of terrain heights recorded in the grid of the topographic map is below a preset threshold. If the distribution of terrain heights is below the threshold, the information processing device 54 proceeds to step S402 with the process shown in Figure 12. If the distribution of terrain heights is greater than the threshold, the information processing device 54 proceeds to step S404 with the process shown in Figure 12. Also, if the number of measurements taken by the measuring device 70 for the area corresponding to the grid is one or less, and the distribution of terrain heights has not been calculated, the information processing device 54 proceeds to step S404 with the process shown in Figure 12. The threshold for the distribution of terrain heights is stored in the storage device 57 or ROM 71.

[0101] In step S402, the information processing device 54 determines whether the grid is estimated to be a work area based on the results of the processing in step S204 in Figure 10. If the grid is estimated to be a work area, the information processing device 54 proceeds to step S403 with the processing shown in Figure 12. If the grid is not estimated to be a work area, the information processing device 54 proceeds to step S404 with the processing shown in Figure 12.

[0102] In step S403, the information processing device 54 classifies the grid into an update area. Then, the information processing device 54 performs the process of step S401 for the next grid.

[0103] In step S404, the information processing device 54 classifies the grid as a non-updated area. Then, the information processing device 54 performs the process in step S401 for the next grid. After performing the processes in steps S401 to S404 for all grids of the topographic map, the information processing device 54 terminates the process shown in Figure 12. Then, the information processing device 54 performs the process in step S206 shown in Figure 10.

[0104] In addition, the flowchart in Figure 12 describes a method for classifying areas into update areas or non-update areas based on the distribution of terrain height and the estimation results of the work area. However, the information processing device 54 may add a step between steps S402 and S403 in Figure 12, for example, classifying an area as an update area if a predetermined time has elapsed since the last measurement time recorded in the grid, and classifying it as a non-update area if a predetermined time has not elapsed since the last measurement time recorded in the grid. Furthermore, the order of steps S401 and S402 may be reversed.

[0105] Figures 13 to 18 illustrate a specific example of the topographic map update process. Note that the topographic maps shown in Figures 13 to 18 illustrate the classification of grids (updated area, non-updated area, or work area) during the topographic map update process, and do not illustrate the processing results for each control cycle (100 ms) of the information processing device 54.

[0106] Figure 13 shows a specific example of the topographic map update process during excavation and loading onto the transport vehicle 200. In Figure 13, as shown by reference numeral 310a, it is assumed that the hydraulic excavator 1 excavates at the bench end 313 and loads the excavated material onto the vessel 201 of the transport vehicle 200. The measuring device 70 is assumed to have a measuring range indicated by the arc of reference numeral 91.

[0107] First, the hydraulic excavator 1 excavates the bench end 313. At this time, the topographic map stored in the map storage unit 86 is classified as shown in reference numeral 310b. Here, the area indicated by the pattern of reference numeral 314 (dot hatching) is the update area. The area indicated by the pattern of reference numeral 315 (diagonal hatching) is the non-update area. The area indicated by the pattern of reference numeral 316 (cross hatching) is the work area. The area indicated by reference numeral 316 is an area that is estimated to be the work area because the terrain changes due to the operation (excavation) of reference numeral 310a. Note that the work area indicated by reference numeral 316 is classified as an update area by the processing in step S205 of Figure 10, but for the sake of clarity, the work area and the update area are shown separately in Figure 13. Also, the area 314 located in the upper left of reference numeral 310b is outside the measurement range of the measuring device 70, and the variance of the terrain height recorded in the grid is not below the threshold, so it is classified as an update area.

[0108] Next, the hydraulic excavator 1 rotates its slewing body 7 in the direction 320 toward the vessel 201 of the transport vehicle 200, as shown in reference numeral 311a. At this time, the topographic map is classified as shown in reference numeral 311b. The area shown in reference numeral 317 is initially estimated as a work area by the operation of reference numeral 310a, and then, based on the topographic height and distribution of topographic height recorded in the grid, it is confirmed that the topographic height has not changed due to the operation of reference numeral 310a, and it is classified as a non-updated area. The area shown in reference numeral 318 is initially estimated as a work area by the operation of reference numeral 310a, and then, based on the topographic height and distribution of topographic height recorded in the grid, it is confirmed that the topographic height has changed due to the operation of reference numeral 310a, and it is classified as an updated area. The area shown in reference numeral 319 is estimated as a work area because the bucket 10 passes through it when the hydraulic excavator 1 transports the excavated soil and sand to the vessel 201 of the transport vehicle 200, and the topography changes due to the falling soil and sand.

[0109] Next, as shown in reference numeral 312a, the hydraulic excavator 1 discharges soil and other materials into the vessel 201 of the transport vehicle 200. At this time, the topographic map is classified as shown in reference numeral 312b. The area shown in reference numeral 321 is initially classified as an updated area by the operation of reference numeral 311a, and then, after being measured multiple times by the measuring device 70, the dispersion of terrain height falls below a threshold, and it is classified as an unupdated area. The area shown in reference numeral 322 is the place where the hydraulic excavator 1 discharged soil, but since the transport vehicle 200 is located directly below the bucket 10, it is not estimated to be a work area and is classified as an unupdated area.

[0110] Figure 14 shows a specific example of the topographic map update process during excavation and soil discharge onto the ground. In Figure 14, as shown by reference numeral 350a, it is assumed that the hydraulic excavator 1 excavates at the bench end 313 and discharges the soil onto the ground at the target location indicated by reference numeral 356 in reference numerals 351a and 352a. The measurement range of the measuring device 70 is the same as in Figure 13.

[0111] First, the hydraulic excavator 1 excavates the bench end 313. At this time, the topographic map stored in the map storage unit 86 is classified as shown in reference numeral 350b. The area shown in reference numeral 354 is presumed to be the work area because the topography changes due to the excavation.

[0112] Next, the hydraulic excavator 1 rotates the slewing body 7 in the direction 355 of the target position 356, as shown in reference numeral 351a. At this time, the topographic map is classified as shown in reference numeral 351b. The area shown in reference numeral 357 is initially estimated as a work area by the operation of reference numeral 350a, and then, based on the topographic height and distribution of topographic height recorded in the grid, it is confirmed that the topographic height has not changed due to the operation of reference numeral 350a, and it is classified as a non-updated area. The area shown in reference numeral 358 is initially classified as a work area by the operation of reference numeral 350a, and then, based on the topographic height and distribution of topographic height recorded in the grid, it is confirmed that the topographic height has changed due to the operation of reference numeral 350a, and it is classified as an updated area. The area shown in reference numeral 359 is estimated to be a work area because the bucket 10 passes through it when the hydraulic excavator 1 transports the excavated soil and sand to the target position 356, and the topography changes due to the falling soil and sand.

[0113] Next, the hydraulic excavator 1 discharges soil and other materials to the target position 356 as shown in reference numeral 352a. At this time, the topographic map is classified as shown in reference numeral 352b. The area shown in reference numeral 360 is initially determined to be an updated area by the operation of reference numeral 351a, and after being measured multiple times by the measuring device 70, the dispersion of terrain height falls below a threshold, and it is classified as a non-updated area. The area shown in reference numeral 361 is the location where the hydraulic excavator 1 discharged soil, and since there is no transport vehicle 200 directly below the bucket 10, it is presumed to be a work area. This area shown in reference numeral 361 is also measured multiple times by the measuring device 70, and if the dispersion of terrain height falls below a threshold, it is classified as a non-updated area, similar to the area shown in reference numeral 360.

[0114] Figure 15 shows a specific example of the topographic map update process during ground leveling. In Figure 15, it is assumed that a hydraulic excavator 1 levels the mound of earth 413 as shown by reference numeral 410a, and then levels it into a flat surface as shown by reference numerals 412a and 415. The measurement range of the measuring device 70 is the same as in Figure 13.

[0115] First, the hydraulic excavator 1 measures the soil mound 413 using the measuring device 70. At this time, the topographic map stored in the map storage unit 86 is classified as shown in reference numeral 410b.

[0116] Next, the hydraulic excavator 1 levels the mound of earth 413 with the bucket 10, as shown in reference numeral 411a. At this time, the topographic map is classified as shown in reference numeral 411b. The area shown in reference numeral 414 is estimated to be the work area because the bucket 10 and the mound of earth 413 (topography) came into contact during leveling.

[0117] Next, as shown in reference numeral 412a, the hydraulic excavator 1 measures the leveled area, flat ground 415, using the measuring device 70. At this time, the topographic map is classified as shown in reference numeral 412b. The area shown in reference numeral 416 is classified as an updated area because the terrain height has changed due to the operation of reference numeral 411a, and the variance of the terrain height recorded in the grid has become greater than the threshold. This area shown in reference numeral 416, like the area shown in reference numeral 360, is classified as a non-updated area after being measured multiple times by the measuring device 70.

[0118] Figure 16 shows a specific example of the topographic map update process during spotting. Spotting is the operation in which the hydraulic excavator 1 uses its bucket 10 to specify the stopping position of the transport vehicle 200, thereby prompting the transport vehicle 200 to stop at the specified stopping position. In Figure 16, as shown by reference numeral 430a, it is assumed that the hydraulic excavator 1 uses its bucket 10 to specify the stopping position 432 of the transport vehicle 200 near the edge of the bench. The measurement range of the measuring device 70 is the same as in Figure 13.

[0119] First, the hydraulic excavator 1 uses its bucket 10 to designate the stopping position 432 for the transport vehicle 200. At this time, the topographic map stored in the map storage unit 86 is classified as shown in reference numeral 430b. The area shown in reference numeral 433 is directly below the bucket 10, but it has not been excavated in advance, and there is no soil or other material inside the bucket 10, so it is not presumed to be a work area. The area shown in reference numeral 433 is classified as a non-updated area.

[0120] Next, the transport vehicle 200 stops at the stopping position 432 designated by the hydraulic excavator 1, as shown in reference numeral 431a. At this time, the topographic map is classified as shown in reference numeral 431b. The area shown in reference numeral 434 is within the measurement range of the measuring device 70, where the transport vehicle 200 is located, but it is classified as a non-updated area at the time of reference numeral 430b, and therefore is not classified as an updated area. The area shown in reference numeral 434 is classified as a non-updated area.

[0121] Figure 17 shows a specific example of the topographic map update process during a jack-up turn. A jack-up turn is an operation in which the hydraulic excavator 1 lifts the vehicle body 5 by pressing the bucket 10 against the ground, and then changes the orientation of the lifted vehicle body 5 by rotating the slewing body 7. In Figure 17, it is assumed that the hydraulic excavator 1 presses the bucket 10 against the ground as shown by reference numeral 470a and changes its posture to the orientation shown by reference numeral 472a. The measurement range of the measuring device 70 is the same as in Figure 13.

[0122] First, the hydraulic excavator 1 presses its bucket 10 against the ground. At this time, the topographic map stored in the map storage unit 86 is classified as shown in reference numeral 470b. The area shown in reference numeral 473 is estimated to be the work area because the hydraulic excavator 1 has pressed its bucket 10 against the ground.

[0123] Next, as shown in reference numeral 471a, the hydraulic excavator 1 changes the orientation of its vehicle 5 and lifts the bucket 10 off the ground. At this time, the topographic map is classified as shown in reference numeral 471b. The area shown in reference numeral 474 is classified as an updated area because the terrain height has changed due to the hydraulic excavator 1 pressing the bucket 10 against the ground.

[0124] Next, the hydraulic excavator 1 changes the orientation of the rotating body 7 as shown in reference numeral 472a. At this time, the topographic map is classified as shown in reference numeral 472b. The area shown in reference numeral 475 is classified as a non-updated area as a result of being measured multiple times by the measuring device 70, similar to the area shown in reference numeral 360.

[0125] Figure 18 shows a specific example of the topographic map update process when leveling soil and sand loaded onto a transport vehicle 200. In Figure 18, as indicated by reference numeral 510a, it is assumed that a hydraulic excavator 1 uses a bucket 10 to level the soil and sand 513 loaded onto the transport vehicle 200. The measurement range of the measuring device 70 is the same as in Figure 13.

[0126] First, the hydraulic excavator 1 extends its bucket 10 toward the soil 513 on the vessel 201 of the transport vehicle 200. At this time, the topographic map stored in the map storage unit 86 is classified as shown in reference numeral 510b. The area shown in reference numeral 514 is not presumed to be a work area because the transport vehicle 200 is located directly beneath the bucket 10 and there is no soil or other material inside the bucket 10. The area shown in reference numeral 514 is classified as a non-updated area.

[0127] Next, the hydraulic excavator 1 levels the soil 513 on the vessel 201 using the bucket 10, as shown in reference numeral 511a. At this time, the topographic map is classified as shown in reference numeral 511b. The area shown in reference numeral 515 is classified as a non-updated area because the bucket 10 is in contact with the soil 513 on the vessel 201, but the bucket 10 is not in contact with the topography recorded on the topographic map.

[0128] Next, as shown in reference numeral 512a, the hydraulic excavator 1 finishes leveling the soil 513 on the vessel 201 using the bucket 10 and then lifts the bucket 10 away from the soil 516. The leveled soil 516 is loaded onto the vessel 201. At this time, the topographic map is classified as shown in reference numeral 512b. The area shown in reference numeral 517 is not presumed to be a work area because the transport vehicle 200 is located directly below the bucket 10 and there is no soil or other material inside the bucket 10. The area shown in reference numeral 517 is classified as a non-updated area.

[0129] A specific example of point cloud separation processing will be explained using Figure 19. Figure 19 is a diagram showing a specific example of point cloud separation processing that separates the point cloud representing the transport vehicle 200 from the point cloud data acquired by the measuring device 70. As indicated by reference numeral 600, Figure 19 assumes a situation in which the hydraulic excavator 1 measures the terrain including the transport vehicle 200 using the measuring device 70.

[0130] In the situation shown by reference numeral 600, when the measuring device 70 measures the transport vehicle 200 and the terrain, the point cloud data 603 acquired by the measuring device 70 is projected onto the same coordinate system as the terrain 604 in the terrain map, as shown by reference numeral 601. Reference numeral 605 indicates the grid division. Reference numeral 606 indicates the grid.

[0131] Of the point cloud data 603 acquired by the measuring device 70, the point clouds assigned to the grid labeled 607 and the point clouds assigned to the grid labeled 609 are projected near the terrain height of terrain 604 in the topographic map, and are therefore determined to be point clouds representing terrain. Of the point cloud data 603 acquired by the measuring device 70, the point clouds assigned to the grid within the range labeled 608 are projected at a position away from the terrain height of terrain 604 in the topographic map, and are therefore determined to be candidates for point clouds representing the transport vehicle 200 (mobile object point clouds). The point clouds assigned to the grid within the range labeled 608 are separated from the point cloud data 603, grouped into one group, and stored in the mobile object storage unit 87. As a result, as shown in the reference numeral 610 of reference numeral 602, only the point clouds representing the transport vehicle 200 are extracted.

[0132] As described above, the hydraulic excavator 1 according to the first embodiment is a work machine comprising a traveling body 5, a slewing body 7 rotatably attached to the traveling body 5, and a work device 2 attached to the slewing body 7 having a boom 8, an arm 9, and a bucket 10. The hydraulic excavator 1 comprises a posture detection device 53 for detecting the posture of each part of the hydraulic excavator 1, a measuring device 70 for measuring objects around the hydraulic excavator 1 and acquiring point cloud data, and an information processing device 54 which has a map storage unit 86 for storing a topographic map that records the terrain height around the hydraulic excavator 1 and performs recognition processing of objects around the hydraulic excavator 1 based on the point cloud data acquired by the measuring device 70 and the topographic map stored in the map storage unit 86. The information processing device 54 includes: a work area estimation unit 81 that estimates the work area, which is the area in the topographic map where the terrain changes due to the operation of the hydraulic excavator 1, based on the position information of each part of the hydraulic excavator 1 calculated from the detection results of the attitude detection device 53; an update area determination unit 82 that classifies the areas in the topographic map into update areas where the terrain height is updated and non-update areas where it is not updated, based on the estimation results of the work area estimation unit 81; a point cloud separation unit 83 that separates the point cloud data acquired by the measuring device 70 into a point cloud representing terrain and a point cloud representing objects other than terrain, based on the terrain height recorded in the non-update areas of the topographic map classified by the update area determination unit 82 and the point cloud data corresponding to the non-update areas; and a map update unit 84 that updates the terrain height recorded in the update area using the point cloud data from which the point cloud representing objects other than terrain separated by the point cloud separation unit 83 has been excluded, and stores the updated topographic map in the map storage unit 86.

[0133] As a result, the hydraulic excavator 1 can appropriately separate the point cloud representing terrain from the point cloud representing non-terrain objects, even in environments where the terrain changes moment by moment, such as a mining site. Therefore, the hydraulic excavator 1 can update the topographic map using point cloud data from which point clouds representing non-terrain objects such as transport vehicles 200 have been excluded. The hydraulic excavator 1 can perform recognition processing on objects using only the point cloud representing non-terrain objects such as transport vehicles 200. Thus, the hydraulic excavator 1 can prevent non-terrain objects such as transport vehicles 200 from being recorded on the topographic map, while generating a topographic map that records the latest terrain of the work site, thereby improving the recognition accuracy of the objects.

[0134] Furthermore, in the hydraulic excavator 1 according to the first embodiment, the area within the topographic map is defined by dividing the area around the hydraulic excavator 1 into a grid. For each grid corresponding to an area, the topographic map records the terrain height, the distribution of terrain height, and the last measurement time, which indicates the time when the measuring device 70 last measured that area.

[0135] This allows the hydraulic excavator 1 to verify whether the terrain height recorded on the grid is stable (i.e., whether the variation in terrain height obtained by measuring the same area multiple times with the measuring device 70 is small). In addition, the hydraulic excavator 1 can verify whether the terrain height recorded on the grid is a value obtained a long time after the last measurement time. If the terrain height is unstable (the variation is large), it is possible that the terrain height was not measured accurately due to some factor. If the terrain height is a value obtained a long time after the last measurement time, it is possible that the terrain height has changed due to the passage of time. By verifying the stability of the terrain height recorded on the grid and verifying the elapsed time since the last measurement time, the hydraulic excavator 1 can record the terrain height recorded on the grid of the topographic map as a more accurate value. As a result, the hydraulic excavator 1 can accurately classify the area within the topographic map into updated areas and non-updated areas, and can accurately separate the point cloud representing terrain from the point cloud representing objects other than terrain. Therefore, the hydraulic excavator 1 can reliably prevent objects other than terrain, such as transport vehicles 200, from being recorded on the topographic map, while more accurately generating a topographic map that records the latest terrain of the work site, thereby further improving the recognition accuracy of such objects.

[0136] Furthermore, in the hydraulic excavator 1 according to the first embodiment, the work area estimation unit 81 performs a contact determination to determine whether the bucket 10 is in contact with the terrain based on the positional relationship between the bucket 10 and the terrain in the topographic map, and estimates the work area based on the result of the contact determination.

[0137] This prevents the hydraulic excavator 1 from mistakenly estimating areas where the terrain does not change due to its operation as work areas and classifying them as update areas. The hydraulic excavator 1 can more reliably prevent non-terrain objects present in areas mistakenly classified as update areas from being recorded on the topographic map, thereby preventing contamination of the topographic map. Therefore, the hydraulic excavator 1 can accurately and reliably separate point clouds representing terrain from point clouds representing non-terrain objects. Thus, the hydraulic excavator 1 can more reliably prevent non-terrain objects such as transport vehicles 200 from being recorded on the topographic map, while accurately and reliably generating a topographic map with the latest terrain of the work site recorded, thereby further improving the recognition accuracy of such objects.

[0138] Furthermore, in the hydraulic excavator 1 according to the first embodiment, the work area estimation unit 81 estimates the work area based on the weight of the load in the bucket 10 and the position the bucket 10 passes through relative to the terrain in the topographic map.

[0139] This prevents the hydraulic excavator 1 from mistakenly estimating areas where the terrain does not change due to its operation as work areas and classifying them as update areas. Therefore, the hydraulic excavator 1 can reliably prevent non-terrain objects present in areas mistakenly classified as update areas from being recorded on the topographic map, thereby preventing contamination of the topographic map. Consequently, the hydraulic excavator 1 can accurately and reliably separate point clouds representing terrain from point clouds representing non-terrain objects. Thus, the hydraulic excavator 1 can more reliably prevent non-terrain objects such as transport vehicles 200 from being recorded on the topographic map, while accurately and reliably generating a topographic map with the latest terrain of the work site recorded, thereby further improving the recognition accuracy of such objects.

[0140] Furthermore, in the hydraulic excavator 1 according to the first embodiment, terrain height and the distribution of terrain height are recorded for each region in the topographic map. The update region determination unit 82 classifies the regions in the topographic map into update regions and non-update regions based on the terrain height and the distribution of terrain height recorded in the topographic map and the work region estimated by the work region estimation unit 81.

[0141] As a result, the hydraulic excavator 1 can more accurately classify areas within the topographic map into updated and unupdated areas, taking into account changes in terrain elevation and the stability of terrain elevation. Therefore, the hydraulic excavator 1 can more accurately separate point clouds representing terrain from point clouds representing non-terrain objects. Thus, the hydraulic excavator 1 can more reliably prevent non-terrain objects such as transport vehicles 200 from being recorded on the topographic map, while more accurately generating a topographic map that records the latest terrain of the work site, thereby further improving the recognition accuracy of such objects.

[0142] Furthermore, in the hydraulic excavator 1 according to the first embodiment, terrain height and the distribution of terrain height are recorded for each region within the topographic map. The point cloud separation unit 83 separates the point cloud data acquired by the measuring device 70 into a point cloud representing terrain and a point cloud representing objects other than terrain by extracting outliers present in the point cloud data corresponding to the unupdated regions of the topographic map classified by the update region determination unit 82, using the terrain height and the distribution of terrain height recorded in those unupdated regions.

[0143] As a result, the hydraulic excavator 1 can easily and accurately separate the point cloud representing terrain from the point cloud representing objects other than terrain. Therefore, the hydraulic excavator 1 can easily and reliably prevent objects other than terrain, such as transport vehicles 200, from being recorded on the topographic map, while easily and accurately generating a topographic map that records the latest terrain of the work site, thereby further improving the recognition accuracy of such objects.

[0144] [Second Embodiment] A second embodiment of the present invention will be described with reference to Figures 20 to 22. In the second embodiment, the same components as in the first embodiment will not be described. Figure 20 is a functional block diagram of the information processing device 54 according to the second embodiment.

[0145] In the second embodiment, the hydraulic excavator 1 acquires location information of a mobile object (transport vehicle 200) present at the work site of the hydraulic excavator 1 via a control system such as FMS, and excludes the location of the mobile object from the topographic map update area. For this purpose, the hydraulic excavator 1 in the second embodiment is equipped with a communication device 100 that receives the location information of the mobile object. The location information of the mobile object is acquired by a GNSS receiver mounted on the mobile object and transmitted to the hydraulic excavator 1 via the control system server. Based on the location information of the mobile object received by the communication device 100, the update area determination unit 82 of the information processing device 54 in the second embodiment classifies the area in the topographic map corresponding to the location of the mobile object as a non-update area.

[0146] The point cloud separation process according to the second embodiment is the same as that of the first embodiment, so its description will be omitted. In the topographic map update process according to the second embodiment, the flowchart showing the details of step S205 shown in Figure 10 is changed from Figure 12 to Figure 21. Figure 21 is a flowchart showing the details of step S205 according to the second embodiment. Compared to Figure 12, in Figure 21, step S405 is added before step S401, and step S406 is added between steps S402 and S403.

[0147] In the explanation of Figures 21 and 22, the topographic map is represented by a grid map, and the areas within the topographic map are defined by the grid. Furthermore, as an example of a moving object other than terrain, a transport vehicle 200 is used.

[0148] In step S405, the information processing device 54 acquires the location information of the transport vehicles 200 located around the hydraulic excavator 1 via the communication device 100. The location information of the transport vehicles 200 is in the same coordinate system as the topographic map stored in the map storage unit 86. After acquiring the location information, the information processing device 54 proceeds to step S401 with the processing shown in Figure 21.

[0149] In step S406, the information processing device 54 determines whether or not there is a transport vehicle 200 within the measurement range of the measuring device 70, based on the location information of the transport vehicle 200 acquired in step S405. If there is a transport vehicle 200, the information processing device 54 proceeds to step S403 with the process shown in Figure 21. If there is no transport vehicle 200, the information processing device 54 proceeds to step S404 with the process shown in Figure 21.

[0150] Using Figure 22, a specific example of the topographic map update process when the transport vehicle 200 passes through the update area will be explained. Please note that the topographic map shown in Figure 22 is an example of the grid classification (updated area, non-updated area, or work area) during the topographic map update process, and does not represent the processing results for each control cycle (100 ms) of the information processing device 54.

[0151] Figure 22 shows a specific example of the topographic map update process when the transport vehicle 200 passes through the update area. In Figure 22, it is assumed that the transport vehicle 200 moves around the hydraulic excavator 1 in the direction of arrow 653, as shown by reference numeral 650a. The topographic map stored in the map storage unit 86 is classified as shown by reference numeral 650b. The area shown by reference numeral 654 is classified as an update area due to its large variation in topographic height. The measurement range of the measuring device 70 is the same as in Figure 13.

[0152] First, the transport vehicle 200 stops at the position indicated by reference numeral 651a. At this time, the topographic map is classified as shown by reference numeral 651b. The area indicated by reference numeral 655 was classified as an update area when there was a large variation in terrain height and the transport vehicle 200 was not present, but since the presence of the transport vehicle 200 was confirmed from the acquired position information of the transport vehicle 200, it is classified as a non-update area.

[0153] Next, the transport vehicle 200 travels again in the direction of arrow 653, as shown in reference numeral 652a. At this time, the topographic map is classified as shown in reference numeral 652b. The area shown in reference numeral 656 was classified as an update area when there was a large variation in terrain height and the transport vehicle 200 was not present, but since the presence of the transport vehicle 200 was confirmed from the acquired location information of the transport vehicle 200, it is now classified as a non-update area. The area shown in reference numeral 657 is now classified as an update area because the absence of the transport vehicle 200 was confirmed from the acquired location information of the transport vehicle 200.

[0154] As described above, the hydraulic excavator 1 according to the second embodiment is equipped with a communication device 100 that receives location information of a mobile object present at the work site of the hydraulic excavator 1. The update area determination unit 82 classifies the area in the topographic map corresponding to the location of the mobile object into a non-update area based on the location information of the mobile object received by the communication device 100.

[0155] As a result, the hydraulic excavator 1 can prevent the moving object from being recorded on the topographic map even when the moving object is moving through an area where the topographic map has not yet been updated (i.e., an update area). Therefore, even in environments where moving objects frequently pass through and it is difficult to update the topographic map, the hydraulic excavator 1 can prevent the moving object from being mistakenly recorded on the topographic map, thereby preventing contamination of the topographic map. Therefore, even in environments where moving objects frequently pass through and it is difficult to update the topographic map, the hydraulic excavator 1 can reliably separate the point cloud representing the terrain from the point cloud representing objects other than terrain. Thus, the hydraulic excavator 1 can reliably prevent objects other than terrain, such as transport vehicles 200, from being recorded on the topographic map, while reliably generating a topographic map that records the latest terrain of the work site, thereby further improving the recognition accuracy of such objects.

[0156] [Third Embodiment] A third embodiment of the present invention will be described with reference to Figures 23 to 25. In the third embodiment, the same components as in the first embodiment will not be described. Figure 23 is a functional block diagram of the information processing device 54 according to the third embodiment.

[0157] The hydraulic excavator 1 according to the third embodiment uses a topographic map classified into update areas and non-update areas to calculate the stopping position of the mobile body (transport vehicle 200) present at the work site of the hydraulic excavator 1, which is the position where the movement of the mobile body is stopped. Then, the hydraulic excavator 1 according to the third embodiment performs a spotting operation using the calculated stopping position of the mobile body. For this purpose, the hydraulic excavator 1 according to the third embodiment includes a communication device 100 that receives information about the size of the mobile body and a control device 40 that controls the operation of each part of the hydraulic excavator 1. The information processing device 54 according to the third embodiment includes a stopping position calculation unit 89 that calculates the stopping position of the mobile body based on the information about the size of the mobile body received by the communication device 100 and the topographic map. The stopping position calculation unit 89 calculates the stopping position of the mobile body within the non-update area classified by the update area determination unit 82. The control device 40 according to the third embodiment controls the operation of each part of the hydraulic excavator 1 to perform spotting, prompting the mobile body to stop at the stopping position calculated by the stopping position calculation unit 89.

[0158] The topographic map update process and point cloud separation process according to the third embodiment are the same as those in the first embodiment, so their explanation will be omitted. The flow of the stop position calculation process performed by the information processing device 54, which includes the stop position calculation unit 89, will be explained using Figure 24. Figure 24 is a flowchart of the stop position calculation process performed by the information processing device 54.

[0159] In the explanation of Figures 24 and 25, the topographic map is represented by a grid map, and the areas within the topographic map are defined by the grid. Furthermore, as an example of a moving object other than terrain, a transport vehicle 200 is used.

[0160] In step S501, the information processing device 54 obtains information regarding the size of transport vehicles 200 located around the hydraulic excavator 1 via the communication device 100. After obtaining the size information, the information processing device 54 proceeds to step S502 with the processing shown in Figure 24.

[0161] In step S502, the information processing device 54 reads a topographic map from the map storage unit 86, which is classified into updated areas and non-updated areas. After reading the topographic map, the information processing device 54 proceeds to step S503 with the process shown in Figure 24.

[0162] In step S503, the information processing device 54 calculates candidate locations (hereinafter referred to as candidate parking positions) where the transport vehicle 200 can be parked within a certain range of the hydraulic excavator 1. Here, the certain range is a value specified in advance. Candidate parking positions for the transport vehicle 200 can be calculated, for example, by generating a two-dimensional map in which grids that are lower than the terrain height on which the hydraulic excavator 1 is located and classified as non-updated areas are set to 1, and all other grids are set to 0, and then using the solution method for the largest rectangle problem to search for a rectangle larger than the transport vehicle 200 from the generated two-dimensional map. After calculating the candidate parking positions, the information processing device 54 proceeds to step S504 with the processing shown in Figure 24.

[0163] In step S504, the information processing device 54 determines whether or not there are candidate stopping positions for the transport vehicle 200. If there are candidate stopping positions, the information processing device 54 proceeds to step S505 with the process shown in Figure 24. If there are no candidate stopping positions, the information processing device 54 proceeds to step S506 with the process shown in Figure 24.

[0164] In step S505, the information processing device 54 determines the stopping position of the transport vehicle 200 from the candidate stopping positions of the transport vehicle 200 calculated in step S503. If multiple candidate stopping positions are calculated in step S503, the information processing device 54 determines the stopping position of the transport vehicle 200 from among the multiple candidate stopping positions based on arbitrary conditions set in advance, such as selecting a wider stopping position, allowing the user to select a stopping position, or selecting a stopping position that facilitates loading work. The information processing device 54 then generates control information for spotting, prompting the transport vehicle 200 to stop at the determined stopping position, and outputs it to the control device 40. The control device 40 controls the operation of each part of the hydraulic excavator 1 to perform spotting, prompting the transport vehicle 200 to stop at the determined stopping position. After outputting the control information for spotting, the information processing device 54 terminates the process shown in Figure 24.

[0165] In step S506, the information processing device 54 generates control information for the measuring device 70 to measure the surrounding terrain and outputs it to the control device 40. This makes it possible to calculate candidate stopping positions in step S503 in subsequent control cycles. After outputting the control information for measurement, the information processing device 54 terminates the process shown in Figure 24.

[0166] Figure 25 illustrates a specific example of the topographic map update process during spotting using a topographic map. Note that the topographic map shown in Figure 25 is an example of grid classification (updated area, non-updated area, or work area) during the topographic map update process, and does not represent the processing results for each control cycle (100 ms) of the information processing device 54.

[0167] Figure 25 shows a specific example of the topographic map update process during spotting using a topographic map. In Figure 25, as indicated by reference numeral 670a, it is assumed that the hydraulic excavator 1 parks the transport vehicle 200 around it. The area indicated by reference numeral 673 represents the ground at a lower position than the ground on which the hydraulic excavator 1 is located. The area indicated by reference numeral 674 represents the ground at the same height as the ground on which the hydraulic excavator 1 is located. Furthermore, the topographic map stored in the map storage unit 86 is assumed to be classified as shown by reference numeral 670b. The measurement range of the measuring device 70 is the same as in Figure 13.

[0168] In the situation shown by reference numeral 670a, if the transport vehicle 200 were to be parked around the hydraulic excavator 1 without considering the classification results of the topographic map, the area encompassing the area shown by reference numeral 675 and the area shown by reference numeral 676 would be a candidate parking location for the transport vehicle 200. However, by considering the classification results of the topographic map, the area shown by reference numeral 675 is determined as the parking location for the transport vehicle 200. As a result, as shown by reference numeral 671a, the information processing device 54 can guide the transport vehicle 200 to a parking location that is easier to recognize.

[0169] As described above, the hydraulic excavator 1 according to the third embodiment includes a communication device 100 that receives information regarding the size of a moving object present at the work site of the hydraulic excavator 1, and a control device 40 that controls the operation of each part of the hydraulic excavator 1. The information processing device 54 includes a stop position calculation unit 89 that calculates the stopping position of the moving object based on the information regarding the size of the moving object received by the communication device 100 and a topographic map. The control device 40 controls the operation of each part of the hydraulic excavator 1 to prompt the moving object to stop at the stopping position calculated by the stop position calculation unit 89.

[0170] As a result, the hydraulic excavator 1 can specify the stopping position of the moving object using a topographic map, and can quickly guide and stop the moving object within an area with arbitrary characteristics, such as an area defined as terrain or a flat area. Therefore, the hydraulic excavator 1 can reduce the waiting time of the moving object until the topographic map is updated, and can improve productivity at the work site. Thus, the hydraulic excavator 1 can prevent objects other than terrain, such as transport vehicles 200, from being recorded on the topographic map, while generating a topographic map that records the latest terrain of the work site, thereby improving the recognition accuracy of such objects and improving productivity at the work site.

[0171] Furthermore, in the hydraulic excavator 1 according to the third embodiment, the stop position calculation unit 89 calculates the stop position of the moving body within the non-updated area classified by the update area determination unit 82.

[0172] As a result, the hydraulic excavator 1 can specify the stopping position of the moving object within the area where the topographic map has been updated, that is, within an area where it is easy to separate the point cloud representing the terrain from the point cloud representing the moving object, and can guide and stop the moving object within that area. Therefore, the hydraulic excavator 1 can reliably separate the point cloud representing the terrain from the point cloud representing the moving object, and can further improve the recognition accuracy of the moving object. Thus, the hydraulic excavator 1 can reliably prevent objects other than terrain, such as transport vehicles 200, from being recorded on the topographic map, while reliably generating a topographic map that records the latest terrain of the work site, and can further improve the recognition accuracy of such objects.

[0173] It should be noted that the present invention is not limited to the embodiments described above, and various modifications are included. For example, the embodiments described above are described in detail to make the present invention easier to understand, and are not necessarily limited to those comprising all the components described. Furthermore, it is possible to replace some of the components of one embodiment with components of another embodiment, and it is also possible to add components of another embodiment to the components of one embodiment. In addition, it is possible to add, delete, or replace some of the components of each embodiment with components of other embodiments.

[0174] Furthermore, each of the above-mentioned components, functions, processing units, or processing means may be implemented in hardware, in whole or in part, for example, by designing them as integrated circuits. Alternatively, each of the above-mentioned components or functions may be implemented in software by a processor interpreting and executing programs that implement each function. Information such as programs, tables, or files that implement each function can be stored in a recording device such as memory, a hard disk, or an SSD (solid state drive), or in a recording medium such as an IC card, SD card, or DVD.

[0175] Furthermore, the control lines and information lines shown are those deemed necessary for explanatory purposes, and not all control lines and information lines are necessarily shown in the actual product. In reality, it can be assumed that almost all components are interconnected.

[0176] 1...Hydraulic excavator (working machine), 5...Traveling body, 7...Slewing body, 8...Boom, 9...Arm, 10...Bucket, 40...Control device, 53...Attitude detection device, 54...Information processing device, 70...Measurement device, 81...Work area estimation unit, 82...Update area determination unit, 83...Point cloud separation unit, 84...Map update unit, 86...Map storage unit (storage unit), 89...Stop position calculation unit, 100...Communication device

Claims

A work machine comprising a traveling body, a slewing body rotatably attached to the traveling body, and a work device attached to the slewing body having a boom, arm and bucket, A posture detection device for detecting the posture of each part of the aforementioned work machine, A measuring device that measures objects around the aforementioned work machine and acquires point cloud data, The information processing device includes a storage unit for storing a topographic map that records the terrain elevation around the work machine, and performs recognition processing of objects around the work machine based on the point cloud data acquired by the measuring device and the topographic map stored in the storage unit, The aforementioned information processing device is A work area estimation unit estimates a work area from within the topographic map, which is an area where the terrain changes due to the operation of the work machine, based on the positional information of each part of the work machine calculated from the detection results of the attitude detection device. Based on the estimation results of the work area estimation unit, the update area determination unit classifies the area within the topographic map into an update area where the topographic height is updated and a non-update area where it is not updated. A point cloud separation unit separates the point cloud data acquired by the measuring device into a point cloud representing the terrain and a point cloud representing objects other than the terrain, based on the terrain height recorded in the non-updated area of ​​the terrain map classified by the update area determination unit and the point cloud data corresponding to the non-updated area. The map update unit updates the terrain height recorded in the update region using the point cloud data from which the point cloud representing objects other than the terrain separated by the point cloud separation unit has been excluded, and stores the updated terrain map in the storage unit. A work machine characterized by the following features. 　 The area within the aforementioned topographic map is defined by dividing the area around the aforementioned work machine into a grid. The topographic map records, for each grid corresponding to the area, the topographic height, the distribution of topographic height, and the last measurement time indicating the time when the measuring device last measured the area. The work machine according to feature 1. 　 The aforementioned work area estimation unit, Based on the positional relationship between the bucket's position and the terrain in the topographic map, a contact determination is performed to determine whether the bucket is in contact with the terrain. The work area is estimated based on the results of the contact determination. The work machine according to feature 1. 　 The work area estimation unit estimates the work area based on the weight of the load present in the bucket and the position the bucket passes through relative to the terrain in the topographic map. The work machine according to feature 1. 　 Within the aforementioned topographic map, the topographic height and the distribution of topographic height are recorded for each region. The update area determination unit classifies the area within the topographic map into the update area and the non-update area based on the topographic height and the distribution of the topographic height recorded in the topographic map and the work area estimated by the work area estimation unit. The work machine according to feature 1. 　 Within the aforementioned topographic map, the topographic height and the distribution of topographic height are recorded for each region. The point cloud separation unit separates the point cloud data acquired by the measuring device into a point cloud representing the terrain and a point cloud representing objects other than the terrain by extracting outliers present in the point cloud data corresponding to the unupdated area of ​​the topographic map classified by the update area determination unit, using the terrain height and the variance of the terrain height recorded in the unupdated area. The work machine according to feature 1. 　 The aforementioned work machine is equipped with a communication device that receives location information of a moving object present at the work site of the work machine, The update area determination unit classifies the area within the topographic map corresponding to the location of the mobile object, based on the location information of the mobile object received by the communication device, into the non-update area. The work machine according to feature 1. 　 The aforementioned work machine is, A communication device that receives information regarding the size of a moving object present at the work site of the aforementioned work machine, The machine comprises a control device that controls the operation of each part of the machine, The information processing device includes a stop position calculation unit that calculates the stop position of the moving object based on information regarding the size of the moving object received by the communication device and the topographic map. The control device controls the operation of each part of the work machine to prompt the moving body to stop at the stop position calculated by the stop position calculation unit. The work machine according to feature 1. 　 The stop position calculation unit calculates the stop position of the moving body within the non-updated region classified by the update region determination unit. The work machine according to feature 8.

Images