Identification system and identification method

Abstract

To identify a loading target among objects present around a work machine.SOLUTION: An identification system comprises a three-dimensional sensor detecting an object and a controller. The controller has: a standard data storage unit for storing standard data indicating a three-dimensional shape of a loading target onto which cargo is to be loaded by a work machine; a data acquisition unit for acquiring detection data of an object detected by the three-dimensional sensor; a removal unit for removing a prescribed part including a contact surface from at least one of the standard data and the detection data; and an identification unit for, after the prescribed part is removed, comparing the standard data with the detection data to identify the loading target among the objects.SELECTED DRAWING: Figure 7

Description

[Technical Field]

[0001] The present disclosure relates to an identification system and an identification method. [Background technology]

[0002] Known in the technical field of work machines is a forklift truck as disclosed in Patent Document 1. In Patent Document 1, the forklift truck is equipped with a laser sensor that detects the distance to a pallet, and a controller that calculates a plane equation for the front surface of the pallet based on detection data from the laser sensor, and estimates the position and orientation of the pallet using the plane equation for the front surface of the pallet. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent Publication No. 2021-042070 Summary of the Invention [Problem to be solved by the invention]

[0004] One example of work performed by a work machine is a loading operation in which a load is loaded onto a loading target. During the loading operation of a work machine, it may be necessary to identify the loading target from objects present around the work machine. There is a possibility that the load may spill from the loading target. If the load spilled from the loading target accumulates on the ground around the loading target, the accuracy of identifying the loading target may decrease.

[0005] The present disclosure aims to identify a loading target from objects present in the vicinity of a work machine. [Means for solving the problem]

[0006] According to the present disclosure, there is provided an identification system including a three-dimensional sensor that detects objects and a controller. The controller has a standard data storage unit that stores standard data indicating the three-dimensional shape of a loading target onto which a load is to be loaded by a work machine, a data acquisition unit that acquires detection data of the object detected by the three-dimensional sensor, a removal unit that removes a predetermined portion including the contact surface from at least one of the standard data and the detection data, and an identification unit that, after the predetermined portion has been removed, compares the standard data with the detection data to identify the loading target from the objects. [Effects of the Invention]

[0007] According to the present disclosure, loading targets are identified from objects present around a work machine. [Brief explanation of the drawings]

[0008] [Figure 1] FIG. 1 is a side view showing a work machine according to an embodiment. [Figure 2] FIG. 2 is a configuration diagram showing a work machine according to an embodiment. [Figure 3] FIG. 3 is a hardware configuration diagram of the controller according to the embodiment. [Figure 4] FIG. 4 is a diagram illustrating the cycle work of the work machine according to the embodiment. [Figure 5] FIG. 5 is a diagram that schematically shows a work machine that performs a load forwarding operation according to the embodiment. [Figure 6] FIG. 6 is a perspective view showing a dump truck according to the embodiment. [Figure 7] FIG. 7 is a block diagram showing a control system for a work machine according to an embodiment. [Figure 8] FIG. 8 is a diagram illustrating standard data according to the embodiment. [Figure 9] FIG. 9 is a diagram illustrating standard point cloud data from which a predetermined portion has been removed according to the embodiment. [Figure 10] FIG. 10 is a diagram illustrating the classification process performed by the classification unit according to the embodiment. [Figure 11]FIG. 11 is a diagram illustrating the relative positions of the three-dimensional sensor, the camera, and the dump truck according to the embodiment. [Figure 12] FIG. 12 is a diagram showing a captured image and a converted image according to the embodiment. [Figure 13] FIG. 13 is a flowchart showing a control method for a work machine according to the embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0009] Hereinafter, embodiments of the present disclosure will be described with reference to the drawings, but the present disclosure is not limited to the embodiments. The components of the embodiments described below can be combined as appropriate. In addition, some components may not be used.

[0010] [Work machinery] FIG. 1 is a side view showing a work machine 1 according to an embodiment. FIG. 2 is a configuration diagram showing a work machine 1 according to an embodiment. The work machine 1 operates at a work site. The work machine 1 is an unmanned work machine. An unmanned work machine is a work machine that operates unmanned without being operated by a driver. The work machine 1 is automatically controlled. In the embodiment, the work machine 1 is a wheel loader.

[0011] The work machine 1 comprises a vehicle body 2, an articulating device 3, a traveling device 5 including wheels 4, and a work implement 6. The work machine 1 travels around a work site using the traveling device 5. The work machine 1 performs work at the work site using the work implement 6. Examples of work that the work machine 1 can perform include excavation work, loading work, and transport work.

[0012] The vehicle body 2 supports the work implement 6. The vehicle body 2 includes a front vehicle body section 2F and a rear vehicle body section 2R. The front vehicle body section 2F is located forward of the rear vehicle body section 2R. The front vehicle body section 2F and the rear vehicle body section 2R are connected by an articulation device 3. The articulation device 3 includes an articulation cylinder 7. The articulation cylinder 7 is a hydraulic cylinder. The articulation cylinder 7 connects the front vehicle body section 2F and the rear vehicle body section 2R. As the articulation cylinder 7 extends and retracts, the front vehicle body section 2F bends left and right relative to the rear vehicle body section 2R. As the front vehicle body section 2F bends relative to the rear vehicle body section 2R, the traveling direction of the work machine 1 is adjusted.

[0013] The traveling device 5 supports the vehicle body 2. The traveling device 5 travels on the ground at the work site. The traveling device 5 includes wheels 4. The wheels 4 are attached to each of the front vehicle body 2F and the rear vehicle body 2R.

[0014] The work implement 6 is supported by the vehicle body 2. The work implement 6 is connected to the vehicle body front portion 2F. The work implement 6 has a boom 8, a bucket 9, a bell crank 10, a bucket link 11, a lift cylinder 12, and a bucket cylinder 13.

[0015] The base end of boom 8 is rotatably connected to the front body 2F. Bucket 9 is a working member that excavates an excavation target. The base end of bucket 9 is rotatably connected to the tip end of boom 8. The middle portion of bell crank 10 is rotatably connected to bracket 14 of boom 8. The lower end of bell crank 10 is rotatably connected to the base end of bucket link 11. The tip end of bucket link 11 is rotatably connected to bracket 15 of bucket 9. Bell crank 10 is connected to bucket 9 via bucket link 11.

[0016] The boom 8 is operated by a lift cylinder 12. The lift cylinder 12 is a hydraulic cylinder. The base end of the lift cylinder 12 is connected to the front body 2F. The tip end of the lift cylinder 12 is connected to the boom 8. The bucket 9 is operated by a bucket cylinder 13. The bucket cylinder 13 is a hydraulic cylinder. The base end of the bucket cylinder 13 is connected to the front body 2F. The tip end of the bucket cylinder 13 is connected to the upper end of the bell crank 10.

[0017] In this embodiment, the work machine 6 is a front-loading type work machine in which the opening of the bucket 9 faces forward during excavation work. The boom 8 is raised or lowered by the extension and contraction of the lift cylinder 12. The bucket cylinder 13 is extended and contracted by the extension and contraction of the bucket 9, causing the bucket 9 to tilt or dump.

[0018] 2, the work machine 1 includes an engine 16, a power take-off (PTO) 17, a power transmission device 18, wheels 4, a brake device 19, a steering pump 20, a steering control valve 21, an articulate cylinder 7, a work implement pump 22, a work implement control valve 23, a lift cylinder 12, a bucket cylinder 13, a three-dimensional sensor 24, a camera 25, and a controller 26. The traveling device 5 includes the power transmission device 18, the brake device 19, and wheels 4.

[0019] The engine 16 is a drive source for the work machine 1. The engine 16 is supported by the vehicle body 2. An example of the engine 16 is a diesel engine. A power take-off 17 distributes the drive force of the engine 16 to a power transmission device 18, a steering pump 20, and a work implement pump 22.

[0020] The power transmission device 18 transmits the driving force of the engine 16 to the wheels 4. The power transmission device 18 controls the traveling speed and direction of travel of the work machine 1. The traveling direction of the work machine 1 includes forward and reverse. The power transmission device 18 may be a transmission having a torque converter, or may be a transmission having multiple speed change gears. The brake device 19 slows down or stops the work machine 1 while it is traveling.

[0021] The steering pump 20 is a hydraulic pump that is operated by the driving force generated by the engine 16. The hydraulic oil discharged from the steering pump 20 is supplied to the articulate cylinder 7 via a steering control valve 21. The steering control valve 21 controls the flow rate and direction of the hydraulic oil supplied from the steering pump 20 to the articulate cylinder 7. The articulate device 3 is operated by the hydraulic oil from the steering pump 20.

[0022] The work implement pump 22 is a hydraulic pump that is operated by the driving force generated by the engine 16. The hydraulic oil discharged from the work implement pump 22 is supplied to each of the lift cylinder 12 and the bucket cylinder 13 via a work implement control valve 23. The work implement control valve 23 controls the flow rate and direction of the hydraulic oil supplied from the work implement pump 22 to each of the lift cylinder 12 and the bucket cylinder 13. The work implement 6 is operated by the hydraulic oil from the work implement pump 22. The work implement 6 is operated by the hydraulic oil from the work implement pump 22.

[0023] The three-dimensional sensor 24 detects objects present in the vicinity of the work machine 1. The three-dimensional sensor 24 detects objects in a non-contact manner. The three-dimensional sensor 24 acquires three-dimensional data that indicates the three-dimensional shape of objects present in the vicinity of the work machine 1. The detection data of objects detected by the three-dimensional sensor 24 is three-dimensional data of the objects. The detection data of objects detected by the three-dimensional sensor 24 is transmitted to the controller 26.

[0024] The three-dimensional sensor 24 is mounted on the front body 2F. The three-dimensional sensor 24 acquires three-dimensional data of at least objects present in front of the work machine 1. An example of the three-dimensional sensor 24 is a laser sensor (LiDAR: Light Detection and Ranging) that detects objects by emitting laser light. The three-dimensional sensor 24 may also be a radar sensor (RADAR: Radio Detection and Ranging) that detects objects by emitting radio waves, or a stereo camera.

[0025] The three-dimensional data of the object includes a point cloud consisting of multiple detection points defined on the surface of the object. The point cloud indicates the relative distance and relative position between the three-dimensional sensor 24 and each of the multiple detection points defined on the surface of the object. In the following description, the detection data of the object detected by the three-dimensional sensor 24 will be referred to as detection point cloud data Ds as appropriate.

[0026] The camera 25 captures images of objects present in the vicinity of the work machine 1. The camera 25 acquires image data that shows a captured image Gp of objects present in the vicinity of the work machine 1. The image data of the captured image Gp captured by the camera 25 is transmitted to the controller 26.

[0027] Camera 25 is mounted on the front body 2F. Camera 25 captures images of objects that are present at least in front of the work machine 1. The detection range of three-dimensional sensor 24 and the imaging range of camera 25 overlap at least partially. When three-dimensional sensor 24 detects a first object, camera 25 captures an image of the first object detected by three-dimensional sensor 24. In parallel with the detection of the first object by three-dimensional sensor 24, the imaging of the first object by camera 25 is carried out.

[0028] 3 is a hardware configuration diagram of a controller 26 according to an embodiment. The controller 26 includes a computer system. The controller 26 has a processor 27 such as a CPU (Central Processing Unit), a main memory 28 including a non-volatile memory such as a ROM (Read Only Memory) and a volatile memory such as a RAM (Random Access Memory), a storage 29, and an interface 30 including an input / output circuit. The functions of the controller 26 are stored in the storage 29 as a computer program. The processor 27 reads the computer program from the storage 29, loads it into the main memory 28, and executes processing in accordance with the computer program. The computer program may be distributed to the controller 26 via a network.

[0029] [Work machine cycle work] FIG. 4 is a diagram illustrating the cycle work of the work machine 1 according to the embodiment. The work machine 1 performs cycle work by repeating a series of tasks. A cycle work is made up of a number of tasks that are interrelated. A cycle work includes the travel of the work machine 1 and the operation of the work implement 6. In the embodiment, the cycle work is made up of a series of six tasks. In the embodiment, the cycle work includes an empty load forward operation M1, an excavation operation M2, a loaded load reverse operation M3, a loaded load forward operation M4, a loading operation M5, and an empty load reverse operation M6.

[0030] The sequence of the series of operations is fixed. After the empty load forwarding operation M1 is performed, the excavation operation M2 is performed. After the excavation operation M2 is performed, the loaded load reversing operation M3 is performed. After the loaded load reversing operation M3 is performed, the loaded load forwarding operation M4 is performed. After the loaded load forwarding operation M4 is performed, the loading operation M5 is performed. After the loading operation M5 is performed, the empty load reversing operation M6 is performed.

[0031] The work machine 1 performs cycle work, which is a series of operations repeated multiple times. After a first empty-load backward movement operation M6 is performed, a second empty-load forward movement operation M1 is performed.

[0032] In cyclical work, the work machine 1 operates based on work data that has been generated in advance. The work data includes travel data that indicates the travel conditions of the work machine 1, and work machine data that indicates the operating conditions of the work machine 6. The travel data includes a target travel path 31 for the work machine 1, a target position for the work machine 1, a target orientation for the work machine 1, and a target travel speed for the work machine 1. The work machine data includes a target movement path for the work machine 6, a target position for the work machine 6, and a target movement speed for the work machine 6.

[0033] The target travel path 31 is defined in a global coordinate system. The target travel path 31 is defined by a trajectory that passes through multiple travel points. The travel points define the target position of the work machine 1. A target heading and target travel speed for the work machine 1 are set for each of the multiple travel points. The multiple travel points are set at intervals. The intervals between the travel points may be uniform or uneven. The target travel path 31 is a virtual line connecting the multiple travel points. The target position refers to the target position of the work machine 1 when the work machine 1 passes through a travel point. The target position of the work machine 1 is defined in a global coordinate system. The target heading of the work machine 1 refers to the target heading of the work machine 1 when it passes through a travel point. The target travel speed of the work machine 1 refers to the target travel speed of the work machine 1 when it passes through a travel point.

[0034] Empty-load forward movement work M1 is work of moving forward to approach the excavation target. In the embodiment, the excavation target is a mound of natural ground 32 placed on the ground. The mound of natural ground 32 refers to a pile of earth and sand. In empty-load forward movement work M1, the work machine 1 moves forward to approach the mound of natural ground 32 based on the traveling data. When the work machine 1 moves forward, the attitude of the work machine 6 is controlled based on the work machine data so that the bucket 9 excavates the mound of natural ground 32. The attitude of the work machine 6 is controlled so that the cutting edge of the bucket 9 approaches the ground. When the cutting edge of the bucket 9 is close to the ground, the work machine 1 moves forward, and the cutting edge of the bucket 9 is inserted into the lower end of the mound of natural ground 32.

[0035] Excavation work M2 is work to excavate an excavation target with the bucket 9 of the work implement 6. Based on the work implement data, the work machine 1 tilts the bucket 9 after the cutting edge of the bucket 9 is inserted into the natural ground 32. As a result, the natural ground 32 is excavated by the bucket 9. The bucket 9 scoops up the excavated material. The excavated material is held in the bucket 9.

[0036] The load reversing operation M3 is an operation in which the work machine 1 moves backward so as to move away from the excavation target with the excavated material held in the bucket 9 of the work machine 6. In the load reversing operation M3, the work machine 1 moves backward so as to move away from the natural ground 32 based on the traveling data. In addition, the attitude of the work machine 6 is controlled so that the excavated material does not spill out of the bucket 9.

[0037] Load forward operation M4 is an operation in which the work machine 1 moves forward to approach the loading target. In load reverse operation M3, the work machine 1 moves backward toward switchback point 31P defined on the target travel path 31. After switching back at switchback point 31P, the work machine 1 transitions to load forward operation M4. Switchback point 31P refers to the position at which the work machine 1 switches back. Switchback refers to the operation in which the work machine 1, while moving backward, changes direction of travel at an acute angle and moves forward.

[0038] In this embodiment, the loading target is the dump body 34 of a dump truck 33 that is capable of traveling on the ground. After the load reversing operation M3 is completed, the work machine 1 moves forward while turning so as to approach the dump truck 33. In load advancing operation M4, the work machine 1 moves forward so as to approach the dump truck 33 based on the traveling data.

[0039] The loading operation M5 is an operation of loading the excavated material held in the bucket 9 of the work implement 6 onto a loading target. Based on the work implement data, the attitude of the work implement 6 is controlled so that the excavated material held in the bucket 9 is loaded into the dump body 34. Based on the work implement data, the attitude of the work implement 6 is controlled so that the excavated material does not spill out of the bucket 9 and so that the bucket 9 is positioned above the upper end of the dump body 34.

[0040] The empty reverse operation M6 is an operation in which the work machine 1 moves backward so as to move away from the loading target. After the loading operation M5 is completed, the work machine 1 moves backward so as to move away from the dump truck 33 based on the traveling data.

[0041] The work machine 1 repeats a cycle of operations including an empty forward operation M1, an excavation operation M2, a loaded reverse operation M3, a loaded forward operation M4, a loading operation M5, and an empty reverse operation M6 until the excavated material is loaded onto the dump body 34 of the dump truck 33 at the target load amount.

[0042] Figure 5 is a diagram that schematically shows a work machine 1 performing a load forward operation M4 according to an embodiment. In the load forward operation M4, after completing the load reverse operation M3, the work machine 1 moves forward while turning from switchback point 31P so as to approach the dump truck 33. The three-dimensional sensor 24 detects objects ahead of the work machine 1, and the camera 25 captures images of the objects ahead of the work machine 1. In the load forward operation M4, the controller 26 identifies the dump truck 33 from multiple objects present ahead of the work machine 1 based on the detection data of the objects detected by the three-dimensional sensor 24. The controller 26 controls the operation of the work machine 1 so that the above-mentioned cyclical work is performed based on the dump truck 33 identification results and the work data.

[0043] Identifying the dump truck 33 includes calculating the position and attitude of the dump truck 33. In the embodiment, the controller 26 converts the captured image Gp captured by the camera 25 into a converted image Gc viewed from a predetermined viewpoint different from the viewpoint of the camera 25, based on the position and attitude of the dump truck 33 and the relative positions of the three-dimensional sensor 24 and the camera 25. The controller 26 can identify the dump truck 33 based on the converted image Gc.

[0044] [Dump truck] 6 is a diagram showing a dump truck 33 according to the embodiment. As shown in FIG. 6, the dump truck 33 has a vehicle body 35, a traveling device 36, a dump body 34, and a hoist cylinder 37.

[0045] The vehicle body 35 has a cab 38. The cab 38 is provided at the front of the vehicle body 35. The traveling device 36 travels on the ground at the work site. The traveling device 36 has a plurality of wheels 39. The wheels 39 include tires. The wheels 39 have axles 39A. The axles 39A are disposed at the centers of the wheels 39. At least a portion of the wheels 39 contact the ground at the work site. The wheels 39 have contact surfaces 39T that contact the ground.

[0046] The hoist cylinder 37 raises and lowers the dump body 34. One end of the hoist cylinder 37 is rotatably connected to the vehicle body 35. The other end of the hoist cylinder 37 is rotatably connected to the dump body 34. The dump body 34 is a container (vessel) of the dump truck 33 into which cargo is loaded. The dump body 34 rises and lowers by operation of the hoist cylinder 37.

[0047] [Identification System] 7 is a block diagram showing a control system 100 for a work machine 1 according to an embodiment. The control system 100 includes an identification system 200 that identifies the dump truck 33 from a plurality of objects present in the vicinity of the work machine 1.

[0048] The identification system 200 includes a three-dimensional sensor 24, a camera 25, and a controller 26. Detection data of an object detected by the three-dimensional sensor 24 is transmitted to the controller 26. A captured image Gp of the object captured by the camera 25 is transmitted to the controller 26.

[0049] Based on the detection data of the object detected by the three-dimensional sensor 24, the controller 26 identifies the dump truck 33 from multiple objects present in the vicinity of the work machine 1. Based on the dump truck 33 identification results and the work data, the controller 26 controls the operation of the work machine 1 so that the above-mentioned cyclic work is carried out.

[0050] The controller 26 includes a standard data storage unit 40 , a data acquisition unit 41 , a removal unit 42 , an identification unit 43 , a processing unit 44 , and an operation control unit 45 .

[0051] The standard data storage unit 40 stores standard data that indicates the three-dimensional shape of the dump truck 33. The standard data is generated in advance and stored in the standard data storage unit 40.

[0052] 8 is a diagram illustrating standard data according to the embodiment. In the embodiment, the standard data includes a plurality of point clouds defined on the surface of the dump truck 33. The plurality of point clouds indicate the three-dimensional shape of the dump truck 33. In the following description, the standard data will be referred to as standard point cloud data Dt as appropriate.

[0053] The standard point cloud data Dt may be generated from design data of the dump truck 33. The design data of the dump truck 33 includes three-dimensional CAD (Computer Aided Design) data. The standard point cloud data Dt does not have to be generated from design data. The dump truck may be detected in advance by a three-dimensional sensor, and the standard point cloud data Dt may be generated based on detection data indicating the three-dimensional shape of the dump truck 33 detected by the three-dimensional sensor. The three-dimensional sensor that detects the dump truck 33 to generate the standard point cloud data Dt may be the three-dimensional sensor 24 mounted on the work machine 1, or may be a three-dimensional sensor separate from the three-dimensional sensor 24. The standard point cloud data Dt may be generated without using three-dimensional CAD or a three-dimensional sensor. For example, the shape of the dump truck 33 may be extracted from an image of the dump truck 33, and the standard point cloud data Dt may be generated based on the extracted shape of the dump truck 33.

[0054] The data acquisition unit 41 acquires detection data of objects in the vicinity of the work machine 1 detected by the three-dimensional sensor 24. In other words, the data acquisition unit 41 acquires detection point cloud data Ds of objects in the vicinity of the work machine 1. The data acquisition unit 41 also acquires image data indicating a captured image Gp of objects in the vicinity of the work machine 1 captured by the camera 25.

[0055] The removal unit 42 removes a predetermined portion Up including the ground surface of the object from at least one of the standard point cloud data Dt and the detected point cloud data Ds. In an embodiment, the removal unit 42 does not remove the predetermined portion from the detected point cloud data Ds, but removes the predetermined portion Up from the standard point cloud data Dt.

[0056] Fig. 9 is a diagram illustrating the standard point cloud data Dt from which the predetermined portion Up has been removed according to the embodiment. As shown in Fig. 9, the removal unit 42 removes the predetermined portion Up including the ground contact surface 39T of the wheel 39 from the standard point cloud data Dt of the dump truck 33. The removed portion Up is a lower portion of the standard point cloud data Dt of the dump truck 33 including the ground contact surface 39T. In the example shown in Fig. 9, the removed portion Up is a portion below the center of the axle 39A of the wheel 39. Note that the removed portion Up may be a portion below the upper end of the axle 39A or a portion below the lower end of the axle 39A.

[0057] The removal unit 42 removes a predetermined portion Up from the standard point cloud data Dt so as to leave the characteristic portion of the standard point cloud data Dt of the dump truck 33. The characteristic portion of the standard point cloud data Dt of the dump truck 33 includes the dump body 34 into which cargo is loaded. The dump body 34 is a container into which cargo is loaded.

[0058] After the predetermined portion Up has been removed, the identification unit 43 compares the standard point cloud data Dt with the detected point cloud data Ds to identify the dump truck 33 from among the objects present in the vicinity of the work machine 1 detected by the three-dimensional sensor 24. In the embodiment, the identification unit 43 compares the standard point cloud data Dt from which the predetermined portion Up has been removed with the detected point cloud data Ds to identify the dump truck 33 from among the objects present in the vicinity of the work machine 1. The identification unit 43 identifies whether the object detected by the three-dimensional sensor 24 is a dump truck 33.

[0059] Collating the standard point cloud data Dt with the detected point cloud data Ds includes matching the standard point cloud data Dt with the detected point cloud data Ds. Matching the standard point cloud data Dt with the detected point cloud data Ds includes searching for relative positions between the standard point cloud data Dt and the detected point cloud data Ds that result in a high degree of match between the standard point cloud data Dt and the detected point cloud data Ds. The identification unit 43 performs a search process to search for relative positions between the standard point cloud data Dt and the detected point cloud data Ds that result in a high degree of match between the standard point cloud data Dt and the detected point cloud data Ds.

[0060] The identification unit 43 performs a search process based on a specified algorithm. Examples of the specified algorithm include an ICP (Iterative Closest Point) algorithm, an NDT (Normal Distributions Transform) scan matching, and a FAST (Features from Accelerated Segment Test) feature point matching. The identification unit 43 matches the standard point cloud data Dt with the detected point cloud data Ds based on the specified algorithm.

[0061] 10 is a diagram illustrating the identification process performed by the identification unit 43 according to the embodiment. There is a possibility that cargo (earth and sand) will spill from the dump body 34 of the dump truck 33. There is a possibility that the cargo spilled from the dump body 34 will be deposited on the ground around the dump truck 33. As shown in FIG. 10, when a cargo deposit Nt is generated on the ground around the dump truck 33, the detection point cloud data Ds will include not only three-dimensional data of the dump truck 33 but also three-dimensional data of the deposit Nt.

[0062] As described above, the work machine 1 performs cyclical work. For example, if a load spills from the dump body 34 during loading operation M5 in the first cyclical work and a deposit Nt is generated on the ground around the dump truck 33, when identifying the dump truck 33 during load forwarding operation M4 in the second cyclical work, the deposit Nt may become noise and reduce the accuracy of identifying the dump truck 33.

[0063] According to the embodiment, a predetermined portion Up including the ground contact surface 39T is removed from the standard point cloud data Dt. The identification unit 43 compares the standard point cloud data Dt of the portion above the axle 39A with the detected point cloud data Ds. The portion below the axle 39A in the detected point cloud data Ds is not a target for comparison with the standard point cloud data Dt. Because the deposit Nt is located below the axle 39A, in the detected point cloud data Ds, the deposit Nt is not a target for comparison with the standard point cloud data Dt. Even if the detected point cloud data Ds includes three-dimensional data of the deposit Nt, the deposit Nt is not a target for comparison with the standard point cloud data Dt, and therefore a decrease in the identification accuracy of the dump truck 33 is suppressed.

[0064] The processing unit 44 converts the captured image Gp of the dump truck 33 into a converted image Gc viewed from a predetermined viewpoint different from the viewpoint of the camera 25 based on the position and posture of the dump truck 33 and the relative positions of the three-dimensional sensor 24 and the camera 25.

[0065] Identifying the dump truck 33 by comparing the standard point cloud data Dt with the detected point cloud data Ds includes calculating the position and attitude of the dump truck 33. By identifying the dump truck 33, the identification unit 43 can calculate the position and attitude of the dump truck 33 relative to the work machine 1. The identification unit 43 can calculate the relative position and relative angle of the dump truck 33 relative to the work machine 1 based on the three-dimensional data of the dump truck 33.

[0066] The relative positions of three-dimensional sensor 24 and camera 25 are known data and are derived in advance by, for example, a calibration process.

[0067] The processing unit 44 converts the captured image Gp into a converted image Gc viewed from a predetermined viewpoint different from the viewpoint of the camera 25 based on the position and posture of the dump truck 33 calculated by identifying the dump truck 33 and the relative position of the three-dimensional sensor 24 and the camera 25, which is known data.

[0068] In the embodiment, the specified viewpoint is a viewpoint that exists in a direction perpendicular to the side of the dump body 34. During the cargo forwarding operation M4, the work machine 1 approaches the dump truck 33 from a direction that is inclined toward the side of the dump body 34. During the cargo forwarding operation M4, the viewpoint of the camera 25 exists in a direction that is inclined toward the side of the dump body 34. During the cargo forwarding operation M4, the processing unit 44 converts the captured image Gp into a converted image Gc viewed from a direction that is perpendicular to the side of the dump body 34.

[0069] 11 is a diagram illustrating the relative positions of the three-dimensional sensor 24, camera 25, and dump truck 33 according to the embodiment. The position where the three-dimensional sensor 24 is attached on the work machine 1 is different from the position where the camera 25 is attached. The coordinate system of the three-dimensional sensor 24 is different from the coordinate system of the camera 25. If the points at the four corners of the side of the dump body 34 are defined as points P1, P2, P3, and P4, the positions of points P1, P2, P3, and P4 in the coordinate system of the three-dimensional sensor 24 are different from the positions of points P1, P2, P3, and P4 in the coordinate system of the camera 25.

[0070] When converting the viewpoint of the camera 25 to a predetermined viewpoint, the identification unit 43 extracts the side of the dump body 34 from the outer surface of the dump truck 33 based on the detected point cloud data Ds. The side of the dump body 34 is substantially flat. The processing unit 44 projects points P1, P2, P3, and P4 on the side of the dump body 34 onto the captured image Gp. The processing unit 44 converts the viewpoint of the camera 25 to the predetermined viewpoint based on the points P1, P2, P3, and P4 projected onto the captured image Gp. The processing unit 44 generates a converted image Gc so that the rectangle defined by the points P1, P2, P3, and P4 projected onto the captured image Gp becomes a rectangle when viewed from the predetermined viewpoint.

[0071] 12 is a diagram showing a captured image Gp and a converted image Gc according to the embodiment. During the cargo forward movement operation M4, the work machine 1 approaches the dump truck 33 from a direction inclined toward the side of the dump body 34. Therefore, the captured image Gp captured by the camera 25 during the cargo forward movement operation M4 is an image seen from the viewpoint of the camera 25 that is located in a direction inclined toward the side of the dump body 34. The processing unit 44 converts the captured image Gp seen from the direction inclined toward the side of the dump body 34 into a converted image Gc seen from a direction perpendicular to the side of the dump body 34, based on the position and posture of the dump truck 33 and the relative positions of the 3D sensor 24 and the camera 25.

[0072] The processing unit 44 can identify the dump truck 33 based on the converted image Gc. The processing unit 44 can identify whether the converted image Gc is the dump truck 33 or not by using a pattern matching technique, for example.

[0073] The operation control unit 45 automatically controls the work machine 1 so that it performs cyclic work, based on the work data and the identification result of the dump truck 33 by the processing unit 44. The operation control unit 45 outputs control commands to control each of the articulating unit 3, traveling unit 5, and work implement 6, based on the work data. The operation control unit 45 outputs control commands to each of the articulating unit 3 and traveling unit 5 so that the work machine 1 travels based on the traveling data. The operation control unit 45 outputs control commands to the work implement 6 so that the work implement 6 operates based on the work implement data.

[0074] The work machine 1 has a position sensor that detects the position of the work machine 1, an orientation sensor that detects the orientation of the work machine 1, and a speed sensor that detects the traveling speed of the work machine 1. The position sensor includes a Global Navigation Satellite System (GNSS) receiver that uses the GNSS to detect the position of the work machine 1 in a global coordinate system. The orientation sensor includes a calculator that calculates the orientation from position data detected by two GNSS antennas provided on the work machine 1. The operation control unit 45 controls the articulating unit 3 and the traveling unit 5 based on the detection data from the position sensor and the detection data from the orientation sensor so that the work machine 1 travels along the target traveling path 31. The operation control unit 45 also controls the traveling unit 5 based on the detection data from the speed sensor so that the work machine 1 travels at a target traveling speed.

[0075] [Identification method] 13 is a flowchart showing a control method for the work machine 1 according to the embodiment. The operation control unit 45 starts cyclic work based on work data including travel data and work machine data. The three-dimensional sensor 24 detects objects present in the vicinity of the work machine 1. The camera 25 captures images of objects present in the vicinity of the work machine 1. The data acquisition unit 41 acquires detected point cloud data Ds, which is detection data of the objects detected by the three-dimensional sensor 24, and image data indicating the captured image Gp captured by the camera 25 (step S1).

[0076] The removal unit 42 removes a predetermined portion Up from the standard point cloud data Dt stored in the standard data storage unit 40 (step S2). Note that the standard point cloud data Dt from which the predetermined portion Up has been removed may be stored in advance in the standard data storage unit 40.

[0077] The identification unit 43 compares the standard point cloud data Dt from which the predetermined portion Up has been removed with the detected point cloud data Ds, and identifies the dump truck 33 from the objects present around the work machine 1 (step S3).

[0078] The processing unit 44 performs viewpoint conversion on the captured image Gp based on the position and posture of the dump truck 33 calculated by the identification unit 43 and the relative position between the three-dimensional sensor 24 and the camera 25 to generate a converted image Gc (step S4).

[0079] The processing unit 44 identifies the dump truck 33 based on the converted image Gc (step S5).

[0080] The operation control unit 45 automatically controls the work machine 1 so that the work machine 1 performs cyclic work based on the identification result and work data of the dump truck 33. In the cargo forwarding work M4 and loading work M5, the operation control unit 45 controls the work machine 1 so that a cargo is loaded onto the identified dump truck 33 (step S6).

[0081] The operation control unit 45 determines whether or not to end the cycle work (step S7). If it is determined in step S7 that the cycle work should not be ended (step S7: No), the process returns to step S1. If it is determined in step S7 that the cycle work should be ended (step S7: Yes), the cycle work ends.

[0082] [effect] As described above, in the embodiment, the identification system 200 includes the 3D sensor 24 that detects objects present in the vicinity of the work machine 1, and the controller 26. The controller 26 includes a standard data storage unit 40 that stores standard point cloud data Dt that indicates the three-dimensional shape of the dump truck 33 onto which a load is to be loaded by the work machine 1, a data acquisition unit 41 that acquires detected point cloud data Ds of the object detected by the 3D sensor 24, a removal unit 42 that removes a predetermined portion Up including the ground contact surface 39T from the standard point cloud data Dt, and an identification unit 43 that compares the standard point cloud data Dt with the detected point cloud data Ds after the predetermined portion Up has been removed, to identify the dump truck 33 from objects present in the vicinity of the work machine 1.

[0083] According to the embodiment, even if a cargo deposit Nt is generated on the ground around the dump truck 33 due to cargo spilled from the dump body 34, the predetermined portion Up including the ground contact surface 39T is removed from the standard point cloud data Dt, thereby suppressing a decrease in the identification accuracy of the dump truck 33. Since the predetermined portion Up including the ground contact surface 39T is removed from the standard point cloud data Dt, even if the detection point cloud data Ds includes three-dimensional data of the deposit Nt, the deposit Nt is not a target for comparison with the standard point cloud data Dt. Therefore, a decrease in the identification accuracy of the dump truck 33 is suppressed.

[0084] The identification unit 43 calculates the position and posture of the dump truck 33 by identifying the dump truck 33. The processing unit 44 converts the captured image Gp of the dump truck 33 captured by the camera 25 into a converted image Gc viewed from a predetermined viewpoint different from the viewpoint of the camera 25, based on the position and posture of the dump truck 33 and the relative positions of the 3D sensor 24 and the camera 25. The processing unit 44 identifies the dump truck 33 based on the converted image Gc. The converted image Gc improves the identification accuracy of the dump truck 33. The operation control unit 45 can accurately perform the cargo forward operation M4 and the loading operation M5 based on the identification result of the dump truck 33 using the converted image Gc.

[0085] [Other embodiments] In the above-described embodiment, the predetermined portion Up to be removed from the standard point cloud data Dt is the portion below the axle 39A. The predetermined portion Up to be removed from the standard point cloud data Dt may be the portion below the lower end of the dump body 34.

[0086] In the above-described embodiment, the removal unit 42 removes the predetermined portion Up from the standard point cloud data Dt so that the dump body 34 remains as a characteristic portion of the standard point cloud data Dt of the dump truck 33. The removal unit 42 may also remove the predetermined portion Up from the standard point cloud data Dt so that the cab 38 remains as a characteristic portion of the standard point cloud data Dt of the dump truck 33. The cab 38 is provided in the front part of the body 35 of the dump truck 33. When the characteristic portion of the standard point cloud data Dt of the dump truck 33 is the cab 38 provided in the front part of the body 35, the identification unit 43 can identify the front and rear of the dump truck 33.

[0087] In the above-described embodiment, the removal unit 42 removes the predetermined portion Up from the standard point cloud data Dt. The removal unit 42 may also remove the predetermined portion including the contact surface of the object from the detected point cloud data Ds. The identification unit 43 may compare the standard point cloud data Dt with the detected point cloud data Ds from which the predetermined portion has been removed to identify the dump truck 33 from the objects in the vicinity of the work machine 1.

[0088] In the above-described embodiment, the loading target is the dump body 34 of the dump truck 33. The loading target may be, for example, a hopper placed on the ground.

[0089] In the above-described embodiment, the three-dimensional sensor 24 is mounted on the work machine 1. The three-dimensional sensor 24 may be mounted on a work machine other than the work machine 1 operating at the work site, or may be installed on a structure present at the work site.

[0090] In the above-described embodiment, the camera 25 is mounted on the work machine 1. The camera 25 may be mounted on a work machine other than the work machine 1 operating at the work site, or may be installed on a structure present at the work site.

[0091] In the above-described embodiment, the processing unit 44 may be omitted. The camera 25 may be omitted. The operation control unit 45 may control the operation of the work machine 1 based on the dump truck 33 identified on the basis of the detection point cloud data Ds.

[0092] In the above-described embodiment, a driver may ride on the work machine 1. In the above-described embodiment, each of the articulating device 3, the traveling device 5, and the work implement 6 may be operated by driving operations by the driver. Alternatively, the articulating device 3 and the traveling device 5 may be automatically controlled, and the work implement 6 may be operated based on driving operations by the driver.

[0093] In the above-described embodiment, a driver does not have to ride on the work machine 1. The work machine 1 may be operated based on an operation signal from a remote control device provided outside the work machine 1. In other words, the work machine 1 may be remotely controlled.

[0094] In the above-described embodiment, the work machine 1 is a wheel loader. The work machine 1 may be any loading machine that loads a load onto a loading target, and may be, for example, a hydraulic excavator.

[0095] [Note] The present disclosure may also have the following configurations. (Appendix 1) a 3D sensor for detecting an object; a controller; The controller a standard data storage unit that stores standard data indicating a three-dimensional shape of a loading target onto which a load is to be loaded by the work machine; a data acquisition unit that acquires detection data of the object detected by the three-dimensional sensor; a removal unit that removes a predetermined portion including a ground contact surface from at least one of the standard data and the detection data; and an identification unit that, after the predetermined portion is removed, compares the standard data with the detection data to identify the loading target from the object. Identification system. (Appendix 2) the removal unit does not remove the predetermined portion from the detection data, but removes the predetermined portion from the standard data. (Appendix 1) An identification system as described in. (Appendix 3) the removal unit removes a predetermined portion from the standard data so that a characteristic portion of the standard data remains; The characteristic portion includes the container to be loaded into which the cargo is to be loaded. An identification system according to (Appendix 1) or (Appendix 2). (Appendix 4) The loading object is a dump truck having wheels, The predetermined portion is a portion below the axle of the wheel. An identification system according to any one of (Appendix 1) to (Appendix 3). (Appendix 5) the container is a dump body of the dump truck, The predetermined portion is a portion lower than the lower end portion of the dump body. (Appendix 4) An identification system as described in. (Appendix 6) The characteristic portion includes a cab of the dump truck. (Appendix 5) An identification system as described in. (Appendix 7) a camera for capturing an image of the object; the data acquisition unit acquires an image of the object captured by the camera, Identifying the loading object by the matching includes calculating a position and an attitude of the loading object; a processing unit that converts the captured image into a converted image viewed from a predetermined viewpoint different from the viewpoint of the camera based on the position and posture of the loading target and the relative position between the three-dimensional sensor and the camera, An identification system according to any one of (Appendix 1) to (Appendix 6). (Appendix 8) The controller generating standard data indicating a three-dimensional shape of a loading target onto which a load is to be loaded by a work machine; acquiring detection data of objects present in the vicinity of the work machine detected by a three-dimensional sensor; removing a predetermined portion including a ground plane from at least one of the standard data and the detection data; After the predetermined portion is removed, the standard data is compared with the detection data to identify the loading target from the object. Identification method. [Explanation of symbols]

[0096] 1...work machine, 2...body, 2F...front body, 2R...rear body, 3...articulation device, 4...wheel, 5...traveling device, 6...work implement, 7...articulation cylinder, 8...boom, 9...bucket, 10...bell crank, 11...bucket link, 12...lift cylinder, 13...bucket cylinder, 14...bracket, 15...bracket, 16...engine, 17...power take-off, 18...power transmission device, 19...brake device, 20...steering pump, 21...steering control valve, 22...work implement pump, 23...work implement control valve, 24...3D sensor, 25...camera, 26...controller, 27...processor, 28...main memory, 29...storage, 30...interface base, 31...target driving route, 31P...switchback point, 32...natural ground, 33...dump truck, 34...dump body, 35...vehicle body, 36...traveling gear, 37...hoist cylinder, 38...cab, 39...wheel, 39A...axle, 39T...contact surface, 40...standard data storage unit, 41...data acquisition unit, 42...removal unit, 43...identification unit, 44...processing unit, 45...motion control unit, 100...control system, 200...identification system, Ds...detected point cloud data (detected data), Dt...standard point cloud data (standard data), Gc...converted image, Gp...captured image, M1...empty load forward operation, M2...excavation operation, M3...loaded load backward operation, M4...loaded load forward operation, M5...loading operation, M6...empty load backward operation, Nt...deposit, Up...specified part.

Claims

1. a three-dimensional sensor for detecting an object; a controller; The controller a standard data storage unit that stores standard data indicating a three-dimensional shape of a loading target onto which a load is to be loaded by a work machine; a data acquisition unit that acquires detection data of the object detected by the three-dimensional sensor; a removal unit that removes a predetermined portion including a ground contact surface from at least one of the standard data and the detection data; and an identification unit that, after the predetermined portion is removed, compares the standard data with the detection data to identify the loading target from the object. Identification system.

2. the removal unit does not remove the predetermined portion from the detection data, but removes the predetermined portion from the standard data. The identification system of claim 1 .

3. the removal unit removes a predetermined portion from the standard data so that a characteristic portion of the standard data remains; The characteristic portion includes the container to be loaded into which the cargo is to be loaded. The identification system of claim 2 .

4. The loading object is a dump truck having wheels, The predetermined portion is a portion below the axle of the wheel. The identification system of claim 3 .

5. the container is a dump body of the dump truck, The predetermined portion is a portion lower than the lower end portion of the dump body. The identification system of claim 4 .

6. The characteristic portion includes a cab of the dump truck. The identification system of claim 5 .

7. a camera for capturing an image of the object; the data acquisition unit acquires an image of the object captured by the camera, Identifying the loading object by the matching includes calculating a position and an attitude of the loading object; a processing unit that converts the captured image into a converted image viewed from a predetermined viewpoint different from a viewpoint of the camera based on the position and posture of the loading target and the relative position between the three-dimensional sensor and the camera, The identification system of claim 1 .

8. The controller generating standard data indicating a three-dimensional shape of a loading target onto which a load is to be loaded by a work machine; Obtaining detection data of an object detected by a three-dimensional sensor; removing a predetermined portion including a ground plane from at least one of the standard data and the detection data; After the predetermined portion is removed, the standard data is compared with the detection data to identify the loading target from the object. Identification method.

Images