Work machine

Through the combination of external measurement and information processing devices, the problem of inaccurate identification of the position and posture of the transport machinery in the prior art is solved, and the correct identification and safety control in various situations is achieved, thereby reducing the collision risk of loading operations. Improves safety and productivity.

CN118946699APending Publication Date: 2024-11-12HITACHI CONSTRUCTION MACHINERY CO LTD
View PDF 5 Cites 0 Cited by

Patent Information

Application Number
CN202380030770.8
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Priority Date
2022-04-04
Filing Date
2023-04-03
Publication Date
2024-11-12

AI Technical Summary

Technical Problem

The prior art is difficult to ensure the accuracy and safety of the identification results when identifying the position and posture of the transport machinery, especially in the absence of learning data or the transparency of the inter-enterprise identification system algorithm, resulting in possible collisions of loading operations.

Method used

The external measurement device, information processing device and control device are used to measure the surrounding environment of the working machine, obtain the position and vehicle parameter information of the transport machine, modify the identification results, and ensure the accuracy of the action control.

Benefits of technology

Correctly identify the position and posture of the handling machinery under various conditions, reduce the collision risk of loading operations, and improve safety and productivity.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118946699A_ABST
    Figure CN118946699A_ABST
Patent Text Reader

Abstract

Provided is a work machine capable of verifying the validity of the recognition result of a transport machine and appropriately controlling the loading work on the transport machine. A hydraulic shovel (1) is provided with: an external measurement device (70) that measures the surroundings of the hydraulic shovel (1); an information processing device (54) that identifies a dump truck (200) that exists in the periphery of the hydraulic shovel (1) on the basis of the measurement results of the external measurement device (70); a control device (40) that controls the operation of the vehicle body of the hydraulic shovel (1) on the basis of the recognition result of the information processing device (54); and a dump truck information acquisition device (56) that acquires the position of the dump truck (200) and vehicle parameter information by external communication. The information processing device (54) modifies the recognition result of the tipping truck (200) on the basis of the information acquired by the tipping truck information acquisition device (56). The control device (40) controls the operation of the vehicle body of the hydraulic shovel (1) on the basis of the modified recognition result of the dump truck (200).
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to a working machine. Background Art

[0002] A multi-jointed working machine (e.g., a hydraulic excavator) having a front working device (e.g., a boom, an arm, and attachments such as a bucket) driven by a hydraulic actuator is known. This working machine performs a loading operation of loading excavated earth and sand and other objects onto a transporting machine (e.g., a dump truck) as a loaded machine.

[0003] When performing loading operations, if the front working device (for example, the height of the bucket) rotates at a position where the height is lower than the transporting machine, there is a possibility that the front working device will collide with the transporting machine. Therefore, a function to assist the operation of the operator of the working machine performing the loading operation and a technology for the working machine to automatically perform the loading operation are required. When the working machine performs the loading operation semi-automatically or fully automatically (hereinafter collectively referred to as "automatic"), the working machine needs to correctly recognize the position and posture of the transporting machine.

[0004] As a conventional technology for identifying a handling machine, there is a technology described in Patent Document 1, for example. Patent Document 1 discloses an image processing system comprising: a data acquisition unit that acquires a photographed image of an unloading object of a conveyed object of a working machine; an area determination unit that determines an area including the unloading object based on the photographed image; and an object determination unit that determines at least one predetermined surface of the unloading object based on the area including the unloading object.

[0005] Prior art literature

[0006] Patent Literature

[0007] Patent Document 1: Japanese Patent Application Publication No. 2020-126363 Summary of the invention

[0008] The technology disclosed in Patent Document 1 detects the position and posture of a handling machine based on images captured by a camera installed on the working machine by using an image recognition system using machine learning to identify the handling machine. However, for a recognition system using machine learning, it is difficult to ensure the validity and accuracy of the recognition results for a site or handling machine for which no learning data has been collected.

[0009] In addition, when another company's system is incorporated as the recognition system of the handling machine, there is a possibility that the details of the recognition system algorithm are not disclosed to the developer of the working machine. It is not easy for the developer of the working machine to verify the recognition result of the recognition system. It is difficult for the developer of the working machine to determine in advance whether the recognition result of the recognition system can be used to properly control the loading operation without colliding with the handling machine.

[0010] The present invention has been made in view of the above circumstances, and an object of the present invention is to provide a working machine capable of accurately recognizing the position and posture of a transport machine in various situations, thereby appropriately controlling the loading work on the transport machine.

[0011] In order to solve the above-mentioned problems, the working machine of the present invention is a working machine that loads an object onto a handling machine, and is characterized in that it has: an external measuring device that measures the surrounding environment of the body of the working machine; an information processing device that identifies the handling machine that exists around the body based on the measurement result of the external measuring device; a control device that controls the movement of the body based on the recognition result of the information processing device; and a handling machine information acquisition device that acquires the position of the handling machine and vehicle parameter information from the outside, the information processing device modifies the recognition result of the handling machine based on the position of the handling machine and the vehicle parameter information acquired by the handling machine information acquisition device, and the control device controls the movement of the body based on the modified recognition result of the handling machine.

[0012] Effects of the Invention

[0013] According to the present invention, it is possible to provide a working machine capable of accurately recognizing the position and posture of a transport machine in various situations, thereby appropriately controlling the loading work on the transport machine.

[0014] Other problems, structures, and effects than those described above will become apparent from the following description of the embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 It is a side view schematically showing the external appearance structure of a hydraulic excavator which is an example of the working machine according to the first embodiment.

[0016] Figure 2 Yes means Figure 1 This is a block diagram showing the configuration of a hydraulic system and a control system mounted on a hydraulic excavator.

[0017] Figure 3 Yes Description Figure 2 FIG. 1 is a diagram showing an example of the operation of the hydraulic excavator.

[0018] Figure 4 Yes means Figure 2 A flowchart showing an example of the operation of the hydraulic excavator is shown.

[0019] Figure 5 Yes Description Figure 2 A block diagram of the functional structure of the information processing device shown.

[0020] Figure 6 This is a diagram showing each coordinate system set for the hydraulic excavator as viewed from the side.

[0021] Figure 7 Observed from above Figure 6 Plots of the various coordinate systems shown.

[0022] Figure 8 This is a diagram showing the body coordinate system of a dump truck as viewed from the side.

[0023] Fig. 9 Observed from above Figure 8 Diagram of the body coordinate system shown.

[0024] Fig.10 This is a flowchart showing an example of recognition result verification processing.

[0025] Fig.11 It is a diagram for explaining the degree of overlap between the recognition area and the estimation area.

[0026] Fig.12 This is a diagram showing a table that defines a criterion for determining the validity of a recognition result and a method for modifying a recognition area for each task of a hydraulic excavator.

[0027] Fig.13 This is a diagram for explaining a method of modifying a recognition area when acquiring a loading task.

[0028] Fig.14 This is a diagram for explaining a method of modifying a recognition area when acquiring an arrival mission.

[0029] Fig.15 This is a diagram showing a display example of a display device when the accuracy of a recognition result is equal to or less than a threshold value.

[0030] Fig.16 This is a block diagram illustrating the functional structure of the information processing device according to the second embodiment.

[0031] Fig.17 Yes Description Fig.16 FIG. 2 is a diagram showing the processing of the dump truck posture estimation unit.

[0032] Fig.18 This is a block diagram illustrating the functional structure of the information processing device according to the third embodiment. DETAILED DESCRIPTION

[0033] Hereinafter, embodiments of the present invention will be described using the drawings. In addition, regarding the configurations marked with the same reference numerals in each embodiment, unless otherwise specified, they have the same functions in each embodiment, and their description will be omitted.

[0034] The working machine of the present embodiment is a working machine that performs a loading operation, and the loading operation is to load the excavated soil and sand and other objects to a transporting machine as a loaded machine. Hereinafter, a hydraulic excavator with a bucket is used as an example of a working machine, and a dump truck is used as an example of a transporting machine. However, the working machine of the present embodiment may also be a hydraulic excavator with an attachment other than a bucket, or may be a working machine other than a hydraulic excavator. The transporting machine of the present embodiment may also be a transporting machine other than a dump truck.

[0035] [Implementation Method 1]

[0036] use Figures 1 to 15 , a hydraulic excavator 1 according to a first embodiment will be described.

[0037] <External structure of hydraulic excavator>

[0038] Figure 1 It is a side view schematically showing the external appearance structure of a hydraulic excavator 1 which is an example of a working machine according to the first embodiment.

[0039] The hydraulic excavator 1 includes a multi-articulated front working device 2 that holds an object and rotates in the vertical direction or the front-rear direction, and a machine body 3 on which the front working device 2 is mounted.

[0040] The machine body 3 includes a lower traveling body 5 that is driven by a right traveling hydraulic motor 4a and a left traveling hydraulic motor 4b provided on the right and left portions of the lower traveling body 5, and an upper rotating body 7 that is mounted on the upper portion of the lower traveling body 5 via a rotating device and is rotated by a rotating hydraulic motor 6 of the rotating device. In addition, in the present embodiment, the right traveling hydraulic motor 4a and the left traveling hydraulic motor 4b are also collectively referred to as "travel hydraulic motors 4a, 4b".

[0041] The front working device 2 is a multi-jointed working device composed of a plurality of front parts mounted on the front of the upper rotating body 7. The front working device 2 is mounted on the upper rotating body 7 and rotates. The front working device 2 includes a boom 8 connected to the front of the upper rotating body 7 so as to be rotatable in the up-down direction, an arm 9 connected to the front end of the boom 8 so as to be rotatable in the up-down direction, and a bucket 10 connected to the front end of the arm 9 so as to be rotatable in the up-down direction.

[0042] The boom 8 is connected to the upper rotating body 7 via the boom pin 8a, and rotates by the extension and contraction of the boom hydraulic cylinder 11. The arm 9 is connected to the front end of the boom 8 via the arm pin 9a, and rotates by the extension and contraction of the arm hydraulic cylinder 12. The bucket 10 is connected to the front end of the arm 9 via the bucket pin 10a and the bucket link 16, and rotates by the extension and contraction of the bucket hydraulic cylinder 13.

[0043] A boom angle sensor 14 is mounted on the boom pin 8a to detect the rotation angle of the boom 8. An arm angle sensor 15 is mounted on the arm pin 9a to detect the rotation angle of the arm 9. A bucket angle sensor 17 is mounted on the bucket link 16 to detect the rotation angle of the bucket 10.

[0044] In addition, the inertial measurement device may detect the angles of the boom 8, the arm 9, and the bucket 10 relative to a reference plane such as a horizontal plane, and convert them into rotation angles, thereby obtaining the rotation angles of the boom 8, the arm 9, and the bucket 10. In addition, the stroke sensor may detect the strokes of the boom hydraulic cylinder 11, the arm hydraulic cylinder 12, and the bucket hydraulic cylinder 13, and convert them into rotation angles, thereby obtaining the rotation angles of the boom 8, the arm 9, and the bucket 10.

[0045] An inclination angle sensor 18 for detecting the inclination angle of the machine body 3 relative to a reference plane such as a horizontal plane is installed on the upper rotating body 7. A rotation angle sensor 19 for detecting the relative angle, i.e., the rotation angle, of the upper rotating body 7 relative to the lower traveling body 5 is installed on the rotating device between the lower traveling body 5 and the upper rotating body 7. An angular velocity sensor (not shown) for detecting the angular velocity of the upper rotating body 7 is installed on the upper rotating body 7. In addition, in the present embodiment, the boom angle sensor 14, the arm angle sensor 15, the bucket angle sensor 17, the inclination angle sensor 18, and the rotation angle sensor 19 are collectively referred to as a "posture detection device 53". The posture detection device 53 detects the various rotation angles of the front working device 2 and the rotation angle of the upper rotating body 7, etc.

[0046] A display device 55 composed of a touch screen or the like capable of accepting input from an operator is provided in the cab provided in the upper rotating body 7. In addition, an operating device for operating a plurality of hydraulic actuators 4a, 4b, 6, 11, 12, and 13 is provided in the cab provided in the upper rotating body 7. Specifically, the operating device includes a right travel lever 23a for operating the right travel hydraulic motor 4a, a left travel lever 23b for operating the left travel hydraulic motor 4b, a right operating lever 22a for operating the boom hydraulic cylinder 11 and the bucket hydraulic cylinder 13, and a left operating lever 22b for operating the arm hydraulic cylinder 12 and the swing hydraulic motor 6. In addition, in the present embodiment, the right travel lever 23a, the left travel lever 23b, the right operating lever 22a, and the left operating lever 22b are collectively referred to as "operating levers 22, 23".

[0047] In addition, an external measuring device 70 for measuring the surrounding environment of the body of the hydraulic excavator 1 is installed on the upper rotating body 7. The external measuring device 70 measures the depth (distance to the object) of an object existing around the hydraulic excavator 1. The external measuring device 70 obtains depth information of the object as a measurement result. The external measuring device 70 may be, for example, a LiDAR (Light Detection And Ranging) or a stereo camera. A plurality of external measuring devices 70 may be installed on the hydraulic excavator 1.

[0048] In addition, a positioning device 60 for measuring the position and orientation of the hydraulic excavator 1 on site is installed on the upper rotating body 7. The positioning device 60 obtains positioning information including the position and orientation of the hydraulic excavator 1 on site as a measurement result. The positioning device 60 can be, for example, a measuring device such as a GNSS receiver and a TS (Total Station). The positioning device 60 is not limited to this, and can also be, for example, a camera fixed on site. In this case, the positioning device 60 can measure the position and orientation of the hydraulic excavator 1 on site based on information calculated from the detected image of the hydraulic excavator 1. In this case, in order to accurately measure the orientation of the hydraulic excavator 1, it is preferred that at least two positioning devices 60 are provided.

[0049] In addition, a dump truck information acquisition device 56 for acquiring dump truck information from the outside is installed on the upper rotating body 7, and the dump truck information includes the position, posture and vehicle parameter information of the dump truck 200. The vehicle parameter information includes information on the model and size of the dump truck 200. The dump truck information acquisition device 56 is, for example, a communication terminal that wirelessly communicates with a position and vehicle allocation management system of the dump truck 200 such as FMS (Fleet Management System). The dump truck information acquisition device 56 can acquire the dump truck information by receiving the dump truck information of the dump truck 200 sent from the position and vehicle allocation management system of the dump truck 200. The dump truck information acquisition device 56 can also be configured as software equipped with the functions of the dump truck information acquisition device 56 and incorporated into the information processing device 54.

[0050] Most of the position and vehicle allocation management systems of the dump truck 200 such as FMS have an adjustment function to make the position information of the dump truck 200 obtained from the GNSS receiver of the dump truck 200 consistent with the path on the map. The dump truck information acquisition device 56 can acquire information related to the position adjustment amount of the dump truck 200 generated based on the adjustment function from the position and vehicle allocation management system simultaneously with the dump truck information of the dump truck 200.

[0051] In addition, a task acquisition device 58 for acquiring a task representing the content of the action (or work content) to be performed subsequently by the hydraulic excavator 1 is installed on the upper rotating body 7. The task acquisition device 58 can be, for example, a communication terminal that wirelessly communicates with a management device that manages the work site. The task acquisition device 58 can acquire the task by receiving the task of the hydraulic excavator 1 sent from the management device. The task acquisition device 58 can be configured by a communication terminal shared with the dump truck information acquisition device 56. The task acquisition device 58 can also be configured as software with the function of the task acquisition device 58 installed, and incorporated into the information processing device 54. In addition, the task acquisition device 58 can also acquire the task by automatically distinguishing the task based on the operation of the operating levers 22 and 23 performed by the operator. The task acquisition device 58 is an input terminal that can be operated by the operator, and can acquire the task by accepting the input of the operator. In the present embodiment, there is no particular limitation on the method of acquiring the task.

[0052] <Internal structure of a hydraulic excavator>

[0053] Figure 2 Yes means Figure 1 1 is a block diagram showing the configuration of a hydraulic system and a control system mounted on the hydraulic excavator 1 shown.

[0054] The engine 103, which is a prime mover mounted on the upper rotating body 7, drives the hydraulic pump 102 and the pilot pump 104. The control device 40 controls the rotation of the front working device 2, the travel of the lower traveling body 5, and the rotation of the upper rotating body 7 (that is, the movement of the body of the hydraulic excavator 1 composed of these movements) based on the operation information (operation amount and operation direction) of the operating levers 22 and 23 generated by the operator. Specifically, the control device 40 detects the operation information (operation amount and operation direction) of the operating levers 22 and 23 generated by the operator through sensors 52a to 52f such as rotary encoders or potentiometers, and outputs control commands corresponding to the detected operation information to the electromagnetic proportional valves 47a to 47l. The electromagnetic proportional valves 47a to 47l are provided in the pilot line 100, and operate when the control command from the control device 40 is input, and output the pilot pressure to the flow control valve 101, so that the flow control valve 101 operates.

[0055] The flow control valve 101 controls the hydraulic oil supplied from the hydraulic pump 102 to the swing hydraulic motor 6, the arm hydraulic cylinder 12, the boom hydraulic cylinder 11, the bucket hydraulic cylinder 13, the right travel hydraulic motor 4a, and the left travel hydraulic motor 4b, respectively, based on the pilot pressure from the electromagnetic proportional valves 47a to 47l. In addition, the electromagnetic proportional valves 47a and 47b output the pilot pressure for controlling the hydraulic oil supplied to the swing hydraulic motor 6 to the flow control valve 101. The electromagnetic proportional valves 47c and 47d output the pilot pressure for controlling the hydraulic oil supplied to the arm hydraulic cylinder 12 to the flow control valve 101. The electromagnetic proportional valves 47e and 47f output the pilot pressure for controlling the hydraulic oil supplied to the boom hydraulic cylinder 11 to the flow control valve 101. The electromagnetic proportional valves 47g and 47h output the pilot pressure for controlling the hydraulic oil supplied to the bucket hydraulic cylinder 13 to the flow control valve 101. The electromagnetic proportional valves 47i and 47j output a pilot pressure for controlling the hydraulic oil supplied to the right travel hydraulic motor 4a to the flow control valve 101. The electromagnetic proportional valves 47k and 471 output a pilot pressure for controlling the hydraulic oil supplied to the left travel hydraulic motor 4b to the flow control valve 101.

[0056] The boom hydraulic cylinder 11, the arm hydraulic cylinder 12, and the bucket hydraulic cylinder 13 are extended and retracted by the supplied hydraulic oil, respectively, so that the boom 8, the arm 9, and the bucket 10 are rotated. As a result, the position and posture of the bucket 10 are changed. The swing hydraulic motor 6 is rotated by the supplied hydraulic oil, so that the upper swing body 7 is rotated. The right travel hydraulic motor 4a and the left travel hydraulic motor 4b are rotated by the supplied hydraulic oil, so that the lower travel body 5 is driven.

[0057] In addition, the control device 40 can automatically (semi-automatically or fully automatically) control the movement of the body of the hydraulic excavator 1 (the turning movement of the front working device 2, the traveling movement of the lower traveling body 5, and the rotating movement of the upper rotating body 7) based on the recognition result of the information processing device 54. The information processing device 54 recognizes the dump truck 200 existing in the vicinity of the hydraulic excavator 1 based on the measurement result of the external measuring device 70. The information processing device 54 is composed of a computer in which a CPU (Central Processing Unit) 73, a RAM (Random Access Memory) 72, a ROM (Read Only Memory) 71, and an external I / F (Interface) 74 are connected to each other through a bus 75. The control device 40, the display device 55, the external measuring device 70, the positioning device 60, the posture detection device 53, the task acquisition device 58, the dump truck information acquisition device 56, and the storage device 57 (for example, a hard disk drive or a large-capacity flash memory, etc.) are connected to the external I / F 74.

[0058] <Overview of Hydraulic Excavator Operation>

[0059] Figure 3 Yes Description Figure 2 FIG. 1 is a diagram showing an example of the operation of the hydraulic excavator 1 .

[0060] The dump truck 200 is parked at a predetermined position where the hydraulic excavator 1 can load an object. The task acquisition device 58 of the hydraulic excavator 1 acquires a loading task of loading the excavated soil and sand or other objects into the dump truck 200 as a subsequent task. When the loading task is acquired, first, the external measurement device 70 of the hydraulic excavator 1 measures the cargo box of the dump truck 200. Then, the information processing device 54 of the hydraulic excavator 1 calculates the recognition area and the estimated area described later for the existence area 210 of the dump truck 200. The information processing device 54 of the hydraulic excavator 1 calculates the position of the bucket 10 when the bucket 10 loads the object into the cargo box of the dump truck 200 (hereinafter also referred to as the "loading position"). And, based on the calculated recognition area and estimated area of ​​the dump truck 200 and the loading position, the control device 40 of the hydraulic excavator 1 outputs a control instruction to move the bucket 10 to the loading position in a manner so that the front working device 2 does not collide with the dump truck 200. Thereby, the hydraulic excavator 1 can automatically and appropriately control the loading operation without colliding with the dump truck 200 .

[0061] In this embodiment, if Figure 3 As shown, it is assumed that the dump truck 200 is parked at a predetermined position where the hydraulic excavator 1 can load an object. Specifically, the dump truck 200 of the present embodiment manages the position of the dump truck 200 at the work site through a position and vehicle allocation management system of the dump truck 200 such as an FMS. However, there is no particular limitation on the control method of the dump truck 200 so that the dump truck 200 is parked at a predetermined position. For example, the dump truck 200 can be parked at a predetermined position by the operator of the dump truck 200. In addition, in the present embodiment, the operator of the hydraulic excavator 1 can visually confirm that the dump truck 200 is parked at the predetermined position, or the information processing device 54 can recognize it based on the measurement result of the external measurement device 70.

[0062] also, Figure 3 The external measuring device 70 is installed toward the left side of the hydraulic excavator 1, but may be installed toward the front of the hydraulic excavator 1. Furthermore, the hydraulic excavator 1 may further include the external measuring device 70 installed toward the right side or rear of the hydraulic excavator 1.

[0063] Figure 4 Yes means Figure 2 Flowchart showing an example of the operation of the hydraulic excavator 1 .

[0064] In step S111 , the task acquisition device 58 of the hydraulic excavator 1 acquires an excavation task of excavating the ground of an excavation target.

[0065] In step S112 , the control device 40 of the hydraulic excavator 1 controls the rotation operation of the front working device 2 according to the acquired excavation task to cause the hydraulic excavator 1 to perform the excavation operation.

[0066] In step S113 , the task acquisition device 58 of the hydraulic excavator 1 acquires a loading task of loading the excavated earth and sand or other objects onto the dump truck 200 .

[0067] In step S114 , the control device 40 of the hydraulic excavator 1 controls the rotation operation of the front working device 2 and the rotation operation of the upper rotating body 7 according to the acquired loading task to make the hydraulic excavator 1 perform the loading operation.

[0068] In step S115, the control device 40 of the hydraulic excavator 1 determines whether the loading onto the dump truck 200 is completed. The operator of the hydraulic excavator 1 may operate the specific operating levers 22 and 23 after visual confirmation, or input information to the display device 55, and notify the control device 40 that the loading onto the dump truck 200 is completed. The control device 40 may determine that the loading onto the dump truck 200 is completed by receiving the notification of the operation from the operator. Alternatively, the control device 40 may determine that the loading onto the dump truck 200 is completed based on the judgment result of the load judgment device that measures the load of the object loaded on the dump truck 200 and judges that the load is appropriate. If it is determined that the loading onto the dump truck 200 is completed, the control device 40 proceeds to step S118. If it is determined that the loading onto the dump truck 200 is not completed, the control device 40 proceeds to step S116.

[0069] In step S116, the task acquisition device 58 of the hydraulic excavator 1 acquires a reaching task for moving the front working device 2 to the ground of the next excavation object.

[0070] In step S117, the control device 40 of the hydraulic excavator 1 controls the swinging operation of the front working device 2, the traveling operation of the lower traveling structure 5, and the rotating operation of the upper rotating structure 7 according to the acquired arrival task to make the hydraulic excavator 1 perform the arrival operation. Then, the control device 40 proceeds to step S111.

[0071] In step S118, the control device 40 of the hydraulic excavator 1 determines whether the operation is completed. The control device 40 may determine that the operation is completed by obtaining the operation completion notification through the task acquisition device 58. Alternatively, the control device 40 may determine that the operation is completed by the operator or the manager stopping the engine 103 of the hydraulic excavator 1. When it is determined that the operation is completed, the control device 40 interrupts the operation of the hydraulic excavator 1 and ends the operation. When it is determined that the operation is not completed, the control device 40 transfers to step S116. The hydraulic excavator 1 continues the arrival operation, the excavation operation, and the loading operation.

[0072] The tasks and actions of the hydraulic excavator 1 are not limited to Figure 4 The tasks and actions shown may also include, for example, moving or leveling, Figure 4 Tasks and actions not shown.

[0073] During the operation of the hydraulic excavator 1, when the task acquisition device 58 acquires each task, information such as the position, posture, and shape of the dump truck 200 may be required. In this case, the information processing device 54 identifies the dump truck 200 based on the measurement result of the external measurement device 70 each time the task acquisition device 58 acquires each task. In addition, the information processing device 54 verifies whether the recognition result of the dump truck 200 is appropriate, that is, whether the recognition result of the dump truck 200 is inappropriate to the extent that it cannot be modified or is appropriate to the extent that it can be modified (hereinafter referred to as "verification of appropriateness"). In addition, if the recognition result of the dump truck 200 is appropriate, the information processing device 54 modifies the recognition result.

[0074] In the present embodiment, the process of verifying the validity of the recognition result of the dump truck 200 and correcting the recognition result is also referred to as “recognition result verification process.” The function of the information processing device 54 that performs the recognition result verification process will be described below.

[0075] <Functional structure of information processing device>

[0076] Figure 5 Yes Description Figure 2 A block diagram of the functional structure of the information processing device 54 shown. Figure 6 This is a diagram showing each coordinate system 300 to 500 set on the hydraulic excavator 1 as seen from the side. Figure 7 Observed from above Figure 6 The coordinate systems shown are 300 to 500.

[0077] The information processing device 54 includes a posture calculation unit 81 , a coordinate conversion unit 82 , a positioning information calculation unit 83 , a task acquisition unit 80 , a recognition unit 84 , a region estimation unit 85 , a verification unit 86 , and a correction unit 87 .

[0078] In the information processing device 54, a reference coordinate system is preset as the reference coordinate system for determining the positions and postures of the components of the hydraulic excavator 1. Figure 6 as well as Figure 7 The body coordinate system 400 shown. The body coordinate system 400 of the hydraulic excavator 1 of the present embodiment is defined as a right-handed coordinate system with the point where the lower traveling body 5 contacts the ground G in the rotation center line 120 of the upper rotating body 7 as the origin. The body coordinate system 400 of the hydraulic excavator 1 defines the forward direction of the lower traveling body 5 as the positive direction of the X-axis. The body coordinate system 400 of the hydraulic excavator 1 defines the direction in which the rotation center line 120 extends upward as the positive direction of the Z-axis. The body coordinate system 400 of the hydraulic excavator 1 defines the direction that is orthogonal to the X-axis and the Z-axis and to the left as the positive direction of the Y-axis. In addition, in the body coordinate system 400 of the hydraulic excavator 1, the rotation angle θsw of the upper rotating body 7 defines the state in which the front working device 2 is parallel to the X-axis as 0 degrees.

[0079] In addition, in this embodiment, as the reference coordinate system of the external measurement device 70, it is defined as follows Figure 6 as well as Figure 7 The sensor coordinate system 300 is shown in FIG. In this embodiment, as the reference coordinate system of the scene, it is defined as follows: Figure 6 as well as Figure 7 A site coordinate system 500 is shown.

[0080] <Posture calculation unit>

[0081] The posture calculation unit 81 calculates the posture of the components of the hydraulic excavator 1 in the body coordinate system 400 of the hydraulic excavator 1 based on the detection signal of the posture detection device 53. Specifically, the posture calculation unit 81 calculates the rotation angle θbm of the boom 8 relative to the X-axis based on the detection signal of the rotation angle of the boom 8 output from the boom angle sensor 14. The posture calculation unit 81 calculates the rotation angle θam of the arm 9 relative to the boom 8 based on the detection signal of the rotation angle of the arm 9 output from the arm angle sensor 15. The posture calculation unit 81 calculates the rotation angle θbk of the bucket 10 relative to the arm 9 based on the detection signal of the rotation angle of the bucket 10 output from the bucket angle sensor 17. The posture calculation unit 81 calculates the rotation angle θsw of the upper rotating body 7 relative to the X-axis (lower traveling body 5) based on the detection signal of the rotation angle of the upper rotating body 7 output from the rotation angle sensor 19. In addition, the posture calculation unit 81 calculates the rotation angular velocity ωsw of the upper rotating body 7 based on the rotation angle θsw of the upper rotating body 7.

[0082] Furthermore, the posture calculation unit 81 calculates the inclination angle of the machine body 3 (lower traveling body 5) relative to the reference plane DP based on the detection signal of the inclination angle of the machine body 3 output from the inclination angle sensor 18. The reference plane DP is, for example, a horizontal plane perpendicular to the direction of gravity. The inclination angle includes a rotation angle θp around the Y axis and a rotation angle θr around the X axis.

[0083] <Coordinate conversion unit>

[0084] The coordinate conversion unit 82 uses the posture information of the hydraulic excavator 1 output from the posture calculation unit 81 to convert the coordinate system expressing the depth information acquired by the external measurement device 70 from the sensor coordinate system 300 to the body coordinate system 400 of the hydraulic excavator 1. The depth information acquired by the external measurement device 70 is given as a set of three-dimensional point data (that is, point cloud data) expressed by the sensor coordinate system 300.

[0085] In converting the point data Ps (Xs, Ys, Zs) in the sensor coordinate system 300 into the point data Pv (Xv, Yv, Zv) in the vehicle body coordinate system 400 of the hydraulic excavator 1 , for example, the following equations (1) to (3) are used.

[0086]

Formula 1

[0087] P v =R sv ·P s +T sv …(1)

[0088]

Formula 2

[0089]

[0090]

Formula 3

[0091] T sv =(Lsx,Lsy,Lsz)…(3)

[0092] In the above equations (1) to (3), Rsv is the rotation matrix from the sensor coordinate system 300 to the vehicle body coordinate system 400. αs, βs, and γs are the angles formed by the axes of the external measurement device 70 in the vehicle body coordinate system 400 (the axes of the sensor coordinate system 300). When the external measurement device 70 is fixed to the hydraulic excavator 1, these formed angles can be obtained by, for example, pre-measuring the posture of the external measurement device 70 in the vehicle body coordinate system 400 and pre-storing it in the storage device 57. In addition, when the external measurement device 70 performs measurement while changing its posture relative to the hydraulic excavator 1, the coordinate transformation matrix can be calculated using the angle detected by the posture measurement sensor, similarly to installing the posture measurement sensor on the external measurement device 70. θsw is the rotation angle of the upper rotating body 7, which is output from the posture calculation unit 81.

[0093] In the above equations (1) to (3), Tsv is a translation vector from the origin of the vehicle body coordinate system 400 to the origin of the sensor coordinate system 300. Lsx, Lsy, and Lsz are equal to the coordinates of the origin of the sensor coordinate system 300 observed from the vehicle body coordinate system 400. In most cases, the installation position of the external measurement device 70 is fixed relative to the hydraulic excavator 1. Therefore, in this case, the installation position of the external measurement device 70 to the hydraulic excavator 1 can be measured in advance and stored in the storage device 57 in advance.

[0094] <Positioning information calculation unit>

[0095] The positioning information calculation unit 83 calculates the position of the origin of the vehicle body coordinate system 400 in the site coordinate system 500 of the hydraulic excavator 1 and the orientation θdir of the front working device 2 in the site coordinate system 500 based on the positioning information acquired by the positioning device 60 .

[0096] <Mission Acquisition Department>

[0097] The task acquisition unit 80 acquires the task to be performed by the hydraulic excavator 1 from the task acquisition device 58. In the present embodiment, the tasks performed by the hydraulic excavator 1 include an excavation task, a loading task, and an arrival task. The excavation task is a task that specifies an excavation action, in which the hydraulic excavator 1 excavates the ground of the excavation object until the object such as soil and sand is held in the bucket 10. The loading task is a task that specifies a loading action, in which the bucket 10 is moved to the top of the cargo box of the dump truck 200 from the state in which the hydraulic excavator 1 has finished the excavation action and the object is unloaded from the bucket 10 to the cargo box. The arrival task is a task that specifies an arrival action, in which the bucket 10 is moved to the position of the ground of the next excavation object from the state in which the hydraulic excavator 1 has finished the loading action. In addition, the tasks performed by the hydraulic excavator 1 are not limited to these, and for example, they may also include a leveling task that specifies a leveling action in which the hydraulic excavator 1 levels the surrounding ground while the dump truck 200 is parked at a specified position on site, and a moving task that specifies a moving action when the hydraulic excavator 1 changes the ground of the excavation object.

[0098] <Identification Department>

[0099] The recognition unit 84 recognizes the dump truck 200 based on the measurement result of the external measurement device 70. Specifically, the recognition unit 84 uses the point cloud data obtained as the measurement result of the external measurement device 70 and converted into the vehicle body coordinate system 400 by the coordinate conversion unit 82 to recognize the position, posture and shape of the dump truck 200. In addition, the recognition unit 84 calculates the area where the recognized dump truck 200 exists, that is, the recognition area. In other words, the recognition area is the recognition result of the recognition unit 84 regarding the existence area 210 of the dump truck 200.

[0100] The method of calculating the recognition area of ​​the dump truck 200 by the recognition unit 84 is, for example, to store the three-dimensional mesh model of the dump truck 200 measured in advance in the storage device 57. And the position, posture and shape are compared between the point cloud data converted into the vehicle body coordinate system 400 obtained from the coordinate conversion unit 82 and the three-dimensional mesh model. Thus, the recognition unit 84 can calculate the position, posture and shape of the dump truck 200 as the object, and thus can calculate the recognition area of ​​the dump truck 200. In addition, the method of calculating the recognition area of ​​the dump truck 200 is not limited to this, and for example, the following method can be used to calculate the recognition area of ​​the dump truck 200 by extracting a specific plane of the dump truck 200 from the point cloud data obtained by the external measurement device 70. Alternatively, the following method can be used to construct a neural network (Neural Network) in advance to calculate the recognition area of ​​the dump truck 200 from the point cloud data including the dump truck 200, and use a recognizer that performs machine learning on the feature quantity of the dump truck 200 to be recognized to perform the calculation.

[0101] In this embodiment, the position, posture, and existence area 210 of the dump truck 200 are subsequently defined.

[0102] Figure 8 This is a diagram of a vehicle body coordinate system 600 of the dump truck 200 as viewed from the side. Fig. 9 Observed from above Figure 8 A diagram of a vehicle body coordinate system 600 is shown.

[0103] In this embodiment, the body coordinate system 600 of the dump truck 200 sets the center of the rear wheel axle of the dump truck 200 as the origin, the direction from the rear wheel to the front wheel as the X axis, the direction in which the rear wheel axle extends as the Y axis, and the height direction of the dump truck 200 as the Z axis. The position of the dump truck 200 refers to the origin Pd (Xd, Yd, Zd) of the body coordinate system 600. The posture of the dump truck 200 refers to the angle θd (θroll, θpitch, θyaw) formed by each axis of the field coordinate system 500 and each axis of the body coordinate system 600 of the dump truck 200. The existence area 210 of the dump truck 200 refers to the rectangular area Sd (Pd1~Pd4) with the points Pd1~Pd4 at the four corners of the cargo box of the dump truck 200 as vertices. In addition, the existence area 210 of the dump truck 200 is not limited to this, and for example, it can also be a hexahedron (such as a cuboid or a cube) including the entire dump truck 200. In the present embodiment, the recognition result of the recognition unit 84 on the existence area 210 of the dump truck 200 , that is, the recognition area of ​​the dump truck 200 is referred to as Sdr ( Pdr1 to Pdr4 ).

[0104] <Regional Estimation Department>

[0105] The area estimation unit 85 calculates an estimated area, which is an area estimated to be where the dump truck 200 exists, based on the position of the dump truck 200 and the vehicle parameter information (dump truck information) acquired by the dump truck information acquisition device 56. That is, the estimated area is the estimation result of the area estimation unit 85 regarding the existing area 210 of the dump truck 200. In order to verify the validity of the recognition result of the dump truck 200, the estimated area is calculated and compared with the recognition area of ​​the dump truck 200 calculated by the recognition unit 84.

[0106] The method of calculating the estimated area of ​​the dump truck 200 by the area estimating unit 85 is, for example, a method of calculating the estimated area of ​​the dump truck 200 based on the dump truck information, which is stored in the storage device 57 in advance for each vehicle model. Then, the estimated area of ​​the dump truck 200 in the site coordinate system 500 is calculated using the dump truck information acquired by the dump truck information acquiring device 56 and the calculation method corresponding to the dump truck information, and is converted into the vehicle body coordinate system 400 of the hydraulic excavator 1. Thus, the area estimating unit 85 can calculate the estimated area of ​​the dump truck 200 that can be compared with the identification area of ​​the dump truck 200 calculated by the identification unit 84. In addition, in the present embodiment, the estimation result of the area estimating unit 85 on the existing area 210 of the dump truck 200, that is, the estimated area of ​​the dump truck 200 is set as Sde (Pde1 to Pde4).

[0107] The conversion from the site coordinate system 500 to the body coordinate system 400 of the hydraulic excavator 1 can be performed using the following equations. In the conversion from the point data Pv (Xv, Yv, Zv) in the body coordinate system 400 of the hydraulic excavator 1 to the point data Pg (Xg, Yg, Zg) in the site coordinate system 500, for example, the following equations (4) to (6) are used.

[0108]

Formula 4

[0109]

[0110]

Formula 5

[0111]

[0112]

Formula 6

[0113] T vg =(x0,y0,z0)…(6)

[0114] In the above equations (4) to (6), Rvg is a rotation matrix from the vehicle body coordinate system 400 to the site coordinate system 500. θr, θp, θy are angles formed by the axes of the vehicle body coordinate system 400 in the site coordinate system 500. These formed angles can be calculated from the value calculated by the posture calculation unit 81, the rotation angle, the tilt angle of the machine body 3, and the orientation calculated by the positioning information calculation unit 83.

[0115] In the above equations (4) to (6), Tvg is a translation vector from the origin of the site coordinate system 500 to the origin of the vehicle body coordinate system 400. x0, y0, and z0 are equal to the coordinates of the origin of the vehicle body coordinate system 400 as viewed from the site coordinate system 500. Tvg can use the calculation result of the positioning information calculation unit 83.

[0116] In addition, the area estimation unit 85 can adjust the position of the dump truck 200 included in the dump truck information using the position adjustment amount of the dump truck 200 acquired simultaneously with the dump truck information by the dump truck information acquisition device 56. Furthermore, the area estimation unit 85 can calculate the estimated area of ​​the dump truck 200 based on the adjusted position of the dump truck 200.

[0117] <Verification Department>

[0118] The verification unit 86 verifies the validity of the recognition result of the dump truck 200 based on the recognition area calculated by the recognition unit 84 and the estimated area calculated by the area estimation unit 85. Specifically, the verification unit 86 calculates the accuracy of the recognition result of the dump truck 200 by comparing the recognition area with the estimated area, and verifies the validity of the recognition result of the dump truck 200 based on the calculated accuracy. In detail, the verification unit 86 calculates the overlap degree indicating the overlap degree between the recognition area and the estimated area as the accuracy of the recognition result of the dump truck 200. And, when the calculated overlap degree is higher than the threshold value, the verification unit 86 determines that the recognition result of the dump truck 200 is valid (can be modified). At this time, the verification unit 86 can change the judgment standard of the validity of the recognition result, that is, the threshold value of the overlap degree, according to the task acquired by the task acquisition unit 80.

[0119] <Revision Department>

[0120] The modification unit 87 modifies the recognition area calculated by the recognition unit 84 based on the estimated area calculated by the area estimation unit 85. Specifically, the modification unit 87 compares the recognition area calculated by the recognition unit 84 with the estimated area calculated by the area estimation unit 85, and modifies the recognition area calculated by the recognition unit 84. Thus, the modification unit 87 modifies the recognition result of the dump truck 200. At this time, the modification unit 87 can change the modification method of the recognition area (recognition result) calculated by the recognition unit 84 according to the task acquired by the task acquisition unit 80.

[0121] The information processing device 54 performs a recognition result verification process of verifying the validity of the recognition result of the dump truck 200 and correcting the recognition result using the verification unit 86 and the correction unit 87. The recognition result verification process will be described in detail below.

[0122] <Recognition result verification processing>

[0123] Fig.10 This is a flowchart showing an example of recognition result verification processing. Fig.11 It is a diagram for explaining the degree of overlap between the recognition area and the estimation area. Fig.12The table is a diagram showing a table that defines a criterion for determining the validity of a recognition result and a method for modifying a recognition area for each task of the hydraulic excavator 1 . Fig.13 This is a diagram for explaining a method of modifying a recognition area when acquiring a loading task. Fig.14 This is a diagram for explaining a method of modifying a recognition area when acquiring an arrival mission. Fig.15 This is a diagram showing a display example of the display device 55 when the accuracy of the recognition result is equal to or less than a threshold value.

[0124] In step S121 , the verification unit 86 of the information processing device 54 acquires the task acquired by the task acquisition unit 80 .

[0125] In step S122 , the verification unit 86 of the information processing device 54 acquires the recognition area calculated by the recognition unit 84 .

[0126] In step S123 , the verification unit 86 of the information processing device 54 acquires the estimated region calculated by the region estimating unit 85 .

[0127] In step S124, the verification unit 86 of the information processing device 54 calculates the accuracy of the recognition result of the dump truck 200. Fig.11 As shown, in this embodiment, the verification unit 86 calculates the overlap degree Acover indicating the degree of overlap between the recognition area calculated by the recognition unit 84 and the estimated area calculated by the area estimation unit 85 as the accuracy of the recognition result of the dump truck 200. The overlap degree Acover is given by the following formula (7).

[0128]

Formula 7

[0129] A cover =S cover / S de …(7)

[0130] In the above formula (7), Sde represents the area of ​​the estimated area calculated by the area estimation unit 85. Scover represents the area of ​​the region where the identification area (Sdr) calculated by the identification unit 84 overlaps with the estimated area (Sde) calculated by the area estimation unit 85. Scover can be reduced to a problem of finding the area of ​​the intersection of two convex polygons, for example, and can therefore be found as a numerical calculation problem.

[0131] In step S125, the verification unit 86 of the information processing device 54 determines whether the calculated overlap Acover is higher than the threshold value Ath. Thus, the verification unit 86 verifies the validity of the recognition result of the dump truck 200. When the calculated overlap Acover is higher than the threshold value Ath, the verification unit 86 determines that the recognition result of the dump truck 200 is valid (can be modified), and the process proceeds to step S126. When the calculated overlap Acover is lower than the threshold value Ath, the verification unit 86 determines that the recognition result of the dump truck 200 is invalid (cannot be modified), and the process proceeds to step S127.

[0132] Here, if Fig.12 As shown, the verification unit 86 can change the threshold value Ath of the overlap degree Acover, which is a criterion for determining the validity of the recognition result, according to the task acquired by the task acquisition unit 80 . Fig.12 The table shown is stored in advance in the storage device 57. Fig.12 As shown in column 1201 of the table shown, when the task acquired by the task acquisition unit 80 is a loading task, the verification unit 86 sets Ath1 as the threshold value Ath. When the task acquired by the task acquisition unit 80 is an arrival task, the verification unit 86 sets Ath2 as the threshold value Ath. Preferably, Ath1 and Ath2 are determined based on the calculation accuracy of each area of ​​the area estimation unit 85 and the recognition unit 84 during the function verification performed in advance, the presence or absence of an object in the bucket 10 during task acquisition, the distance between the front working device 2 and the dump truck 200, etc.

[0133] In the loading operation specified by the loading task, the hydraulic excavator 1 is started in a state where there is an object in the bucket 10 and the distance between the front working device 2 and the dump truck 200 is large. Therefore, it is considered that the damage when the front working device 2 collides with the dump truck 200 is greater in the loading operation than in the arrival operation. Therefore, it is preferable that the threshold value Ath1 set when the loading task is obtained is greater than or equal to the threshold value Ath2 set when the arrival task is obtained.

[0134] In step S126, the modification unit 87 of the information processing device 54 modifies the recognition result of the dump truck 200. Specifically, the modification unit 87 modifies the recognition area calculated by the recognition unit 84. Furthermore, the modification unit 87 outputs to the control device 40 a judgment result that the recognition result of the dump truck 200 is appropriate (can be modified), the accuracy (degree of overlap) of the recognition result, and the modified recognition area. Furthermore, the modification unit 87 may also display the information output to the control device 40 on the display device 55 to notify the operator, or send it to the management device that manages the work site to notify the manager. Then, the information processing device 54 Fig.10 The identification result verification process shown ends.

[0135] Here, if Fig.12 As shown, the modification unit 87 can change the modification method of the recognition area calculated by the recognition unit 84 according to the task acquired by the task acquisition unit 80. Fig.12 As shown in column 1202 of the table shown in FIG. 1 , when the task acquired by the task acquisition unit 80 is a loading task, the modification unit 87 modifies the recognition area calculated by the recognition unit 84 so that it becomes as follows: Fig.13 The region (Sdc) including the identification region (Sdr) and the estimated region (Sde) is shown. The region including the identification region (Sdr) and the estimated region (Sde) is the region (Sdc) including the OR region occupied by the identification region (Sdr) or the estimated region (Sde). Fig.13 In the example of , the correction unit 87 corrects the recognition region (Sdr) calculated by the recognition unit 84 to a rectangular region (Sdc) including the recognition region (Sdr) and the estimated region (Sde) and having the smallest area.

[0136] In the loading operation specified by the loading task, the hydraulic excavator 1 moves the bucket 10 from the ground outside the dump truck 200 to the upper part of the cargo box inside the dump truck 200 to the loading position. Therefore, if the recognition area calculated by the recognition unit 84 is modified to include the recognition area and the estimated area, the risk of the front working device 2 moving from the outside to the inside of the dump truck 200 colliding with the dump truck 200 during the loading operation is reduced. Therefore, when the task acquired by the task acquisition unit 80 is a loading task, the modification unit 87 modifies the recognition area calculated by the recognition unit 84 to include the recognition area and the estimated area.

[0137] The modification unit 87 can calculate the position of the modified recognition area in the height direction (Z-axis direction) as described later. That is, in order to avoid the collision between the front working device 2 and the dump truck 200, the modification unit 87 can use the maximum value of the Z-axis component of the coordinates of each vertex of the recognition area and the estimated area as the position of the modified recognition area in the height direction.

[0138] In addition, the correction unit 87 can calculate the corrected direction vdc of the recognition area using the direction vector vdr of the recognition area and the direction vector vde of the estimated area, for example, according to the following equations (8) to (9).

[0139]

Formula 8

[0140] v dc =V dr +G k (v de -v dr )…(8)

[0141]

Formula 9

[0142]

[0143] Δdr (Δxdr, Δydr, Δzdr) used in the calculation of the gain Gk in the above-mentioned equation (9) is information related to the calculation accuracy of the recognition area in the recognition unit 84. As Δdr, for example, information obtained by measuring the measurement accuracy of the external measurement device 70 and the calculation accuracy of the recognition unit 84 in advance can be used. In addition, when a recognizer using machine learning for the recognition algorithm of the recognition unit 84 is used, information on the recognition accuracy output by the recognizer can also be used as Δdr.

[0144] Δde (Δxde, Δyde, Δzde) used in the calculation of the gain Gk in the above-mentioned equation (9) is information related to the calculation accuracy of the estimated area in the area estimation unit 85. As Δde, the position adjustment amount of the dump truck 200 that can be obtained by the dump truck information acquisition device 56 together with the dump truck information can be used. That is, the correction unit 87 can correct the recognition area calculated by the recognition unit 84 using the position adjustment amount of the dump truck 200 obtained by the dump truck information acquisition device 56. When the dump truck information acquisition device 56 cannot obtain the position adjustment amount, the result of a preliminary investigation of the error of the positioning information obtained by the positioning device 60 at the site of the hydraulic excavator 1 can be used as Δde.

[0145] In addition, the modification unit 87 calculates the loading position Pdc (Xdc, Ydc, Zdc) of the bucket 10. Specifically, the modification unit 87 calculates, for example, the center position Pdr0 of the tank in the identification area calculated by the recognition unit 84 and the center position Pde0 of the tank in the estimated area calculated by the area estimation unit 85. The modification unit 87 can perform calculations according to the following formula (10), for example, using the center position Pdr0 of the tank in the identification area and the center position Pde0 of the tank in the estimated area.

[0146]

Formula 10

[0147] P dc =P dr +G k (P de -P dr )…(10)

[0148] In the above formula (10), the gain Gk is given by the above formula (9) including Δde. That is, the correction unit 87 can calculate the loading position of the bucket 10 based on the position adjustment amount of the dump truck 200 acquired by the dump truck information acquisition device 56, the identification area, and the estimated area. The correction unit 87 outputs the calculated loading position of the bucket 10 to the control device 40.

[0149] On the other hand, when the task acquired by the task acquisition unit 80 is an arrival task, the modification unit 87 modifies the recognition area (Sdr) calculated by the recognition unit 84 so that it becomes as follows: Fig.14 The area (Sdc) included in the area where the identified area (Sdr) overlaps with the estimated area (Sde) is shown. The area (Sdc) included in the area where the identified area (Sdr) overlaps with the estimated area (Sde) is shown. c ) is an area (Sdc) included in the AND area occupied by both the identification area (Sdr) and the estimated area (Sde). Fig.14 In the example of , the correction unit 87 corrects the recognition region (Sdr) calculated by the recognition unit 84 to a rectangular region (Sdc) including the recognition region (Sdr) and the estimated region (Sde) and having the largest area.

[0150] In the arrival action specified by the arrival task, the hydraulic excavator 1 moves the bucket 10 from the loading position above the cargo box inside the dump truck 200 to the ground outside the dump truck 200. Therefore, if the recognition area calculated by the recognition unit 84 is modified to be included in the area where the recognition area overlaps with the estimated area, the risk of the front working device 2 moving from the inside to the outside of the dump truck 200 colliding with the dump truck 200 during the arrival action is reduced. Therefore, when the task acquired by the task acquisition unit 80 is the arrival task, the modification unit 87 modifies the recognition area calculated by the recognition unit 84 so that it becomes an area included in the area where the recognition area overlaps with the estimated area.

[0151] In addition, the modification unit 87 can use the maximum value of the Z-axis component of the coordinates of each vertex of the recognition area and the estimated area as the position of the modified recognition area in the height direction (Z-axis direction) in the same manner as when the loading task is obtained. In addition, the modification unit 87 can calculate the orientation of the modified recognition area according to the above-mentioned formulas (8) to (9) in the same manner as when the loading task is obtained.

[0152] In step S127, the verification unit 86 of the information processing device 54 outputs the judgment result that the recognition result of the dump truck 200 is inappropriate (cannot be modified), the accuracy (coincidence) of the recognition result, the recognition area, and the estimated area to the control device 40. In addition, the verification unit 86 may display the information output to the control device 40 on the display device 55 to notify the operator, or send it to the management device that manages the work site to notify the manager. At this time, the verification unit 86 confirms with the user such as the operator or the manager whether to continue the operation of the hydraulic excavator 1.

[0153] For example, the verification unit 86 may Fig.15 As shown, a screen 551 is displayed on the display device 55, which notifies the user of the judgment result that the recognition result of the dump truck 200 is not appropriate (cannot be modified) and confirms whether the operation of the hydraulic excavator 1 continues. The screen 551 includes a message 552 for notifying the user of the judgment result that the recognition result of the dump truck 200 is not appropriate (cannot be modified) and confirming whether the operation of the hydraulic excavator 1 continues. The screen 551 includes a button 553 for the user to input the confirmation result of continuing the operation of the hydraulic excavator 1. The screen 551 includes a button 554 for the user to input the confirmation result of not continuing the operation of the hydraulic excavator 1. The screen 551 includes a button 555 for the user to input the switching of the hydraulic excavator 1 from automatic control to manual operation of the operator. In addition, the screen 551 shows the calculation results of the recognition area (Sdr) and the estimated area (Sde) of the dump truck 200, and notifies the user of the positional relationship between the dump truck 200 and the hydraulic excavator 1.

[0154] After receiving the confirmation result of whether to continue from the user, the verification unit 86 outputs the confirmation result to the control device 40. Then, the information processing device 54 Fig.10 The identification result verification process shown ends.

[0155] <Motion control of control device>

[0156] The control device 40 controls the movement of the body of the hydraulic excavator 1 (for example, the rotation movement of the front working device 2, the travel movement of the lower traveling body 5, and the rotation movement of the upper rotating body 7) based on the result of verifying the validity of the recognition result of the dump truck 200 obtained by the information processing device 54.

[0157] When the recognition result is appropriate (can be modified), the control device 40 controls the action of the hydraulic excavator 1 based on the task acquired by the task acquisition device 58 and the modified recognition area output from the information processing device 54. Specifically, when the control device 40 acquires the loading task, the control device 40 controls the loading action of the hydraulic excavator 1 so that the bucket 10 moves along a trajectory that does not cause the front working device 2 to collide with the dump truck 200 from the ground until it reaches the loading position. When the control device 40 acquires the arrival task, the control device 40 controls the arrival action of the hydraulic excavator 1 so that the bucket 10 moves along a trajectory that does not cause the front working device 2 to collide with the dump truck 200 from the loading position until it reaches the ground of the next excavation object.

[0158] Furthermore, when the recognition result is not valid (cannot be corrected), the control device 40 controls the operation of the hydraulic excavator 1 based on the confirmation result after the information processing device 54 outputs the confirmation result of whether the operation of the hydraulic excavator 1 is to continue.

[0159] <Effects>

[0160] As described above, the hydraulic excavator 1 of the first embodiment is a working machine that loads an object onto the dump truck 200. The hydraulic excavator 1 includes an external measuring device 70 that measures the surrounding environment of the body of the hydraulic excavator 1. The hydraulic excavator 1 includes an information processing device 54 that recognizes the dump truck 200 that exists around the body of the hydraulic excavator 1 based on the measurement result of the external measuring device 70. The hydraulic excavator 1 includes a control device 40 that controls the movement of the body of the hydraulic excavator 1 based on the recognition result of the information processing device 54. The hydraulic excavator 1 includes a dump truck information acquisition device 56 that acquires the position of the dump truck 200 and the vehicle parameter information from the outside. The information processing device 54 modifies the recognition result of the dump truck 200 based on the position of the dump truck 200 and the vehicle parameter information acquired by the dump truck information acquisition device 56. The control device 40 controls the movement of the body of the hydraulic excavator 1 based on the modified recognition result of the dump truck 200.

[0161] Thus, the hydraulic excavator 1 of the first embodiment can use the correct recognition result of the dump truck 200 in various situations to control the loading operation on the dump truck 200. Thus, according to the first embodiment, it is possible to provide a working machine that can correctly recognize the position and posture of a transporting machine in various situations, thereby appropriately controlling the loading operation on the transporting machine.

[0162] In the past, when the hydraulic excavator 1 performed a loading operation on the dump truck 200, it was difficult to verify the recognition result of the dump truck 200 based on the measurement result of the external measurement device 70. In particular, in the recognition system using machine learning, it is difficult to properly evaluate the uncertainty of the recognition result with respect to the site or the transport machine 200 for which the learning data has not been collected. In addition, when the developer of the hydraulic excavator 1 and the developer of the recognition system are different, there is a possibility that the details of the recognition system algorithm are not disclosed to the developer of the hydraulic excavator 1. It is difficult for the developer of the hydraulic excavator 1 to perform a sufficient risk assessment in advance to determine whether the recognition result of the recognition system can be used to properly control the loading operation without colliding with the dump truck 200.

[0163] In contrast, the hydraulic excavator 1 of the first embodiment can verify the validity of the recognition result of the dump truck 200 based on the information from the third party acquired by the dump truck information acquisition device 56, and modify the recognition result. Furthermore, the hydraulic excavator 1 of the first embodiment can appropriately control the loading operation, etc. based on the modified recognition result. That is, the hydraulic excavator 1 of the first embodiment can control the operation of the body of the hydraulic excavator 1 based on the correct recognition result of the position and posture of the dump truck 200. Therefore, the hydraulic excavator 1 of the first embodiment can reduce the risk of collision with the dump truck 200, control the loading operation and the arrival operation, and thus improve the safety and productivity of the loading operation. Therefore, according to the first embodiment, it is possible to provide a hydraulic excavator 1 that can correctly recognize the position and posture of the dump truck 200 in various situations, thereby appropriately controlling the loading operation to the dump truck 200.

[0164] Furthermore, the information processing device 54 includes an identification unit 84 that identifies the dump truck 200 based on the measurement result of the external measurement device 70 and calculates the identification area, which is the area where the identified dump truck 200 exists. The information processing device 54 includes an area estimation unit 85 that calculates the estimated area, which is the area estimated to be where the dump truck 200 exists, based on the position of the dump truck 200 and the vehicle parameter information acquired by the dump truck information acquisition device 56. The information processing device 54 includes a modification unit 87 that modifies the identification area based on the estimated area.

[0165] Thus, the hydraulic excavator 1 of the first embodiment can modify the recognition result based on a relatively simple index such as the existing area of ​​the dump truck 200. Therefore, the hydraulic excavator 1 of the first embodiment can correctly and easily recognize the position and posture of the dump truck 200. The hydraulic excavator 1 of the first embodiment can further improve the safety and productivity of the loading operation. Thus, according to the first embodiment, it is possible to provide a hydraulic excavator 1 that can correctly recognize the position and posture of the dump truck 200 in various situations, thereby being able to appropriately control the loading operation on the dump truck 200.

[0166] Then, the information processing device 54 (verification unit 86) compares the recognition area with the estimated area to calculate the accuracy of the recognition result of the dump truck 200. The correction unit 87 corrects the recognition area according to the calculated accuracy.

[0167] Thus, the hydraulic excavator 1 of the first embodiment can quantitatively verify the validity of the recognition result of the dump truck 200, and can modify the recognition area accurately and finely. The hydraulic excavator 1 of the first embodiment can further improve the safety and productivity of the loading operation. Thus, according to the first embodiment, it is possible to provide a hydraulic excavator 1 that can accurately and finely recognize the position and posture of the dump truck 200 in various situations, thereby being able to more appropriately control the loading operation on the dump truck 200.

[0168] The information processing device 54 (verification unit 86) calculates the degree of overlap indicating the degree of overlap between the recognition area and the estimated area as the accuracy of the recognition result. The correction unit 87 corrects the recognition area based on the calculated degree of overlap.

[0169] Thus, the hydraulic excavator 1 of the first embodiment can calculate the accuracy of the recognition result according to a clear reference, and can modify the recognition area more accurately and finely. The hydraulic excavator 1 of the first embodiment can further improve the safety and productivity of the loading operation. Thus, according to the first embodiment, it is possible to provide a hydraulic excavator 1 that can accurately and finely recognize the position and posture of the dump truck 200 in various situations, thereby being able to more appropriately control the loading operation on the dump truck 200.

[0170] Furthermore, the dump truck information acquisition device 56 also acquires information on the position adjustment amount of the dump truck 200. The region estimation unit 85 calculates the estimated region using the position adjustment amount acquired by the dump truck information acquisition device 56.

[0171] Thus, the hydraulic excavator 1 of the first embodiment can calculate the estimated area using a value that conforms to reality more than the error of the position information of the dump truck 200 obtained by prior verification or the like. Therefore, the hydraulic excavator 1 of the first embodiment can modify the recognition area more accurately. The hydraulic excavator 1 of the first embodiment can further improve the safety and productivity of the loading operation. Thus, according to the first embodiment, it is possible to provide a hydraulic excavator 1 that can more accurately recognize the position and posture of the dump truck 200 in various situations, thereby being able to more appropriately control the loading operation on the dump truck 200.

[0172] Furthermore, the information processing device 54 further includes a task acquisition unit 80 that acquires a task to be subsequently performed by the hydraulic excavator 1. The modification unit 87 changes the modification method of the recognition area according to the acquired task.

[0173] Thus, the hydraulic excavator 1 of the first embodiment can be modified to the optimal recognition area for each acquired task, so that the hydraulic excavator 1 can perform the optimal action for each task. The hydraulic excavator 1 of the first embodiment can further improve the safety and productivity of the loading operation. Thus, according to the first embodiment, it is possible to provide a hydraulic excavator 1 that can more accurately recognize the position and posture of the dump truck 200 in various situations, thereby being able to more appropriately control the loading operation on the dump truck 200.

[0174] Furthermore, when acquiring a loading task of loading an object on the dump truck 200 , the correction unit 87 corrects the recognition area so as to include the recognition area and the estimated area.

[0175] Thus, in the hydraulic excavator 1 of the first embodiment, the risk of the front working device 2 moving from the outside to the inside of the dump truck 200 colliding with the dump truck 200 during the loading operation can be further reduced. The hydraulic excavator 1 of the first embodiment can further improve the safety and productivity of the loading operation. Thus, according to the first embodiment, it is possible to provide a hydraulic excavator 1 that can more accurately recognize the position and posture of the dump truck 200 in various situations, thereby being able to more appropriately control the loading operation on the dump truck 200.

[0176] Furthermore, the hydraulic excavator 1 further includes a bucket 10 for holding an object. The dump truck information acquisition device 56 also acquires a position adjustment amount of the dump truck 200. When acquiring a loading task of loading an object onto the dump truck 200, the modification unit 87 calculates the position of the bucket 10 when the object is loaded from the bucket 10 onto the dump truck 200 based on the position adjustment amount acquired by the dump truck information acquisition device 56, the identification area, and the estimated area. The control device 40 controls the loading operation of the hydraulic excavator 1 based on the position of the bucket 10 calculated by the modification unit 87.

[0177] Thus, the hydraulic excavator 1 of the first embodiment can set the control target value of the control device 40 such as the loading position of the bucket 10 to a more appropriate value. Therefore, the hydraulic excavator 1 of the first embodiment can further improve the safety and productivity of the loading operation. Thus, according to the first embodiment, it is possible to provide a hydraulic excavator 1 that can more accurately recognize the position and posture of the dump truck 200 in various situations, thereby being able to more appropriately control the loading operation on the dump truck 200.

[0178] Furthermore, when acquiring an arrival task performed after the completion of the loading task of loading the object on the dump truck 200 , the correction unit 87 corrects the recognition area so that it becomes an area included in the area where the recognition area and the estimated area overlap.

[0179] Thus, the hydraulic excavator 1 of the first embodiment can further reduce the risk of the front working device 2 moving from the inside to the outside of the dump truck 200 colliding with the dump truck 200 during the arrival operation. Therefore, the hydraulic excavator 1 of the first embodiment can further improve the safety and productivity of the loading operation. Thus, according to the first embodiment, it is possible to provide a hydraulic excavator 1 that can more accurately recognize the position and posture of the dump truck 200 in various situations, thereby being able to more appropriately control the loading operation on the dump truck 200.

[0180] Furthermore, when the information processing device 54 (verification unit 86) determines that the recognition result of the dump truck 200 cannot be modified, the display device 55 displays a screen 551, which notifies the user of the determination result that the recognition result cannot be modified, and confirms with the user whether the operation of the body of the hydraulic excavator 1 continues. After receiving the confirmation result of whether to continue from the user, the information processing device 54 (verification unit 86) outputs the confirmation result to the control device 40. The control device 40 controls the operation of the body of the hydraulic excavator 1 based on the confirmation result.

[0181] The hydraulic excavator 1 may determine that the recognition result of the dump truck 200 cannot be corrected due to the influence of the on-site environment such as dust existing in the surroundings, abnormality of the sensor, etc. Even in this case, the hydraulic excavator 1 of the first embodiment can continue the operation of the hydraulic excavator 1 under appropriate judgment by following the instructions of the operator or the manager. Therefore, according to the first embodiment, it is possible to provide the hydraulic excavator 1 which can appropriately control the loading operation on the dump truck 200 even when the recognition result of the dump truck 200 cannot be corrected.

[0182] [Implementation Method 2]

[0183] use Fig.16 as well as Fig.17 , a working machine according to Embodiment 2 will be described. In the working machine according to Embodiment 2, description of the same configuration and operation as those of Embodiment 1 will be omitted.

[0184] Fig.16 This is a block diagram illustrating the functional structure of the information processing device 54 according to the second embodiment. Fig.17 Yes Description Fig.16 FIG. 1 is a diagram showing the processing of the dump truck posture estimating unit 88.

[0185] The dump truck information acquisition device 56 of the first embodiment acquires dump truck information including the position, posture, and vehicle parameter information of the dump truck 200. However, although the two-dimensional position and orientation of the dump truck 200 on the map are managed by the position and vehicle allocation management system of the dump truck 200, the posture of the dump truck 200 is not sometimes managed.

[0186] Therefore, the hydraulic excavator 1 of the second embodiment estimates the posture of the dump truck 200 from the terrain information around the hydraulic excavator 1. Specifically, Fig.16 As shown, the hydraulic excavator 1 of the second embodiment includes a terrain information acquisition device 59 for acquiring terrain information of the vicinity of the hydraulic excavator 1. The information processing device 54 of the second embodiment includes a dump truck posture estimation unit 88 for estimating the posture of the dump truck 200 based on the terrain information acquired by the terrain information acquisition device 59. The area estimation unit 85 of the second embodiment calculates an estimated area using the posture of the dump truck 200 estimated by the dump truck posture estimation unit 88.

[0187] The terrain information acquisition device 59 can be installed on the upper rotating body 7 of the hydraulic excavator 1. The terrain information acquisition device 59 can, for example, share the external measurement device 70 and the sensor to measure the terrain around the hydraulic excavator 1, and acquire the terrain information from the measurement result in the information processing device 54. Alternatively, the terrain information acquisition device 59 can also have a wireless communication function, receive the terrain information sent from the external system that measures the on-site terrain, and thereby acquire the terrain information. In this embodiment, the method for acquiring the terrain information is not particularly limited.

[0188] The topographic information in this embodiment is provided as point cloud data, but the topographic information is not limited to point cloud data, and may be, for example, data in a grid format that divides the vicinity of the hydraulic excavator 1 into grids and stores height information for each grid.

[0189] The dump truck posture estimation unit 88 acquires the position, orientation, and vehicle parameter information of the dump truck 200 included in the dump truck information acquired by the dump truck information acquisition device 56, and the terrain information acquired by the terrain information acquisition device 59. The dump truck posture estimation unit 88 extracts the terrain information in the existence area 210 of the dump truck 200 based on the acquired information. The dump truck posture estimation unit 88 performs plane estimation of the terrain in the existence area 210 from the extracted terrain information. As the plane estimation, for example, the least squares approximation method can be used. In addition, the dump truck posture estimation unit 88 calculates the inclination angle of the plane obtained by the plane estimation, thereby estimating the posture of the dump truck 200. Specifically, the dump truck posture estimation unit 88 calculates the angles θroll and θpitch representing the posture of the dump truck 200 based on the normal vector nt of the plane obtained by the plane estimation and the angles formed by the Y axis and the Z axis of the site coordinate system 500.

[0190] The area estimation unit 85 of the second embodiment calculates the estimated area of ​​the dump truck 200 based on the position, orientation and vehicle parameter information of the dump truck 200 acquired by the dump truck information acquisition device 56 and the posture of the dump truck 200 estimated by the dump truck posture estimation unit 88. The calculation method of the estimated area is the same as that of the first embodiment.

[0191] As described above, the hydraulic excavator 1 of the second embodiment further includes the terrain information acquisition device 59. The information processing device 54 of the second embodiment includes the dump truck posture estimation unit 88 for estimating the posture of the dump truck 200 based on the terrain information acquired by the terrain information acquisition device 59. The area estimation unit 85 of the second embodiment calculates the estimated area using the posture of the dump truck 200 estimated by the dump truck posture estimation unit 88.

[0192] Thus, the hydraulic excavator 1 of the second embodiment can correctly modify the recognition area even when the position and vehicle allocation management system of the dump truck 200 does not manage the posture of the dump truck 200. Thus, according to the second embodiment, it is possible to provide a hydraulic excavator 1 that can more accurately recognize the position and posture of the dump truck 200 in various situations, thereby being able to more appropriately control the loading operation on the dump truck 200.

[0193] [Implementation method 3]

[0194] use Fig.18 , a hydraulic excavator 1 according to a third embodiment will be described. In the hydraulic excavator 1 according to the third embodiment, description of the same configuration and operation as those of the first embodiment will be omitted.

[0195] Fig.18 This is a block diagram for explaining the functional structure of the information processing device 54 according to the third embodiment.

[0196] The information processing device 54 of the third embodiment recognizes the dump truck 200 using only the point cloud data of the dump truck 200 and its surroundings among the point cloud data acquired by the external measurement device 70 .

[0197] Specifically, the information processing device 54 of the third embodiment further includes a screening unit 89 that extracts point cloud data within a predetermined area including the estimated area calculated by the area estimating unit 85 from the point cloud data acquired by the external measurement device 70. The recognition unit 84 of the third embodiment recognizes the dump truck 200 from the point cloud data extracted by the screening unit 89, and calculates the recognition area of ​​the dump truck 200. The calculation method of the recognition area is the same as that of the first embodiment.

[0198] In the case where the point cloud data acquired by the external measurement device 70 includes data other than the dump truck 200 (other working machines and the ground), there is generally a possibility that the accuracy of the recognition processing will be reduced and the processing time will be prolonged. The hydraulic excavator 1 of embodiment 3 can use the estimated area calculated by the area estimation unit 85 to remove the point cloud data that becomes interference. As a result, the hydraulic excavator 1 of embodiment 3 can seek to improve the accuracy of the recognition processing and shorten the processing time. Therefore, according to embodiment 3, a hydraulic excavator 1 can be provided, which can more accurately and quickly identify the position and posture of the dump truck 200 in a variety of situations, thereby being able to more appropriately and quickly control the loading operation on the dump truck 200.

[0199] [other]

[0200] In addition, the present invention is not limited to the above-mentioned embodiments, and includes various modified examples. For example, the above-mentioned embodiments are described in detail in order to explain the present invention in an easy-to-understand manner, but are not limited to having all the described structures. In addition, a part of the structure of a certain embodiment can be replaced with the structure of other embodiments, and the structure of other embodiments can be added to the structure of a certain embodiment. In addition, for a part of the structure of each embodiment, other structures can be added, deleted, or replaced.

[0201] In addition, the above-mentioned various structures, functions, processing units, processing mechanisms, etc. can be implemented by hardware by partially or entirely designing them, for example, by integrated circuits. In addition, the above-mentioned various structures, functions, etc. can also be implemented by software by interpreting and executing programs that implement various functions by a processor. Information such as programs, data, and files that implement various functions can be placed in a memory, a recording device such as a hard disk, an SSD (solid state drive), or a recording medium such as an IC card, an SD card, or a DVD.

[0202] In addition, the control lines and information lines are shown as those considered necessary for explanation, and are not limited to showing all the control lines and information lines required for the product. In fact, it can be considered that almost all components are connected to each other.

[0203] Description of Reference Numerals

[0204] 1…hydraulic excavator (working machine), 54…information processing device, 55…display device, 56…dump truck information acquisition device (transporting machinery information acquisition device), 59…terrain information acquisition device, 70…external measuring device, 80…task acquisition unit, 84…identification unit, 85…area estimation unit, 86…verification unit, 87…modification unit, 88…dump truck posture estimation unit, 89…screening unit, 200…dump truck (transporting machinery), Sde…estimation area, Sdr…identification area.

Claims

1. A working machine for loading an object onto a transporting machine, wherein the working machine has: An external measuring device for measuring the surrounding environment of the vehicle body of the working machine; an information processing device for identifying the transport machine existing around the vehicle body based on the measurement result of the external measuring device; a control device for controlling the movement of the vehicle body based on the recognition result of the information processing device; and A handling machine information acquisition device, which acquires the position and vehicle parameter information of the handling machine from the outside, The information processing device modifies the identification result of the transport machine based on the position of the transport machine and the vehicle parameter information acquired by the transport machine information acquisition device, The control device controls the movement of the vehicle body based on the modified recognition result of the transport machine.

2. The working machine according to claim 1, characterized in that: The information processing device comprises: an identification unit that identifies the transport machine based on the measurement result of the external measurement device and calculates an area where the transport machine is located, that is, an identification area; an area estimating unit that calculates an estimated area, which is an area estimated to be where the transport machine exists, based on the position of the transport machine and the vehicle parameter information acquired by the transport machine information acquiring device; and A modification unit modifies the recognition area based on the estimated area.

3. The working machine according to claim 2, characterized in that: The information processing device calculates the accuracy of the recognition result by comparing the recognition area with the estimated area. The modification unit modifies the recognition area according to the calculated accuracy.

4. The working machine according to claim 3, characterized in that: the information processing device calculates, as the accuracy, a degree of overlap indicating a degree of overlap between the recognition area and the estimation area; The modification unit modifies the recognition area according to the calculated overlap degree.

5. The working machine according to claim 2, characterized in that: The transport machine information acquisition device further acquires information on a position adjustment amount of the transport machine, and the region estimation unit calculates the estimated region using the position adjustment amount acquired by the transport machine information acquisition device.

6. The working machine according to claim 2, characterized in that: The information processing device further includes a task acquisition unit that acquires a task to be performed subsequently by the working machine. The modification unit changes a modification method of the recognition area according to the acquired task.

7. The working machine according to claim 6, characterized in that: The modification unit modifies the recognition area so as to include the recognition area and the estimated area when a loading task of loading the object onto the transport machine is acquired.

8. The working machine according to claim 6, characterized in that: Also comprising a bucket for holding the object, The transporting machine information acquisition device further acquires information related to a position adjustment amount of the transporting machine. When the modification unit acquires a loading task of loading the object onto the transporting machine, the modification unit calculates the position of the bucket when the object is loaded from the bucket onto the transporting machine based on the position adjustment amount acquired by the transporting machine information acquisition device, the identification area, and the estimated area. The control device controls the loading operation of the object onto the transport machine based on the position of the bucket calculated by the correction unit.

9. The working machine according to claim 6, characterized in that: The modification unit modifies the recognition area so as to be included in an area where the recognition area overlaps with the estimated area, when acquiring an arrival task to be performed after completion of a loading task of loading the object on the transport machine.

10. The working machine according to claim 2, characterized in that: When the information processing device determines that the recognition result of the transport machine cannot be modified, the information processing device notifies the user of the determination result that the recognition result cannot be modified, and displays a screen on the display device for confirming to the user whether the movement of the vehicle body is to continue. After receiving the confirmation result of whether to continue from the user, the confirmation result is output to the control device. The control device controls the movement of the vehicle body according to the confirmation result.

11. The working machine according to claim 2, characterized in that: The apparatus further comprises a terrain information acquisition device for acquiring terrain information around the working machine. The information processing device further includes a posture estimating unit for estimating a posture of the transport machine based on the topographic information acquired by the topographic information acquiring device. The region estimating unit calculates the estimated region using the posture of the transport machine estimated by the posture estimating unit.

12. The working machine according to claim 2, characterized in that: The external measurement device obtains point cloud data of the surrounding environment as the measurement result, The information processing device further includes a screening unit for extracting the point cloud data within a predetermined area including the estimated area from the point cloud data acquired by the external measurement device. The recognition unit recognizes the transport machine based on the point cloud data extracted by the screening unit, and calculates the recognition area.

Citation Information

Patent Citations

  • Indoor autonomous unloading system and method for unmanned loader

    CN113309169A

  • System and method for detection of load and dump locations

    US20190180627A1

  • Stop-position determination device for transporting machine, and loading machine provided with same device

    WO2013191017A1

  • Remote operation system and operation assistance system

    WO2017042873A1

  • Position detection system

    WO2022065117A1