Method for controlling shoveling process of construction machine
By adaptively adjusting the bucket insertion depth using a depth camera and angle sensor, the problem of adjusting the bucket insertion depth during the loading process of construction machinery has been solved, achieving efficient material loading and energy optimization.
Patent Information
- Application Number
- CN202410813356.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-06-21
- Publication Date
- 2025-12-23
AI Technical Summary
During the automatic loading process of construction machinery, it is difficult to quickly adjust the insertion depth of the bucket to ensure that the bucket is close to full each time, resulting in low work efficiency.
A depth camera is used to acquire images and depth information of the bucket. By calibrating the material plane and the upper curved surface of the material in the bucket, and combining the boom and bucket swing arm angle sensors, the insertion depth of the bucket is adaptively adjusted to match the full bucket state.
It increases the loading rate of the bucket, improves the working efficiency of construction machinery, and reduces energy loss and costs.
Smart Images
Figure CN121183802A_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The present disclosure relates to the field of engineering machinery and electronic technology, and in particular, to a method for controlling a scooping process of an engineering machinery. BACKGROUND
[0002] With the development of sensor technology, the field of engineering machinery is also developing towards intelligence and automation, and the automatic scooping and intelligent scooping technology of engineering machinery (such as a loader) is also increasingly mature. In the process of automatic scooping of engineering machinery (such as a loader), it is necessary to ensure that the material loaded in the bucket during each scooping process is close to full bucket as much as possible, so as to improve the working efficiency of the loader as much as possible.
[0003] In actual operation, the amount of material loaded in the bucket during each scooping process is generally adjusted by controlling the insertion depth of the bucket to the material, so that the loaded material is close to full bucket. However, in actual operation, when the material to be scooped changes, it is difficult to quickly adjust the insertion depth of the bucket to the material to the right position.
[0004] Therefore, a method for quickly and adaptively adjusting the insertion depth of the bucket to the material is needed, so that the loaded material quickly approaches full bucket, thereby improving the working efficiency of the engineering machinery. SUMMARY
[0005] The starting point of the present disclosure is to provide a method for controlling a scooping process of an engineering machinery to solve the above-mentioned problems existing in the prior art.
[0006] Embodiments of the present disclosure provide a method for controlling a scooping process of an engineering machinery, wherein a depth camera is arranged on the engineering machinery, and the method comprises:
[0007] controlling the bucket of the engineering machinery to scoop material, and in the process of the bucket being in a bucket closing state and the boom of the engineering machinery being raised, when the shooting angle of the depth camera is parallel to the back plate of the bucket of the engineering machinery, obtaining a bucket image and depth information captured by the depth camera;
[0008] calibrating a bucket material flat plane in the bucket image, and judging whether there is material above the bucket material flat plane according to the depth information;
[0009] in the case that there is no material above the bucket material flat plane, gradually increasing the insertion depth of the bucket in each subsequent scooping process until there is material above the bucket material flat plane;
[0010] in the case that there is material above the bucket material flat plane, calibrating a material upper curved surface, and comparing the material upper curved surface with a set full bucket upper curved surface;
[0011] in the case that the material upper curve is lower than the bucket upper curve, increasing the bucket insertion depth step by step in each subsequent loading process until the height difference between the material upper curve and the bucket upper curve is less than a set height difference threshold; and
[0012] in the case that the material upper curve is equal to the bucket upper curve, decreasing the bucket insertion depth step by step in each subsequent loading process until the material upper curve is lower than the bucket upper curve and the height difference between the material upper curve and the bucket upper curve is less than the height difference threshold.
[0013] Optionally, the depth camera is mounted on a front frame of the engineering machine.
[0014] Optionally, the depth camera is an RGBD (Red Green Blue Depth) camera.
[0015] Optionally, in the case that there is material above the bucket material flat plane and the material upper curve is lower than the bucket upper curve, when the height difference between the material upper curve and the bucket upper curve is larger, the increase value of the bucket insertion depth is larger, and when the height difference between the material upper curve and the bucket upper curve is smaller, the increase value of the bucket insertion depth is smaller.
[0016] Optionally, the engineering machine is further provided with a boom angle sensor and a bucket swing arm angle sensor.
[0017] Optionally, the method further comprises:
[0018] when the shooting angle of the depth camera is parallel to the bucket back plate of the engineering machine, recording the angle position information of the boom acquired by the boom angle sensor and the angle position information of the bucket swing arm acquired by the bucket swing arm angle sensor, and when the boom and the bucket swing arm reach the same angle position again, driving the depth camera to capture the bucket image and the depth information.
[0019] Optionally, the bucket insertion depth is increased by increasing the lock pressure of the large cavity of the lifting oil cylinder when the bucket inserts into the material, and the bucket insertion depth is decreased by decreasing the lock pressure of the large cavity of the lifting oil cylinder when the bucket inserts into the material.
[0020] Optionally, the engineering machine comprises a loader.
[0021] The method for controlling the loading process of the engineering machine of the present disclosure has at least the following advantages:
[0022] In the present disclosure, whether there is material above the material flat plane of the bucket and the height relationship between the material upper curve and the full bucket upper curve is determined according to the bucket image and depth information captured by the depth camera, and the bucket insertion depth is adjusted in each subsequent loading process according to the determined material state and height relationship, so as to adjust the material loaded in the bucket to match the full bucket rate of the bucket.
[0023] In the present disclosure, in the case that there is no material above the material flat plane of the bucket, the bucket insertion depth is increased in each subsequent loading process until there is material above the material flat plane of the bucket, so as to improve the loading rate of the bucket and the working efficiency of the engineering machinery.
[0024] In the present disclosure, in the case that there is material above the material flat plane of the bucket and the material upper curve is lower than the full bucket upper curve, the bucket insertion depth is increased in each subsequent loading process until the height difference between the material upper curve and the full bucket upper curve is less than the set height difference threshold, so as to further improve the loading rate of the bucket and the working efficiency of the engineering machinery.
[0025] In the present disclosure, in the case that there is material above the material flat plane of the bucket and the material upper curve is equal to the full bucket upper curve, the bucket insertion depth is decreased in each subsequent loading process until the material upper curve is lower than the full bucket upper curve and the height difference between the material upper curve and the full bucket upper curve is less than the height difference threshold, so as to reduce the excess energy loss in the loading process and reduce the cost.
[0026] In the present disclosure, when the height difference between the material upper curve and the full bucket upper curve is large, the increase value of the bucket insertion depth is large, and when the height difference between the material upper curve and the full bucket upper curve is small, the increase value of the bucket insertion depth is small, so that the increase value of the bucket insertion depth is adapted to the actual state of the material.
[0027] In the present disclosure, when the shooting angle of the depth camera is parallel to the bucket back plate of the engineering machinery, the angle position information of the boom acquired by the boom angle sensor and the angle position information of the bucket swing arm acquired by the bucket swing arm angle sensor can be recorded, and when the boom and the bucket swing arm reach the same angle position next time, the depth camera is driven to capture the bucket image and depth information, so that it is not necessary to judge again whether the shooting angle of the depth camera is parallel to the bucket back plate of the engineering machinery, thereby reducing the amount of calculation and reducing the occupation of computing resources. BRIEF DESCRIPTION OF DRAWINGS
[0028] Other details and advantages of the present disclosure will become apparent from the detailed description provided below. It should be understood that the following drawings are merely schematic and not drawn to scale, and therefore should not be considered as limiting the present disclosure, which will be described in detail below with reference to the accompanying drawings, in which:
[0029] Figure 1 A method flowchart for controlling a scooping process of a construction machine according to one embodiment of the present disclosure is shown.
[0030] Figure 2 A construction machine according to one embodiment of the present disclosure is shown. DETAILED DESCRIPTION
[0031] Embodiments of the present disclosure are described below with reference to the accompanying drawings. In the following description, numerous specific details are set forth in order to provide a thorough understanding of the present disclosure. However, it will be apparent to one skilled in the art that the present disclosure can be practiced without some or all of these specific details. In other instances, well known process steps have not been described in detail in order not to unnecessarily obscure the present disclosure. In addition, it will be understood that the present disclosure is not limited to the particular embodiments described herein but can be practiced with any combination of the features and elements described below, whether directed to different embodiments or not. Accordingly, the following aspects, features, embodiments and advantages are merely illustrative and are not to be construed as limiting the scope of the claims unless expressly recited in such claims.
[0032] Reference is now made to Figure 1 , a method for controlling a scooping process of a construction machine according to a first embodiment of the present disclosure is shown. The method can be performed by any suitable processing device having data transceiving and data processing functions, such as an ECM (Electronic Control Module) of the construction machine, without departing from the scope of the present disclosure. Figure 2 A construction machine according to one embodiment of the present disclosure is shown, which includes a depth camera 1, a boom angle sensor 2 and a bucket swing arm angle sensor 3. As shown in Figure 1 , the method includes the following steps:
[0033] At step S101, a bucket of the construction machine is controlled to scoop material, and during the process that the bucket is in a bucket closing state and a boom of the construction machine is raised, a bucket image and depth information captured by the depth camera are obtained when a shooting angle of the depth camera is parallel to a back plate of the bucket of the construction machine.
[0034] It can be understood by those skilled in the art that the depth camera can be installed at any suitable position of the construction machine, without departing from the scope of the present disclosure. For example, the depth camera can be installed on a front frame of the construction machine.
[0035] Those skilled in the art can understand that any suitable type of depth camera can be adopted, and these variants do not exceed the protection scope of the present disclosure. For example, an RGBD camera can be adopted.
[0036] Step S102, calibrating the bucket material flat plane in the bucket image, and judging whether there is material above the bucket material flat plane according to the depth information.
[0037] Figure 2 The bucket material flat plane 4 and the bucket back plate 7 identified are also shown in FIG. 4. Specifically, the bucket material flat plane 4 can be calibrated by identifying the bucket back plate 7 in the bucket image. Those skilled in the art can understand that any suitable image recognition method and algorithm can be adopted to process the bucket image and the depth information, and these variants do not exceed the protection scope of the present disclosure.
[0038] Step S103, in the case that there is no material above the bucket material flat plane, increasing the bucket insertion depth step by step in each subsequent loading process until there is material above the bucket material flat plane.
[0039] Those skilled in the art can understand that any suitable method can be adopted to increase the bucket insertion depth step by step, and these variants do not exceed the protection scope of the present disclosure. For example, the locking pressure of the lifting cylinder large cavity when the bucket inserts into the material can be increased step by step, thereby increasing the bucket insertion depth step by step. The step-by-step increase value of the locking pressure of the lifting cylinder large cavity can be set according to the actual situation (for example, the shape and size of the bucket), for example, the step-by-step increase value of the locking pressure of the lifting cylinder large cavity can be linear or nonlinear, and these variants do not exceed the protection scope of the present disclosure.
[0040] Step S104, in the case that there is material above the bucket material flat plane, calibrating the material upper curved surface, and comparing the material upper curved surface with the set full-bucket upper curved surface.
[0041] Figure 2 The set full-bucket upper curved surface 6 and the height difference 5 between the bucket material flat plane 4 and the full-bucket upper curved surface 6 are also shown in FIG. 5. Those skilled in the art can understand that the specific position of the full-bucket upper curved surface can be set according to the actual situation (for example, the shape and size of the bucket), and these variants do not exceed the protection scope of the present disclosure.
[0042] Step S105, in the case that the material upper curved surface is lower than the full-bucket upper curved surface, increasing the bucket insertion depth step by step in each subsequent loading process until the height difference between the material upper curved surface and the full-bucket upper curved surface is less than the set height difference threshold.
[0043] Specifically, the increment of the bucket insertion depth can be set according to the size of the height difference between the material upper surface and the full bucket upper surface. In the case that there is material above the bucket material full surface and the material upper surface is lower than the full bucket upper surface, when the height difference between the material upper surface and the full bucket upper surface is larger, the increment of the bucket insertion depth can be larger, and when the height difference between the material upper surface and the full bucket upper surface is smaller, the increment of the bucket insertion depth can be smaller.
[0044] Those skilled in the art can understand that the bucket insertion depth can be incrementally increased in any suitable manner, and these variations do not exceed the protection scope of the present disclosure. For example, the locking pressure of the lifting cylinder large cavity when the bucket inserts the material can be incrementally increased, so as to incrementally increase the bucket insertion depth. For example, the increment of the locking pressure of the lifting cylinder large cavity can be set according to the size of the height difference between the material upper surface and the full bucket upper surface.
[0045] Those skilled in the art can understand that the specific value of the height difference threshold value can be set according to the actual situation (for example, the shape and size of the bucket), and these variations do not exceed the protection scope of the present disclosure. In the case that the height difference between the material upper surface and the full bucket upper surface is less than the set height difference threshold value, it can be considered that the material in the bucket has reached the full bucket state.
[0046] Step S106, in the case that the material upper surface and the full bucket upper surface are equal in height, the bucket insertion depth is incrementally reduced in each subsequent bucket loading process until the material upper surface is lower than the full bucket upper surface and the height difference between the material upper surface and the full bucket upper surface is less than the height difference threshold value.
[0047] Those skilled in the art can understand that the bucket insertion depth can be incrementally reduced in any suitable manner, and these variations do not exceed the protection scope of the present disclosure. For example, the locking pressure of the lifting cylinder large cavity when the bucket inserts the material can be incrementally reduced, so as to incrementally reduce the bucket insertion depth. The incrementally reduced value of the locking pressure of the lifting cylinder large cavity can be set according to the actual situation, and these variations do not exceed the protection scope of the present disclosure.
[0048] In the case that the material upper surface is lower than the full bucket upper surface and the height difference between the material upper surface and the full bucket upper surface is less than the height difference threshold value, it can be considered that the full bucket amount of material is just loaded into the bucket and there is no excess energy loss.
[0049] In addition, when the shooting angle of the depth camera is parallel to the back plate of the bucket of the engineering machine, the angle position of the boom acquired by the boom angle sensor and the angle position of the bucket swing arm acquired by the bucket swing arm angle sensor can be recorded, and when the boom and the bucket swing arm reach the same angle position next time, the depth camera is driven to capture the bucket image and the depth information without the need to judge again whether the shooting angle of the depth camera is parallel to the back plate of the bucket of the engineering machine.
[0050] It can be understood by those skilled in the art that the method for controlling the loading process of the engineering machine in the present disclosure can be used for any suitable engineering machine, such as a loader, and these variations do not exceed the protection scope of the present disclosure.
[0051] Industrial applicability
[0052] In the present disclosure, whether there is material above the bucket material flat plane and the height relationship between the material upper curve and the full bucket upper curve is determined according to the bucket image and the depth information captured by the depth camera, and the bucket insertion depth is adjusted in each subsequent loading process according to the determined material state and height relationship, so as to adjust the material loaded in the bucket to match the full bucket rate of the bucket.
[0053] In the present disclosure, in the case that there is no material above the bucket material flat plane, the bucket insertion depth is increased in each subsequent loading process until there is material above the bucket material flat plane, so as to improve the loading rate of the bucket and improve the working efficiency of the engineering machine.
[0054] In the present disclosure, in the case that there is material above the bucket material flat plane and the material upper curve is lower than the full bucket upper curve, the bucket insertion depth is increased in each subsequent loading process until the height difference between the material upper curve and the full bucket upper curve is less than the set height difference threshold, so as to further improve the loading rate of the bucket and improve the working efficiency of the engineering machine.
[0055] In the present disclosure, in the case that there is material above the bucket material flat plane and the material upper curve is equal to the full bucket upper curve, the bucket insertion depth is decreased in each subsequent loading process until the material upper curve is lower than the full bucket upper curve and the height difference between the material upper curve and the full bucket upper curve is less than the height difference threshold, so as to reduce the excess energy loss in the loading process and reduce the cost.
[0056] In the present disclosure, when the height difference between the material upper curve and the full bucket upper curve is large, the increase value of the bucket insertion depth is large, and when the height difference between the material upper curve and the full bucket upper curve is small, the increase value of the bucket insertion depth is small, so that the increase value of the bucket insertion depth is adapted to the actual state of the material.
[0057] In the present disclosure, when the shooting angle of the depth camera is parallel to the back plate of the bucket of the engineering machine, the angle position information of the boom acquired by the boom angle sensor and the angle position information of the bucket slewing arm acquired by the bucket slewing arm angle sensor can be recorded, and when the boom and the bucket slewing arm reach the same angle position next time, the depth camera is driven to capture the bucket image and the depth information, thereby, without the need to judge again whether the shooting angle of the depth camera is parallel to the back plate of the bucket of the engineering machine, the calculation amount can be reduced, and the occupation of the computing resources can be reduced.
[0058] Although the present disclosure has been disclosed in the preferred embodiments as above, the present disclosure is not limited to this. Any modifications and changes made by any person skilled in the art without departing from the spirit and scope of the present disclosure shall be included in the protection scope of the present disclosure, and therefore the protection scope of the present disclosure shall be subject to the scope defined by the claims.
Claims
1. A method for controlling the loading process of construction machinery, characterized in that, The engineering machinery is equipped with a depth camera, and the method includes: The bucket of the construction machinery is controlled to load materials. When the bucket is in the retracted state and the boom of the construction machinery is raised, the depth camera captures the bucket image and depth information when the shooting angle of the depth camera is parallel to the rear plate of the bucket of the construction machinery. The bucket material level plane is marked in the bucket image, and the presence of material above the bucket material level plane is determined based on the depth information. If there is no material above the material level of the bucket, the bucket insertion depth is gradually increased in each subsequent loading process until there is material above the material level of the bucket. When there is material above the flat surface of the bucket, mark the upper curved surface of the material and compare the upper curved surface of the material with the set upper curved surface of the full bucket. When the upper curved surface of the material is lower than the upper curved surface of the full bucket, the bucket insertion depth is gradually increased in each subsequent loading process until the height difference between the upper curved surface of the material and the upper curved surface of the full bucket is less than a set height difference threshold; and, When the height of the upper curved surface of the material is equal to that of the upper curved surface of the full bucket, the insertion depth of the bucket is gradually reduced in each subsequent loading process until the upper curved surface of the material is lower than the upper curved surface of the full bucket and the height difference between the upper curved surface of the material and the upper curved surface of the full bucket is less than the height difference threshold.
2. The method according to claim 1, wherein, The depth camera is mounted on the front frame of the engineering machinery.
3. The method according to claim 1, wherein, The depth camera is an RGBD camera.
4. The method according to claim 1, wherein, When there is material above the flat surface of the bucket and the upper curved surface of the material is lower than the upper curved surface of the full bucket, the increase in bucket insertion depth is greater when the height difference between the upper curved surface of the material and the upper curved surface of the full bucket is large, and the increase in bucket insertion depth is smaller when the height difference between the upper curved surface of the material and the upper curved surface of the full bucket is small.
5. The method according to claim 1, wherein, The construction machinery is also equipped with a boom angle sensor and a bucket swing arm angle sensor.
6. The method according to claim 5, wherein, The method further includes: When the shooting angle of the depth camera is parallel to the rear plate of the bucket of the construction machinery, the angle position of the boom obtained by the boom angle sensor and the angle position of the bucket swing arm obtained by the bucket swing arm angle sensor are recorded. When the boom and the bucket swing arm reach the same angle position again, the depth camera is driven to capture the bucket image and depth information.
7. The method according to claim 1, wherein, The insertion depth of the bucket is increased by increasing the locking pressure of the large chamber of the lifting cylinder when the bucket inserts into the material, and the insertion depth of the bucket is decreased by decreasing the locking pressure of the large chamber of the lifting cylinder when the bucket inserts into the material.
8. The method according to claim 1, wherein, The construction machinery includes loaders.