A back digging system and method for an excavator

Abstract

The application discloses a kind of excavator anti-lifting systems and methods, its method includes;Through angle encoder, the rotation angle information of each joint of excavator boom, stick, bucket is acquired;Through rotation angle information, determine the position of binocular camera in stick, and utilize binocular camera real-time acquisition depth information of excavating area;Depth image in depth information is handled, and whether excavator is visual lifting state is judged according to its processing result;Through controller, the control current of boom cylinder, stick cylinder and bucket cylinder is collected in real time, and whether current excavator is in working state is judged;According to current visual lifting state result and working state result, and in combination with the rotation angle information of each joint of excavator, whether excavator is in the state of being lifted is judged;According to the state of being lifted, control operation is carried out to excavator;The application can improve the safety of excavator operation by the lifting judgment of excavator.

Description

Technical Field

[0001] This invention relates to an excavator anti-jacking system and method, belonging to the field of excavator technology. Background Technology

[0002] Excavators are common construction machinery widely used in various earthmoving operations. During excavation, excavators frequently experience the problem of their bodies being lifted up. The main reason is that excavators cannot determine the internal structure of the material being excavated, making it impossible to identify large or hard objects. When the excavator bucket encounters such an object while descending, the excavator body is lifted. When this happens, it generally relies on the operator's quick thinking and immediate retraction, making it highly dependent on the operator. Even with the advancement and implementation of autonomous driving technology, excavators that are automatically driven can still experience this problem; however, the operator cannot control the lifted excavator. Furthermore, if the excavator's center of gravity deviates from a safe range when lifted, it may tip over, compromising the operator's safety. Summary of the Invention

[0003] The purpose of this invention is to overcome the shortcomings of the prior art and provide an excavator anti-jacking system and method. It can acquire depth information of the excavation process through a binocular camera, and use a regional processing method to obtain pHash from the depth image in the depth information. It can accurately acquire the regional changes during the material shoveling process, and combine the real-time angle and current of the excavator to determine whether the excavator is in a jacking state and feed it back to the excavator control in real time, thereby improving the safety and reliability of the excavator during operation.

[0004] To achieve the above objectives, the present invention is implemented using the following technical solution:

[0005] On one hand, the present invention provides a method for preventing excavators from being lifted out of their sockets, characterized by comprising:

[0006] An angle encoder is used to obtain the rotation angle information of each joint of the excavator's boom, stick, and bucket.

[0007] The position of the binocular camera on the boom is determined by the rotation angle information, and the depth information of the excavation area is obtained in real time by the binocular camera.

[0008] The depth image in the depth information is processed, and the excavator is determined to be in a visual lifting state based on the processing result.

[0009] The controller collects the control current of the boom cylinder, stick cylinder, and bucket cylinder in real time and determines whether the excavator is currently in operation.

[0010] Based on the current visual lifting status results and working status results, and combined with the rotation angle information of each joint of the excavator, it is determined whether the excavator is in a lifting state.

[0011] Based on whether the excavator is in a lifted state, control operations are performed on the excavator.

[0012] Optionally, the rotation angle information of each joint includes: the change in the upper arm angle Δδ corresponding to a unit time Δt. a Change in boom angle Δδ b and the change in bucket angle Δδ d .

[0013] Optionally, the method for determining the position of the binocular camera on the boom using the rotation angle information includes:

[0014] Based on the bucket rotation angle information, the bucket digging angle range is calibrated;

[0015] The maximum distance from the binocular camera to the tip of the bucket teeth is determined by the range of bucket digging angles.

[0016] The position of the binocular camera on the boom is determined based on the maximum distance, bucket size, and effective measurement range of the binocular camera.

[0017] Optionally, the depth image in the depth information is processed, including:

[0018] The depth image is filtered and smoothed using a mean filter, and a region of interest (ROI) is set to obtain a depth map of the ROI region.

[0019] The depth map of the ROI region is converted into a grayscale image, and the grayscale image is scaled.

[0020] The scaled grayscale image is converted into a frequency domain image, and the high-frequency region is extracted from it to generate a hash fingerprint image.

[0021] The hash value is calculated using the hash fingerprint.

[0022] Optionally, determining whether the excavator is in a visually lifted state based on the processing results of the depth image includes:

[0023] Calculate the hash value of the hash fingerprint map between the two frames and calculate the Hamming distance between the depth maps of the ROI region in the two frames to obtain the similarity of the depth maps of the ROI region.

[0024] If the similarity of the depth maps of the ROI region is high for 3-5 consecutive frames, then the excavator is in a visually lifted state.

[0025] Optionally, the ROI area is further divided into region 1, region 2 and region 3 according to the changes in materials during the excavation process; region 1 and region 2 are material separation areas on both sides of the bucket, and region 3 is the main material stacking area of ​​the bucket.

[0026] Optionally, when determining the similarity of the depth maps of ROI regions, the weight of region 3 in the ROI region is greater than that of region 1 and region 2.

[0027] Optionally, the method for determining whether the excavator is currently in a working state includes:

[0028] Determine whether the control current of the boom cylinder, stick cylinder, and bucket cylinder is greater than the normal digging start current.

[0029] The excavator is in operation when the control current of the boom cylinder, stick cylinder, and bucket cylinder is greater than the normal starting current for digging.

[0030] Optionally, the step of determining whether the excavator is in a lifted state based on the current visual lifting state result and working state result, combined with the rotation angle information of each joint of the excavator, includes:

[0031] When the excavator is in the visual jacking state and in working state, and the boom angle change Δδ a Change in boom angle Δδ b and the change in bucket angle Δδ d When all values ​​are greater than 0, the excavator is in a lifted state.

[0032] On the other hand, the present invention provides an excavator anti-lifting system, applied to the excavator anti-lifting method described in the first aspect, characterized in that it includes:

[0033] The data acquisition module is used to collect rotation angle information of each joint of the excavator boom, stick, and bucket, as well as depth information of the excavation area;

[0034] The data processing module is used to receive information collected by the data acquisition module and to make real-time judgments and processes.

[0035] The data processing module includes an image processing unit and a judgment unit; the image processing unit is used to process the depth image in the depth information; the judgment unit is used to determine whether the excavator is being lifted.

[0036] The control module is used to receive commands indicating whether the excavator is being lifted and to control the excavator accordingly.

[0037] Compared with the prior art, the beneficial effects achieved by the present invention are as follows:

[0038] This invention provides an excavator anti-jacking system and method, which can acquire depth information during the excavation process through a binocular camera, and obtain pHash by processing the depth image in the depth information in a regional manner. This can accurately obtain the regional changes during the material shoveling process, and, combined with the real-time angle and current of the excavator, determine whether the excavator is in a jacking state and feed back to the excavator control in real time, thereby improving the safety and reliability of the excavator during operation and reducing the excavator's dependence on the driver. Attached Figure Description

[0039] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the embodiments will be briefly described below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort, wherein:

[0040] Figure 1 The diagram shown is a flowchart of an excavator anti-jacking method according to an embodiment of the present invention;

[0041] Figure 2 The diagram shown is a schematic representation of the excavator bucket rotation angle according to an embodiment of the present invention.

[0042] Figure 3 The diagram shown is a schematic diagram of pHash processing of a depth image according to an embodiment of the present invention;

[0043] Figure 4 The diagram shown is a flowchart illustrating the process of determining whether the excavator is in a lifted state according to an embodiment of the present invention.

[0044] Figure 5 The figure shown is a block diagram of an excavator anti-jacking system according to an embodiment of the present invention. Detailed Implementation

[0045] The present invention will be further described below with reference to the accompanying drawings. The following embodiments are only used to more clearly illustrate the technical solution of the present invention, and should not be used to limit the scope of protection of the present invention.

[0046] In the description of this invention, it should be understood that the terms "center," "longitudinal," "lateral," "upper," "lower," "front," "rear," "left," "right," "vertical," "horizontal," "top," "bottom," "inner," and "outer," etc., indicating orientations or positional relationships based on the orientations or positional relationships shown in the accompanying drawings, are only for the convenience of describing the invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of the invention. Furthermore, the terms "first," "second," etc., are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, a feature defined with "first," "second," etc., may explicitly or implicitly include one or more of that feature. In the description of this invention, unless otherwise stated, "a plurality of" means two or more.

[0047] In the description of this invention, it should be noted that, unless otherwise explicitly specified and limited, the terms "installation," "connection," and "linking" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal connection of two components. Those skilled in the art will understand the specific meaning of the above terms in this invention based on the specific circumstances.

[0048] The application principle of the present invention will be described in detail below with reference to the accompanying drawings.

[0049] Example 1

[0050] like Figure 1 As shown, this embodiment of the invention provides a method for preventing an excavator from being lifted out of its shell, comprising the following steps:

[0051] Step 1: Obtain the rotation angle information of each joint of the excavator boom, stick, and bucket using an angle encoder;

[0052] Step 2: Determine the position of the binocular camera on the boom using the rotation angle information, and use the binocular camera to acquire the depth information of the excavation area in real time;

[0053] Step 3: Process the depth image in the depth information, and determine whether the excavator is in a visual lifting state based on the processing result;

[0054] Step 4: The controller collects the control current of the boom cylinder, stick cylinder and bucket cylinder in real time and determines whether the excavator is currently in operation.

[0055] Step 5: Based on the current visual lifting status results and working status results, and combined with the rotation angle information of each joint of the excavator, determine whether the excavator is in a lifting state.

[0056] Step 6: Determine whether the excavator is in a lifted state and then control the excavator accordingly.

[0057] In one specific embodiment of the present invention, obtaining the rotation angle information of each joint of the excavator boom, stick, and bucket using an angle encoder includes:

[0058] Angle encoders are installed at the joints of the excavator boom, stick, and bucket.

[0059] Accurate excavator vehicle model information can be obtained by creating an excavator URDF model file.

[0060] Based on the vehicle model information and the angle encoders installed on each joint of the excavator boom, stick, and bucket, the rotation angle information of each joint is obtained.

[0061] Specifically, the rotation angle information of each joint includes: the change in the upper arm angle Δδ corresponding to a unit time Δt. a Change in boom angle Δδ b and the change in bucket angle Δδ d .

[0062] In one specific embodiment of the present invention, the position of the binocular camera on the boom is determined by the rotation angle information, and the depth information of the excavation area is acquired in real time using the binocular camera, including:

[0063] The binocular camera is mounted on the stick of the excavator. The mounting position is determined by the maximum distance from the binocular camera to the tip of the bucket teeth and the effective measurement range of the binocular camera to ensure the accuracy of the data collected by the binocular camera.

[0064] Based on the bucket rotation angle information, the bucket digging angle range δ1~δ2 is calibrated;

[0065] Adjust the installation position of the binocular camera according to the bucket digging angle range δ1~δ2 and the size of the bucket so that the binocular camera can fully acquire depth information within the digging angle range.

[0066] Specifically, such as Figure 2 The diagram showing the excavator bucket rotation angle indicates that δ1 represents the excavator bucket in the hooked state, i.e., the minimum angle between the bucket and the stick; δ2 represents the excavator bucket in the anti-loose state, i.e., the maximum angle between the bucket and the stick; when the excavator bucket rotation angle is δ2, the maximum distance from the binocular camera to the tip of the bucket teeth can also be determined.

[0067] In one specific embodiment of the present invention, such as Figure 3 As shown, the depth image in the depth information is processed, and the excavator is determined to be in a visual lifting state based on the processing result, including:

[0068] Step 3.1: Use a mean filter to smooth the depth image and reduce the impact of noise;

[0069] Step 3.2: Set the ROI region on the filtered depth image to obtain the ROI region depth map;

[0070] The ROI area depth map is divided according to the different changes in material during the excavation process. This mainly includes material separation areas on both sides of the bucket: Area 1 and Area 2; and the main material storage area of ​​the bucket: Area 3. The area dimensions are adjusted in principle according to the size of the bucket and the coarseness of the material loaded. In this embodiment, the material loaded in the bucket is sand and gravel.

[0071] Step 3.3: Convert the depth map of the ROI region into a grayscale image, and scale the grayscale image; the specific scaling size is 32×32, which is used to extract image details.

[0072] The scaled grayscale image is converted into a frequency domain image, and the high-frequency region is extracted from it to generate a hash fingerprint image.

[0073] The hash value is calculated using the hash fingerprint.

[0074] Step 3.4: Calculate the hash value of the hash fingerprint map in the two frames before and after, and calculate the Hamming distance between the depth maps of the ROI region in the two frames to obtain the similarity of the depth maps of the ROI region; since the material changes in region 3 can better reflect the actual material state, the similarity of region 3 is given a higher weight.

[0075] If the similarity of the depth maps of the ROI region is high for 3-5 consecutive frames, the excavator is in a visual lifting state; in this embodiment, the binocular camera acquisition frequency is 30Hz.

[0076] In one specific embodiment of the present invention, determining whether the excavator is in a lifted state includes the following:

[0077] First, determine whether the excavator is in the visually lifted state (as described in detail in step 3);

[0078] Secondly, to determine whether the excavator is currently in operation, the specific method is as follows:

[0079] The controller collects the boom cylinder control current E at the same time. a , boom cylinder control current Eb and bucket control current E d ;

[0080] Determine the current control current E of the boom cylinder a , boom cylinder control current E b and bucket control current E d Whether all values ​​are greater than the threshold is used to determine whether the excavator is currently in operation; the threshold is the normal excavation start-up current, which can be set to zero in this embodiment.

[0081] Furthermore, based on the rotation angle information obtained in step 2, the current joint movement trend of the excavator is determined; specifically, when the boom angle changes by Δδ... a Change in boom angle Δδ b and the change in bucket angle Δδ d When all values ​​are greater than 0, the excavator joints are in motion;

[0082] Finally, when the excavator is simultaneously in visual jacking, working, and moving states, it is determined that the excavator is in a jacking state. Figure 4 As shown.

[0083] In one specific embodiment of the present invention, the control operation of the excavator is performed based on whether the excavator is in a lifted state. This includes adjusting the control current of the boom cylinder, stick cylinder and / or bucket cylinder when the excavator is in a lifted state, and retracting the boom, stick and / or bucket.

[0084] Example 2

[0085] like Figure 5 As shown, based on the excavator anti-lifting method described in Embodiment 1, this embodiment of the invention provides an excavator anti-lifting system, comprising:

[0086] The data acquisition module is used to collect rotation angle information of each joint of the excavator boom, stick, and bucket, as well as depth information of the excavation area;

[0087] The data processing module is used to receive information collected by the data acquisition module and to make real-time judgments and processes.

[0088] The data processing module includes an image processing unit and a judgment unit; the image processing unit is used to process the depth image in the depth information; the judgment unit is used to determine whether the excavator is being lifted.

[0089] The control module is used to receive commands indicating whether the excavator is being lifted and to control the excavator accordingly.

[0090] The database acquisition module includes angle encoders installed on the joints of the excavator boom, stick, and bucket, and a binocular camera installed at an appropriate position on the stick; the binocular camera faces the excavator bucket.

[0091] The embodiments of the present invention have been described above with reference to the accompanying drawings. However, the present invention is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of the present invention without departing from the spirit and scope of the claims. All of these forms are within the protection scope of the present invention.

Claims

1. A method of preventing a back dig of an excavator, characterized by, The application relates to a method for judging whether a shovel is in a visual lifting state, and a shovel control system. The method comprises the following steps: acquiring the rotation angle information of each joint of a shovel arm, a dipper arm and a bucket through an angle encoder; determining the position of a binocular camera on the dipper arm according to the rotation angle information, and acquiring the depth information of a digging area in real time by using the binocular camera; processing the depth image in the depth information, and judging whether the shovel is in a visual lifting state according to the processing result; collecting the control current of the arm cylinder, the dipper cylinder and the bucket cylinder in real time by using a controller, and judging whether the current shovel is in a working state; judging whether the shovel is in a lifted state according to the current visual lifting state result and the working state result, and combining the rotation angle information of each joint of the shovel; controlling the shovel according to the judgment result of whether the shovel is in a lifted state; processing the depth image in the depth information, which comprises the following steps: filtering and smoothing the depth image by using a mean filter, and setting an ROI area to obtain the ROI area depth map; converting the ROI area depth map into a gray image, and scaling the gray image; converting the scaled gray image into a frequency domain image, extracting the high-frequency part area from the frequency domain image, and generating a hash fingerprint image; calculating the hash value according to the hash fingerprint image; the judgment method of whether the shovel is in a visual lifting state according to the processing result of the depth image, which comprises the following steps: calculating the hash value of the hash fingerprint image of two adjacent frames, and calculating the Hamming distance of the ROI area depth map of the two frames to obtain the similarity of the ROI area depth map; 2. The anti-crowding method for excavators according to claim 1, characterized in that, The rotation angle information of each joint includes: unit time The corresponding boom angle change amount The corresponding arm angle change amount And the corresponding bucket angle change amount .

3. The anti-crowding method for excavators according to claim 1, characterized in that, if the similarity of the ROI area depth map of three to five continuous frames is high, the shovel is in a visual lifting state. the method for determining the position of the binocular camera on the dipper arm according to the rotation angle information, which comprises the following steps: calibrating the bucket digging angle range according to the bucket rotation angle information; determining the maximum distance from the binocular camera to the bucket tooth tip according to the bucket digging angle range; 4. The anti-crowding method for excavators according to claim 1, characterized in that, determining the position of the binocular camera on the dipper arm according to the maximum distance, the bucket size and the effective measurement range of the binocular camera.

5. The anti-crowding method for excavators according to claim 1, characterized in that, The ROI area is divided into region 1, region 2 and region 3 according to the change of the material in the digging process; the region 1 and the region 2 are the material segmentation zones on both sides of the bucket, and the region 3 is the main material stacking zone of the bucket.

6. The anti-crowding method for excavators according to claim 1, characterized in that, When judging the similarity of the ROI area depth map, the weight of the region 3 in the ROI area is greater than that of the region 1 and the region 2. The method for judging whether the current shovel is in a working state, which comprises the following steps: judging whether the control current of the arm cylinder, the dipper cylinder and the bucket cylinder is greater than the normal digging working starting current; 7. The anti-crowding method for excavators according to claim 2, characterized in that, when the control current of the arm cylinder, the dipper cylinder and the bucket cylinder is greater than the normal digging working starting current, the current shovel is in a working state. When the excavator is in a vision dozing state and in a working state, and the swing angle change amount , the arm angle change amount , and the bucket angle change amount are all greater than 0, the excavator is in a dozed state.

8. A system for preventing a back digging of an excavator, applied to the method for preventing a back digging of an excavator according to any one of claims 1 to 7, characterized in that, the method for judging whether the shovel is in a lifted state according to the current visual lifting state result and the working state result, and combining the rotation angle information of each joint of the shovel, which comprises the following steps: The application relates to a method for judging whether a shovel is in a visual lifting state, and a shovel control system. The method comprises the following steps: a data acquisition module is arranged for acquiring the rotation angle information of each joint of a shovel arm, a dipper arm and a bucket, and the depth information of a digging area; A data processing module is configured to receive information collected by the data collection module and to judge and process in real time. The data processing module comprises an image processing unit and a judging unit. The image processing unit is configured to process depth images in the depth information. The judging unit is configured to judge whether the excavator is being lifted. A control module is configured to receive a judgment command of whether the excavator is being lifted and to control the excavator.

Images