Material digging method based on image recognition

Abstract

This application provides a material handling method based on image recognition, belonging to the field of image recognition application technology. It is implemented using a material pool, an excavator, an image acquisition device, a LiDAR, and a control unit; the excavator includes a digging arm and an opening / closing bucket assembly. The method includes: the control unit constructing a global 3D map of the material pool by combining data from two types of sensors, dividing the area into grids to filter peak regions and planning the operation path; after the excavator moves to the target point, the digging mode is selected based on the distance between the peak and the excavator; if the distance is less than the effective width of the bucket, the first mode is used, otherwise the second mode is used; there is also a corresponding digging strategy when there are no peaks, and the material pile data is updated in real time, supporting remote image monitoring. This method improves the reliability of perception in complex environments and digging efficiency, protects equipment, and is adaptable to multiple scenarios.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image recognition application, and in particular to a material digging method based on image recognition. BACKGROUND

[0002] A material pool is a device for storing raw materials, which is generally used for storing materials such as grains and sand, and since it is set below the ground level, the digging operation is mostly completed by a excavator. During the storage process of the material pool, the unloading is generally completed by a truck, and during the unloading process, a material pile is formed in the material pool, and the material pile forms a wave crest. At present, the digging operation is mostly completed by manual operation of the excavator, but the manual operation needs to rely on the experience of the operator for judgment, and since the material pool has a certain depth, the operation is complex. In order to improve the degree of automation, some schemes attempt to use sensors to sense the material pile and automatically dig the material. However, the existing digging mode of the excavator is relatively single, and it is difficult to adapt to different digging strategies according to the different distances between the wave crest of the material pile and the excavator, resulting in low efficiency of the excavator. SUMMARY

[0003] The purpose of the present application is to solve the above problems and provide a material digging method based on image recognition.

[0004] The technical scheme of the present application is as follows:

[0005] The present application provides a material digging method based on image recognition, which is realized based on a material pool and a excavator movably arranged beside the material pool;

[0006] The excavator comprises a digging arm and two mutually openable and closable buckets; an image acquisition device and a laser radar are installed on the edge of the material pool towards the inside of the material pool; and a control unit is used to control the digging arm and the buckets to dig the material;

[0007] The digging method comprises the following steps:

[0008] S1, the control unit constructs a global three-dimensional map of the material pool based on the data of the image acquisition device and the laser radar; divides the material pool plane according to a preset rule, calculates the average height H(i,j) of each grid and the overall average height Havg of the material pile; selects the grid with H(i,j) > Havg, clusters it into a continuous area, and obtains the wave crest area and the center coordinates thereof; and plans a global operation path for the excavator according to the positions of all the wave crests, wherein i represents the row number of the grid of the material pool plane, and j represents the column number of the grid;

[0009] S2, the control unit controls the excavator to move to a target operation point along the global operation path, and determines the coordinate position of the current excavator;

[0010] S3, calculate the straight-line distance L1 between the wave peak and the side of the stockpile close to the excavator, and determine whether the straight-line distance L1 is greater than or equal to the effective digging width D of the bucket: if yes, the second digging mode is adopted; otherwise, the first digging mode is adopted.

[0011] S4, after the digging is completed, return to step S3 to re-perceive the state in the current working point range until there is no workable material at the current point, and then return to step S2;

[0012] The first digging mode is to control the bucket to dig downward close to the inner wall of the side of the stockpile close to the excavator, and the second digging mode is to adjust the angle of the bucket so that the bucket lip is directed toward the slope of the material close to the side of the excavator, and then control the two buckets to be inserted into the material for digging.

[0013] In step S1, the global working path of the excavator is determined as follows:

[0014] S101, compare the difference between the average height of each wave peak area and the overall average height of the stockpile, and the wave peak area with a difference greater than a threshold value is preferentially included in the working range; if the height differences of multiple wave peak areas are similar, then further compare the areas of the wave peak areas, and the wave peak area with a larger area is preferentially worked;

[0015] S102, according to the above priority rules, sort all wave peak areas from high to low to form a working target sequence of the excavator;

[0016] S103, take the initial deployment position of the excavator as the starting point of the path, and set the center coordinates of each sorted wave peak area as the path nodes in turn; when planning the path, the shortest route between adjacent nodes is selected, and finally the global working path is formed.

[0017] Based on the further improvement of the above structure, the excavator further comprises a pressure sensor for detecting the pressure acting on the bucket lip, and if the pressure exceeds a threshold value during the digging in the first digging mode, it is determined that the bottom is touched, and after the bottom is touched, it returns to step S3.

[0018] Based on the further improvement of the above structure, in step S1, the image acquisition device and the laser radar further acquire the slope angle of the material in the wave peak area;

[0019] The excavator further comprises an angle sensor for detecting the digging angle of the bucket;

[0020] The second digging mode further comprises adjusting the angle of the bucket according to the slope angle, and calibrating through the angle sensor so that the bucket lip is directed toward the slope of the material close to the side of the excavator, and then controlling the two buckets to be inserted into the material for digging.

[0021] Based on the further improvement of the above structure, the image acquisition device comprises a binocular camera;

[0022] The preset rule is to divide the stockyard plane by 1m*1m grid;

[0023] The method for obtaining the wave peak region and the center coordinates thereof comprises: scanning the stockpile in the stockyard by a laser radar to generate two-dimensional coordinates (x, y) of all points on the surface of the stockpile relative to the excavator; filtering background points and noise points outside the stockyard through point cloud preprocessing to retain effective points on the surface of the stockpile; removing isolated points by statistical filtering; taking the center coordinates (x1, y1) of each region as the initial position of the wave peak according to the grid with H(i, j)>Havg obtained in step S1;

[0024] The method for obtaining the position of the wave peak of the material in the stockyard by the image acquisition device and the laser radar further comprises: performing secondary calibration on the initial position of the wave peak by the image acquisition device to obtain the final position of the wave peak:

[0025] The image acquisition device is aimed at the initial position (x1, y1) output by the laser radar, and the camera verifies and corrects the coordinates through visual features to output the final wave peak position, which comprises:

[0026] The camera and the laser radar ensure that the data acquisition time is consistent through a time synchronizer; based on the preset calibration parameters of the camera, the initial wave peak coordinates (x1, y1) of the laser radar are mapped to the two-dimensional image pixel coordinates (u, v) of the camera to locate the corresponding wave peak candidate region in the image; a sub-region of 50*50 pixels around (u, v) in the image is intercepted, and the visual features of the region are analyzed:

[0027] The relative height gradient of the region is calculated through the parallax of the binocular camera to confirm whether the center position (u, v) is the highest point in the region; if the visual feature verification (x1, y1) is true, the coordinates are directly retained;

[0028] If the highest point in the region is offset through visual detection, the two-dimensional coordinates (x2, y2) after the offset are inversely calculated through coordinate mapping; and (x2, y2) is output as the final position of the wave peak;

[0029] The straight line distance L1 is calculated according to the final position coordinates (x2, y2) of the wave peak with the coordinates of the excavator as the origin coordinates (0, 0);

[0030] The method for obtaining the slope angle of the wave peak through the final position of the wave peak comprises:

[0031] A rectangular calculation region is formed by expanding from the final position of the wave crest towards the direction close to the excavator, the length of the rectangular calculation region is arranged along the extension direction of the slope, and the width is arranged perpendicular to the extension direction of the slope; from the three-dimensional point cloud of the stockpile collected by the laser radar, the point cloud with coordinates falling within the rectangular calculation region is screened to form a local point cloud subset; the local point cloud subset is subjected to denoising processing to remove isolated points and retain effective point clouds reflecting the surface state of the slope of the stockpile; the fitting plane matched with the surface shape of the slope region of the stockpile is obtained by performing plane fitting processing on the denoised effective point cloud; and the included angle between the fitting plane and the horizontal plane is calculated, which is the slope angle of the corresponding slope region of the stockpile.

[0032] During the excavation process, the image acquisition device transmits the excavation image in real time for remote manual monitoring.

[0033] Based on the further improvement of the above structure, the two buckets are a first bucket and a second bucket, and the first bucket and the second bucket form a bucket assembly, wherein the back of the first bucket is arranged in a straight line structure, and the back of the second bucket is arranged in an arc structure.

[0034] Based on the further improvement of the above structure, the bucket assembly further comprises:

[0035] A first upper arm and a second upper arm, the first upper arm is installed on the first bucket, the second upper arm is installed on the second bucket, and the first upper arm and the second upper arm are hingedly connected to each other;

[0036] A support frame, the hinge point of the first upper arm and the second upper arm is rotatably connected to the inner side of the support frame through a rotating shaft;

[0037] A first hydraulic cylinder and a second hydraulic cylinder, the cylinder end of the first hydraulic cylinder is hingedly connected to the inner top side of the support frame, and the telescopic end is hingedly connected to the upper end of the first upper arm, the cylinder end of the second hydraulic cylinder is hingedly connected to the inner top side of the support frame, and the telescopic end is hingedly connected to the upper end of the second upper arm, and the pressure sensor has two, which are installed at the pressure measuring port of the first hydraulic cylinder and the second hydraulic cylinder respectively;

[0038] A connecting frame, the front end of the connecting frame is rotatably connected to the support frame, and the rear end is rotatably connected to the excavating arm of the excavator;

[0039] A third hydraulic cylinder, the cylinder end of the third hydraulic cylinder is hingedly connected to the inner side of the top of the connecting frame, and the telescopic end is hingedly connected to the outer side of the support frame;

[0040] The control unit is connected with and controls the excavating arm, the first hydraulic cylinder, the second hydraulic cylinder and the third hydraulic cylinder, the control unit drives the opening and closing of the first excavating bucket and the second excavating bucket by controlling the extension and retraction of the first hydraulic cylinder and / or the second hydraulic cylinder, and adjusts the orientation of the first excavating bucket and the second excavating bucket by controlling the extension and retraction of the third hydraulic cylinder.

[0041] Based on the further improvement of the above structure, the first excavating mode comprises the following steps:

[0042] S11, the control unit controls the first hydraulic cylinder and / or the second hydraulic cylinder, so that the first excavating bucket and the second excavating bucket are opened to the maximum angle, the excavating arm action is adjusted, so that the back of the first excavating bucket is close to the inner side wall of the material pool, and the entire excavating bucket assembly is driven to move vertically downward, so that the first excavating bucket and the second excavating bucket are inserted into the material pile;

[0043] S12, the control unit controls the first hydraulic cylinder and / or the second hydraulic cylinder to close the first excavating bucket and the second excavating bucket to each other, and after the excavating bucket assembly is pulled out by controlling the excavating arm of the excavator, the excavating is completed.

[0044] In step S11, during the downward movement of the excavating bucket assembly, when the excavating bucket touches the bottom, the downward movement of the excavating bucket is stopped, and then in step S12, during the mutual closing of the first excavating bucket and the second excavating bucket, the lip of the first excavating bucket and the lip of the second excavating bucket are ensured to be close to the inner bottom surface of the material pool.

[0045] Based on the further improvement of the above structure, the second excavating mode comprises the following steps:

[0046] S21, the control unit controls the first hydraulic cylinder and / or the second hydraulic cylinder, so that the first excavating bucket and the second excavating bucket are opened to the maximum angle, the angle of the excavating bucket assembly is adjusted by the third hydraulic cylinder, so that the lip of the first excavating bucket and the lip of the second excavating bucket are directed towards the slope close to the side of the excavator, and the control unit controls the excavating arm, so that the lip of the first excavating bucket and the lip of the second excavating bucket are inserted into the slope close to the side of the excavator.

[0047] S22, the control unit controls the first hydraulic cylinder and / or the second hydraulic cylinder to close the first excavating bucket and the second excavating bucket to each other, and after the excavating bucket assembly is pulled out by controlling the excavating arm of the excavator, the excavating is completed.

[0048] Based on the further improvement of the above structure, the laser radar defines the area with a relative height ≥0.5m or the area with a slope angle >30° as the wave crest.

[0049] The relative height is a difference between an overall average height Havg of the material pile and a height of the target material pile;

[0050] The excavating method further comprises: if a wave crest is not detected, the control unit controls the excavator to excavate in place, and for a region outside the effective excavating width D of the bucket, the following steps are adopted to excavate:

[0051] S311, the control unit controls the first hydraulic cylinder and / or the second hydraulic cylinder, so that the first bucket and the second bucket are opened to a maximum angle, the angle of the bucket assembly is adjusted by the extension and retraction of the third hydraulic cylinder, so that the lip of the first bucket and the lip of the second bucket are both inserted into the material pile at a vertical angle, and the control unit controls the excavator arm, so that the lip of the first bucket and the lip of the second bucket are both inserted into the material pile at a vertical angle;

[0052] S312, the control unit controls the first bucket and the second bucket to be closed to each other, and after the bucket assembly is pulled out by controlling the excavator arm, the excavating is completed.

[0053] For a region within the effective excavating width D of the bucket, the first excavating mode is adopted to excavate.

[0054] The advantages or beneficial effects of the above technical solutions at least include: the material excavating method based on image recognition of the present application effectively avoids data deviation of a single sensor in a complex environment such as strong light and high dust, reduces system misjudgment, ensures the accuracy of wave crest positioning and slope angle calculation, and ensures stable operation; at the same time, by constructing a global three-dimensional map of the material pool, dividing the grid to select the wave crest area and planning the global operation path, selecting the adaptive excavating mode according to the distance between the wave crest and the excavator, shortening the extension distance of the excavator arm as much as possible, reducing the time consumption of single excavating, and improving the overall operation efficiency; the bottom touch judgment in the first excavating mode is realized by means of the pressure sensor, the equipment is prevented from being damaged by excessive downward movement of the bucket, the opening and closing angles and the torque of the bucket assembly are improved by the structural optimization of the hydraulic cylinder and the upper arm, the corresponding excavating modes are designed for different scenarios such as with or without wave crest and different wave crest distances, the excavating requirements are adapted and the equipment wear is reduced; in addition, the wave crest position is adjusted by real-time updating of the three-dimensional point cloud data of the material pile, the operation is carried out based on real-time working conditions to improve the excavating accuracy, the image acquisition device can also transmit the excavating image in real time to support remote manual monitoring, which is convenient for timely processing of abnormal situations and further ensures the stability of the operation. BRIEF DESCRIPTION OF DRAWINGS

[0055] The accompanying drawings, which are included to provide a further understanding of the application and are incorporated in and constitute a part of this specification, illustrate embodiments of the application and together with the description serve to explain the principles of the application.

[0056] Figure 1 A schematic diagram showing the position relationship between the excavator and the material pool according to an embodiment of the present application;

[0057] Figure 2 A schematic diagram showing the structure of the bucket assembly according to an embodiment of the present application;

[0058] Figure 3 A schematic diagram showing the change of the orientation of the bucket after the extension and retraction of the third oil cylinder according to an embodiment of the present application;

[0059] Figure 4 A schematic diagram showing the assembly of the bucket assembly according to an embodiment of the present application;

[0060] Figure 5 A schematic diagram showing the assembly of the first bucket and the second bucket according to an embodiment of the present application;

[0061] Figure 6 A schematic diagram showing the steps of the first digging mode according to an embodiment of the present application, wherein step a is to make the back of the first bucket close to the sidewall of the material pool, step b is to dig the first bucket and the second bucket into the material with the lip of the bucket touching the bottom, and steps c and d are to close the first bucket and the second bucket to complete the digging of the material;

[0062] Figure 7 A schematic diagram showing the orientation of the bucket towards the slope in the second digging mode according to an embodiment of the present application, wherein the dotted arrow represents the digging direction of the bucket;

[0063] Figure 8 A schematic diagram showing the wave crest appearing outside the digging width D according to an embodiment of the present application;

[0064] Figure 9 A schematic diagram showing the wave crest appearing inside the digging width D according to an embodiment of the present application;

[0065] Figure 10 A schematic diagram showing the rotation of the rotating platform of the excavator towards the target area and the rotation of the connecting frame to make the first bucket close to the sidewall of the material pool according to an embodiment of the present application;

[0066] Figure 11 A schematic diagram showing the comparison of the opening angle of the bucket assembly according to the present application and the bucket assembly according to the prior art, wherein the upper half of the diagram is the bucket assembly according to the present application, and the lower half of the diagram is the bucket assembly according to the prior art;

[0067] Figure 12A comparison diagram of the digging directions of the first digging mode and the second digging mode is shown, the upper half of the diagram is the first digging mode, the lower half of the diagram is the second digging mode, the blue arrow represents the digging direction of the bucket, and the red arrow represents the dumping direction of the material pile after being dug;

[0068] Figure 13 A schematic diagram of the deployment position of the distance sensor is shown; the red dashed line in the diagram is the detection line of the distance sensor, X and Y represent the X axis and Y axis of the coordinate system, and O represents the origin of the coordinate system.

[0069] Reference signs: 10, material pool; 20, excavator; 21, excavator arm; 22, bucket assembly; 221, first bucket; 222, second bucket; 223, support frame; 224, connecting frame; 225, first hydraulic cylinder; 226, second hydraulic cylinder; 227, third hydraulic cylinder; 2201, first upper arm; 2202, second upper arm; 23, base; 30, distance sensor. DETAILED DESCRIPTION

[0070] The embodiment of the present application provides a material digging method based on image recognition, which comprises the following steps: Figure 1 As shown, the method is realized based on a material pool and an excavator movably arranged beside the material pool;

[0071] The excavator comprises an excavator arm and two buckets that can be opened and closed to each other; an image acquisition device and a laser radar are installed on the edge of the material pool and face the material pool; and a control unit is used for controlling the excavator arm and the buckets to dig the material.

[0072] The digging method comprises the following steps:

[0073] S1, the control unit constructs a global three-dimensional map of the material pool based on the data of the image acquisition device and the laser radar; the material pool plane is divided according to a preset rule, specifically, the material pool plane is divided according to a 1m*1m grid, the average height H(i,j) of each grid and the overall average height Havg of the material pile are calculated, the grid with H(i,j)>Havg is screened out and clustered into a continuous area to obtain a wave peak area and the center coordinates thereof; and a global working path is planned for the excavator according to the positions of all the wave peaks, wherein i represents the row number of the grid of the material pool plane, and j represents the column number of the grid; specifically, i corresponds to the grid serial number in the y-axis direction of the coordinate system, that is, the row, j corresponds to the grid serial number in the x-axis direction, that is, the column, and H(i,j) is the average value of all effective point cloud heights in the grid of the "i row and j column", which is used for subsequent comparison with the overall average height of the material pile to screen the wave peak area.

[0074] S2, the excavator is controlled to move to a target working point along the global working path, and the coordinate position of the current excavator is determined;

[0075] S3, calculate the straight-line distance L1 between the wave peak and the side of the pool close to the excavator, and determine whether the straight-line distance L1 is greater than or equal to the effective digging width D of the bucket: if yes, the second digging mode is adopted; the second digging mode is: adjusting the angle of the bucket so that the bucket lip is all directed towards the slope close to the excavator side of the material, and then controlling the two buckets to insert into the material for digging; otherwise, the first digging mode is adopted, and the first digging mode is: controlling the bucket to dig downwards close to the inner wall of the pool close to the excavator side.

[0076] S4, after the digging is completed, returning to step S3 to re-perceive the state in the current working point range until there is no workable material at the current point, and then returning to step S2.

[0077] In step S1, the global working path of the excavator is determined as follows:

[0078] S101. Compare the height difference of each wave peak region with the overall average height of the material pile, and the wave peak region with a height difference greater than the threshold value is preferentially included in the working range; if the height differences of multiple wave peak regions are similar, further compare the areas of the wave peak regions, and the wave peak region with a larger area is preferentially worked: first, based on the average height H(k) of each wave peak region obtained in step S1, where k is the wave peak number, for example: k = 1, 2,..., n, the overall average height Havg of the material pile also needs to be obtained, and the height difference of each wave peak region is calculated: ΔH(k) = H(k) - Havg, where ΔH(k) represents the difference between the average height of the kth wave peak region and the overall average height of the material pile, and the greater the positive value, the more prominent the material accumulation of the wave peak region.

[0079] Set the height difference threshold value ΔH0 (which can be preset according to the material characteristics, for example, ΔH0 = 0.3m), and preferentially include the wave peak region with ΔH(k) > ΔH0 in the working range (such wave peak is a "high-priority wave peak" that needs to be preferentially processed to avoid material accumulation being too high to cause collapse or difficulty in digging).

[0080] If the height differences of multiple wave peak regions satisfy |ΔH(k1) - ΔH(k2)| ≤ 0.1m (i.e. the height differences are similar, and 0.1m is the similar determination threshold value which can be adjusted according to the actual scene), further compare the areas S(k) of the wave peak regions (S(k) is the total number of grids contained in the continuous region obtained by clustering in step S1 multiplied by the area of a single grid, for example, in the case of 1m x 1m grid, S(k) = grid number x 1m 2 ), and the wave peak region with a larger area (i.e. containing more material) is preferentially worked.

[0081] S102. According to the above priority rules, all wave peak regions are sorted from high to low to form the working target order of the excavator:

[0082] First step: sort all the peaks according to AH(k) from large to small, the larger the AH(k), the higher the peak is sorted;

[0083] Second step: for the peaks with similar AH(k) (satisfying |AH(k1)-AH(k2)|≤0.1m), sort them according to their areas S(k) from large to small, the larger the area, the higher the peak is sorted;

[0084] Finally, the job target order list is formed: [k1, k2,..., kn], where k1 is the first job peak, and kn is the last job peak.

[0085] For example: if there are 3 peaks, their AH is 0.8m, 0.75m, and 0.4m, and their areas are 5m 2 , 8m 2 , and 10m 2 . Since the difference between 0.8m and 0.75m is 0.05m (≤0.1m), it belongs to similar height, at this time, compare the area: 8m 2 > 5m 2 , so the order is [k2, k1, k3].

[0086] S103. Take the initial deployment position of the excavator as the starting point of the path, and set the center coordinates of each peak area sorted as path nodes in turn; when planning the path, choose the route with the shortest distance between adjacent nodes, and finally form the global operation path:

[0087] Take the initial deployment position of the excavator as the starting point of the path, recorded as coordinates P0(x0,y0) (this coordinate can be obtained by the positioning device of the excavator itself, such as GPS or laser radar and the relative positioning of the material pool edge fixed device).

[0088] The center coordinates of the peak area sorted in step S22 are taken as path nodes in turn, that is, P1(x1,y1), P2(x2,y2),..., Pn(xn,yn), where (xk,yk) is the center coordinates of the kth peak, which is determined by the method of "peak area and its center coordinates" in step S1.

[0089] When planning the path between adjacent nodes, take "shortest distance" as the optimization target, that is, choose the route with the shortest straight line distance between adjacent nodes Pi(xi,yi) and Pj(xj,yj), and the distance calculation formula is: ;

[0090] The global operation path is the continuous route from the starting point P0 to P1→P2→...→Pn, that is, the path sequence is: P0→P1→P2→...→Pn.

[0091] The path planning manner can ensure that the excavator completes all work of the peak area with the shortest moving distance, reduces invalid moving time, and improves digging efficiency; meanwhile, by preferentially processing the peak with high height and large material amount, the excavator avoids increasing digging difficulty or safety hazards caused by local overhigh material stacking.

[0092] The current excavator coordinate position can be determined in various ways:

[0093] The first way is to determine whether the position of the excavator reaches the target working point coordinate by using a Beidou positioning system or a GPS positioning system.

[0094] The second way is as shown in Figure 13 , distance sensors are respectively installed at both ends of each side of the opening at the top of the material pool to detect the distance between the excavator and each distance sensor, and the distance is corresponded with the coordinate, which is a coordinate system with a corner of the material pool as the origin, as shown in Figure 13 , the black thick line in the figure represents the coordinate system, the upper and lower two red dashed lines are parallel to the X axis of the coordinate system, and the left and right two red dashed lines are parallel to the Y axis of the coordinate system, so that the degree of the distance sensor can be read to obtain the coordinate of the excavator in the coordinate system.

[0095] Based on the further improvement of the above structure, the excavator further comprises a pressure sensor for detecting the pressure acting on the bucket lip, and in the first digging mode, if the pressure exceeds a threshold value, it is judged that the bottom is touched, and after the bottom is touched, it returns to step S3.

[0096] It should be noted that the effective digging width D of the bucket is the digging width after one shovel digging and closing after the two buckets are opened to the maximum angle, rather than the maximum working range after the excavator arm 21 is stretched.

[0097] The first digging mode is as shown in Figure 12 , the upper half, the bucket is controlled to be close to the inner wall of the material pool 10 close to one side of the excavator 20 to dig downward, and whether the bottom is touched is judged by the pressure sensor during the digging, and after the bottom is touched, it returns to step S1; in the figure, the blue arrow is the digging direction, and the red arrow is the direction in which the material on the peak collapses after the digging, losing the support of the material in the blue arrow area;

[0098] The second digging mode is as shown in Figure 12 , the lower half, the angle of the bucket is adjusted so that the bucket lip of the bucket is directed to the slope of the material close to the excavator 20, and then the two buckets are controlled to be inserted into the material for digging; in the figure, the blue arrow is the digging direction, and the red arrow is the direction in which the material on the peak collapses after the digging, losing the support of the material in the blue arrow area.

[0099] Whether it is the first digging mode or the second digging mode, the purpose is to dig the part of the material closest to the excavator 20 as much as possible, so that the excavator 20 can reduce the elongation distance of the excavator arm 21 as much as possible, thereby shortening the digging time of a single digging.

[0100] The image acquisition device includes a binocular camera.

[0101] The image acquisition device includes a binocular camera.

[0102] The method for obtaining the final position of the wave peak includes:

[0103] The laser radar scans the material pile in the material pool to generate two-dimensional coordinates (x, y) of all points on the surface of the material pile relative to the excavator; the point cloud is preprocessed to filter out background points and noise points outside the material pool, and valid points on the surface of the material pile are retained; statistical filtering is used to remove isolated points; the material pool plane is divided into a 1m×1m grid, and the valid points are attributed to the corresponding grid; the average height H(i,j) of each grid and the overall average height Havg of the material pile are calculated; the initial wave peak region is screened out from the grid with H(i,j)>Havg, clustered into a continuous region, and the center coordinates (x1,y1) of each region are taken as the initial position of the wave peak.

[0104] The method for obtaining the final position of the wave peak includes:

[0105] The image acquisition device is aimed at the initial position (x1,y1) output by the laser radar, and the camera verifies and corrects the coordinates through visual features to output the final wave peak position.

[0106] The camera and the laser radar ensure that the data acquisition time is consistent through a time synchronizer; based on the preset calibration parameters of the camera, the initial wave peak coordinates (x1,y1) of the laser radar are mapped to the two-dimensional image pixel coordinates (u,v) of the camera, and the corresponding wave peak candidate region in the image is located; a sub-region of 50×50 pixels around (u,v) in the image is intercepted, and the visual features of the region are analyzed.

[0107] The relative height gradient of the area is calculated by the parallax of the binocular camera, and it is determined whether the center position (u, v) is the highest point in the area; if the visual feature verification (x1, y1) is true, the highest point of the real wave peak is directly reserved;

[0108] If the visual feature verification (x1, y1) is true, the highest point of the real wave peak is directly reserved;

[0109] The straight line distance L1 is calculated according to the coordinates (x2, y2) of the final position of the wave peak with the coordinates (0, 0) of the excavator as the origin coordinates;

[0110] The method for obtaining the slope angle of the wave peak through the final position of the wave peak comprises:

[0111] A rectangular calculation area is formed by expanding towards the excavator from the center of the final position of the wave peak, the length of the rectangular calculation area is arranged along the extension direction of the slope, and the width is arranged perpendicular to the extension direction of the slope; the points cloud falling within the rectangular calculation area is screened from the three-dimensional point cloud of the stockpile collected by the laser radar, and a local point cloud subset is formed; the local point cloud subset is subjected to denoising processing to remove isolated points and retain effective point clouds reflecting the surface state of the slope of the stockpile; the effective point clouds after denoising are subjected to plane fitting processing to obtain a fitting plane matching the surface shape of the slope of the stockpile; the included angle between the fitting plane and the horizontal plane is calculated, and the included angle is the slope angle of the corresponding slope region of the stockpile;

[0112] The method for obtaining the slope angle of the wave peak through the final position of the wave peak further comprises: adding at least two adjacent rectangular verification regions around the final position of the wave peak, repeating the above steps, and calculating the slope angles corresponding to each verification region respectively; the slope angle and the slope angles corresponding to each verification region are subjected to weighted average processing, and the weighted weights are distributed according to the distance between the regions and the final position of the wave peak, and the closer the distance, the higher the weight, to obtain the final slope angle of the stockpile;

[0113] Specifically, the main calculation region with the final position of the wave peak as the center is A0, and the calculated slope angle is θ0; the two adjacent verification regions added are A1 and A2, and the calculated slope angles are θ1 and θ2; if more verification regions are added subsequently, the logic can be extended, such as A3 and θ3;

[0114] I. Set the distance parameter:

[0115] Let: the straight-line distance from the center point of each region to the "peak final position" be d0, d1, d2; Key rule: the center point of the main region A0 is the peak final position, so d0 = 0 (the closest distance); A1 and A2 are adjacent regions, the distance satisfies d1 < d2 (or adjust according to the actual layout, as long as "the closer to the peak, the smaller the d value").

[0116] II. Mathematical model calculation

[0117] Step 1: Calculate the weight of each region (W0, W1, W2);

[0118] Weight core logic: the smaller the distance d, the greater the weight W, and the sum of the weights of all regions is 1.

[0119] Use "distance reciprocal" to calculate the basic weight, and then use "normalization" to ensure that the weight sum is 1, as follows:

[0120] 1. First, calculate the basic weight:

[0121] Main region A0: since d0 = 0, set its basic weight to a fixed value (such as K, recommended to take 2 to ensure its maximum weight);

[0122] Verification region A1: basic weight = 1 / d1;

[0123] Verification region A2: basic weight = 1 / d2;

[0124] 1. Then normalize the weight (ensure W0 + W1 + W2 = 1):

[0125] Total basic weight = K + (1 / d1) + (1 / d2);

[0126] Main region weight: W0 = K / [K + (1 / d1) + (1 / d2)];

[0127] Verification region A1 weight: W1 = (1 / d1) / [K + (1 / d1) + (1 / d2)];

[0128] Verification region A2 weight: W2 = (1 / d2) / [K + (1 / d1) + (1 / d2)];

[0129] Step 2: Calculate the final slope angle; (θ_final)

[0130] Through the weighted sum of "weight x slope angle", the final result is obtained:

[0131] θ_final = W0 x θ0 + W1 x θ1 + W2 x θ2;

[0132] Example: Suppose in the actual scenario:

[0133] Main area A0: θ0=35°, d0=0, take K=2;

[0134] Verification area A1: distance d1=1m, θ1=33°;

[0135] Verification area A2: distance d2=2m, θ2=31°;

[0136] First step, calculate the weight:

[0137] 1. Basic weight:

[0138] A0 basic weight=2; A1 basic weight=1 / 1=1; A2 basic weight=1 / 2=0.5;

[0139] Total basic weight=2+1+0.5=3.5;

[0140] 1. Normalized weight:

[0141] W0=2 / 3.5≈0.57 (57%); W1=1 / 3.5≈0.29 (29%); W2=0.5 / 3.5≈0.14 (14%); In line with "the closer the distance, the higher the weight": A0>A1>A2;

[0142] Second step, calculate the final slope angle:

[0143] θ_final=0.57×35°+0.29×33°+0.14×31°≈33.8°.

[0144] It should be noted that the point cloud collection quality of the laser radar is easy to decline in strong light (such as noon sunlight directly on the material pile, reflected signal interference point cloud quality), high dust (such as dust generated by sand excavation shielding laser), which may cause the wave peak positioning and slope angle calculation to fail; the image acquisition device can be used as a redundant sensing means to assist in confirming the approximate position of the wave peak and judging the slope direction when the laser radar data is abnormal, relying on the visual image (even in strong light / dust environment, the approximate outline of the material pile can still be identified) to avoid the failure of the whole system due to the failure of a single sensor, and to improve the adaptability of the scheme in complex working environment.

[0145] During the excavation process, the image acquisition device can also transmit the excavation image in real time for manual remote monitoring.

[0146] The first excavation mode in the above is to control one of the buckets to tightly approach the inner wall of the material pool 10 close to one side of the excavator 20 to excavate downward, and during the excavation process in the first excavation mode, the value of the pressure sensor is detected in real time, and when the preset value is reached, it is judged that the bucket lip touches the bottom, and the downward movement of the bucket is stopped;

[0147] The second digging mode is to adjust the angle of the bucket according to the slope angle, calibrate through the angle sensor, make the lip of the bucket face the slope on the side close to the excavator 20 after the material, and control the two buckets to insert into the material for digging;

[0148] When the laser radar calculates the slope angle by local point cloud plane fitting, if there are local collapses (such as depressions formed by loose accumulation of grains) and protrusions (such as local clusters of sand) on the surface of the material pile, it is easy to cause the fitted plane to deviate from the actual slope shape; the camera can extract the visual contour line corresponding to the slope calculation area, and assist in verifying the slope angle calculated by the laser radar through the angle between the contour line and the horizontal line. If the deviation exceeds the threshold, the laser radar can be triggered to re-perform point cloud fitting to ensure that the final slope angle is consistent with the actual slope shape of the material pile, providing a reliable basis for the "adjusting the angle of the bucket according to the slope angle" in the second digging mode, and avoiding insufficient insertion depth of the bucket or excessive disturbance of the material pile due to deviation of the slope angle.

[0149] After each digging is completed, the three-dimensional point cloud data of the material pile is re-acquired by the laser radar to update the position of the wave crest, instead of digging based on the data acquired for the first time, until the material taking operation is completed.

[0150] As shown in Figure 2 , the two buckets in the above are a first bucket 221 and a second bucket 222, and the first bucket 221 and the second bucket 222 form a bucket assembly 22, wherein the back of the first bucket 221 is arranged in a straight line structure, and the back of the second bucket 222 is arranged in an arc structure.

[0151] The bucket assembly 22 further comprises:

[0152] A first upper arm 2201 and a second upper arm 2202, the first upper arm 2201 is installed on the first bucket 221, and the second upper arm 2202 is installed on the second bucket 222, and the first upper arm 2201 and the second upper arm 2202 are hingedly connected to each other;

[0153] A support frame 223, the hinge point of the first upper arm 2201 and the second upper arm 2202 hingedly connected to each other is rotatably connected to the inner side of the support frame 223 through a rotating shaft;

[0154] As shown in Figure 4 , Figure 5 , and Figure 7As shown, there is a first hydraulic cylinder 225 and a second hydraulic cylinder 226. The cylinder end of the first hydraulic cylinder 225 is hinged to the inner top side of the support frame 223, and the telescopic end is hinged to the upper end of the first upper arm 2201. The cylinder end of the second hydraulic cylinder 226 is hinged to the inner top side of the support frame 223, and the telescopic end is hinged to the upper end of the second upper arm 2202. There are two pressure sensors, which are respectively installed at the pressure measuring ports of the first hydraulic cylinder 225 and the second hydraulic cylinder 226. The pressure from the bucket lip will be transmitted to the first hydraulic cylinder 225 and the second hydraulic cylinder 226. The pressure measuring port indicates whether the bucket lip has touched the bottom.

[0155] like Figure 2 As shown, a first hydraulic cylinder 225 is mounted above the second bucket 222, driving the first bucket 221 to rotate. A second hydraulic cylinder 226 is mounted above the first bucket 221, driving the second bucket 222 to rotate. The two hydraulic cylinders can be controlled independently. This design also increases the torque required to rotate the first and second buckets 221 and 222 via the first and second upper arms 2201 and 2202. Compared to existing technology that directly hinges the telescopic ends of the hydraulic cylinders to the bucket back, this design not only achieves greater torque under the same hydraulic cylinder thrust but also increases the bucket's opening and closing angle. Figure 11 As shown.

[0156] The connecting frame 224 is rotatably connected to the support frame 223 at its front end. An angle sensor is installed at the pivot of the connecting frame 224 and the support frame 223 to identify the bucket's entry angle. The rear end is rotatably connected to the excavator arm 21 of the excavator 20. Specifically, a motor is installed at the bottom of the connecting frame 224, and a pivot is fixed at the front end of the excavator arm 21. The pivot rotatably passes through the upper end of the connecting frame 224 and is equipped with a first gear. A second gear that meshes with the first gear is installed at the output end of the motor, thereby rotating the connecting frame 224 (the motor, the first gear, and the second gear are not shown in the figure).

[0157] In the first digging mode, one of the buckets needs to be pressed against the side wall of the material pool 10. Therefore, the rotating platform of the excavator 20 needs to rotate towards the target area on both sides, and then the output end of the drive motor rotates, driving the second gear through the first gear to rotate, thereby realizing the rotation of the connecting frame 224. This allows the bucket back of one of the buckets to be pressed against the side wall of the material pool 10. Figure 10 As shown.

[0158] The third hydraulic cylinder 227 has its cylinder end hinged to the inner side of the top of the connecting frame 224, and its telescopic end hinged to the outer side of the support frame 223.

[0159] The control unit is connected with and controls the excavating arm 21, the first hydraulic cylinder 225, the second hydraulic cylinder 226 and the third hydraulic cylinder 227. The control unit drives the opening and closing of the first excavating bucket 221 and the second excavating bucket 222 by controlling the extension and retraction of the first hydraulic cylinder 225 and / or the second hydraulic cylinder 226. The control unit adjusts the orientation of the first excavating bucket 221 and the second excavating bucket 222 by controlling the extension and retraction of the third hydraulic cylinder 227.

[0160] The first excavating mode includes the following steps:

[0161] S11, the control unit controls the first hydraulic cylinder 225 and / or the second hydraulic cylinder 226 to open the first excavating bucket 221 and the second excavating bucket 222 to the maximum angle, adjusts the action of the excavating arm 21 so that the back of the first excavating bucket 221 is in close contact with the inner side wall of the material pool 10, drives the entire excavating bucket assembly 22 to move vertically downward, so that the first excavating bucket 221 and the second excavating bucket 222 are inserted into the material pile, as shown in Figure 6 Steps a to b;

[0162] S12, the control unit controls the first hydraulic cylinder 225 and / or the second hydraulic cylinder 226 to close the first excavating bucket 221 and the second excavating bucket 222 to each other, and after the excavating bucket assembly 22 is pulled out by controlling the excavating arm 21 of the excavator 20, the excavating is completed, as shown in Figure 6 Steps c to d;

[0163] In step S11, during the downward movement of the excavating bucket assembly 22, when the excavating bucket touches the bottom, the downward movement of the excavating bucket is stopped, and then in step S12, during the mutual closing of the first excavating bucket 221 and the second excavating bucket 222, the lip of the first excavating bucket 221 and the lip of the second excavating bucket 222 are ensured to be in close contact with the inner bottom surface of the material pool 10.

[0164] The second excavating mode includes the following steps:

[0165] S21, the control unit controls the first hydraulic cylinder 225 and / or the second hydraulic cylinder 226 to open the first excavating bucket 221 and the second excavating bucket 222 to the maximum angle, adjusts the angle of the excavating bucket assembly 22 by the third hydraulic cylinder 227, so that the lip of the first excavating bucket 221 and the lip of the second excavating bucket 222 are directed towards the slope close to the side of the excavator 20, and the control unit controls the excavating arm 21 to insert the lip of the first excavating bucket 221 and the lip of the second excavating bucket 222 into the slope close to the side of the excavator 20;

[0166] S22, the control unit controls the first hydraulic cylinder 225 and / or the second hydraulic cylinder 226 to close the first excavating bucket 221 and the second excavating bucket 222 to each other, and after the excavating bucket assembly 22 is pulled out by controlling the excavating arm 21 of the excavator 20, the excavating is completed.

[0167] As shown in Figure 8As shown, if the position of the wave crest is calculated to be at point A, and it is determined that point A1 is the wave crest close to the slope of the excavator 20, the distance between point A and the side of the stockpile 10 close to the excavator 20 is the actual straight-line distance L1, which is obviously within the effective digging width D of the bucket, and the control unit controls the excavator 20 to dig at the position of point A1 in the second digging mode; as shown, Figure 9 As shown, if the position of the wave crest is calculated to be at point B, the distance between point B and the side of the stockpile 10 close to the excavator 20 is the actual straight-line distance L1, which is obviously not within the effective digging width D of the bucket, and point B is close to the side of the stockpile 10 close to the excavator 20, the control unit controls the excavator 20 to dig at the position of point B in the first digging mode.

[0168] If the wave crest is outside the maximum working range of the excavator 20, the excavator 20 is moved until the wave crest is within the maximum working range.

[0169] Further, the region with a relative height ≥ 0.5 m, or the region with a slope angle > 30° is defined as the wave crest;

[0170] The relative height is the difference between the average height Havg of the entire stockpile and the height of the target stockpile;

[0171] The digging method further comprises: if no wave crest is detected, the control unit controls the excavator to dig in place, and for a region greater than the effective digging width D of the bucket, the following steps are used for digging:

[0172] S311, the control unit controls the first hydraulic cylinder 225 and / or the second hydraulic cylinder 226, so that the first bucket 221 and the second bucket 222 are opened to the maximum angle, the angle of the bucket assembly 22 is adjusted by the extension and retraction of the third hydraulic cylinder 227, so that the lip of the first bucket 221 and the second bucket 222 faces the surface of the stockpile, and the control unit controls the excavating arm 21, so that the lip of the first bucket 221 and the second bucket 222 are inserted into the material pile at a vertical angle;

[0173] S312, the control unit controls the first bucket 221 and the second bucket 222 to close to each other, and after the bucket assembly 22 is pulled out by controlling the excavating arm 21 of the excavator 20, the digging is completed;

[0174] For a region within the effective digging width D of the bucket, the first digging mode is used for digging.

[0175] In the description of the application, it needs to be explained that the terms "center", "upper", "lower", "left", "right", "vertical", "horizontal", "inner", "outer" and the like indicate the orientation or positional relationship based on the orientation or positional relationship shown in the drawings, and are only for the convenience of describing the application and simplifying the description, and do not indicate or imply that the device or element referred to must have a particular orientation, be constructed and operated in a particular orientation, and therefore cannot be understood as a limitation on the application.

[0176] Those skilled in the art should understand that the above embodiments are only for clearly illustrating the application, and do not limit the scope of the application. Based on the above disclosure, other changes or modifications can also be made by those skilled in the art, and these changes or modifications are still within the scope of the application.

Claims

1. An image recognition based material excavation method, characterized by: The method is realized based on a stockpile and a excavator movably arranged beside the stockpile; An image acquisition device and a laser radar are installed on the edge of the stockpile towards the stockpile, the excavator comprises a digging arm and two mutually openable buckets, and a control unit is used to control the digging arm and the buckets to dig the material; The digging method comprises the following steps: S1, the control unit constructs a global three-dimensional map of the stockpile based on the data of the image acquisition device and the laser radar, divides the stockpile plane according to a preset rule, calculates the average height H(i,j) of each grid and the overall average height Havg of the stockpile, selects the grid with H(i,j)>Havg, clusters it into a continuous area, obtains the peak area and the center coordinates thereof, and plans a global operation path for the excavator according to the positions of all the peaks, wherein i represents the row number of the grid of the stockpile plane, and j represents the column number of the grid; S2, the control unit controls the excavator to move to a target operation point along the global operation path and determines the coordinate position of the current excavator; S3, the linear distance L1 between the peak and the side of the stockpile close to the excavator is calculated, and it is judged whether the linear distance L1 is greater than or equal to the effective digging width D of the bucket: if yes, the second digging mode is adopted; otherwise, the first digging mode is adopted; S4, after the digging is completed, step S3 is returned to re-perceive the state in the current operation point range until there is no workable material at the current point, and then step S2 is returned; The first digging mode is that the bucket is controlled to tightly adhere to the inner wall of the side of the stockpile close to the side of the excavator to dig downward, and the second digging mode is that the angle of the bucket is adjusted so that the bucket lip is all directed to the slope of the material close to the side of the excavator, and then the two buckets are controlled to be inserted into the material to dig; In step S1, the global operation path of the excavator is planned according to the following steps: S101, the difference between the average height of each peak area and the overall average height of the stockpile is compared, and a priority rule is established: the peak area with a difference greater than a threshold value is preferentially included in the operation range; if the height differences of multiple peak areas are similar, the areas of the peak areas are further compared, and the peak area with a larger area is preferentially operated; S102, all the peak areas are sorted from high to low according to the priority rule to form an operation target sequence of the excavator; S103, the initial deployment position of the excavator is taken as the starting point of the path, and the center coordinates of each sorted peak area are sequentially taken as the path nodes; when planning the path, the shortest route between adjacent nodes is selected, and finally the global operation path is formed.

2. The image recognition based material excavation method of claim 1, wherein: The excavator further comprises a pressure sensor for detecting the pressure acting on the bucket lip, and if the pressure exceeds a threshold value during the digging in the first digging mode, it is judged that the bottom is touched, and after the bottom is touched, step S3 is returned.

3. The image recognition based material excavation method of claim 1, wherein: In step S1, the image acquisition device and the laser radar further acquire the slope angle of the material in the area where the peak is located; The excavator further comprises an angle sensor for detecting the digging angle of the bucket. The second digging mode further comprises adjusting the angle of the bucket according to the slope angle, calibrated by the angle sensor, so that the bucket lip of the bucket is close to the slope on the side close to the excavator, and then controlling the two buckets to insert into the material for digging.

4. The image recognition based material excavation method of claim 1, wherein: The image acquisition device comprises a binocular camera; The preset rule is to divide the material pool plane by 1m*1m grid; The method for obtaining the wave peak region and the center coordinates thereof comprises: scanning the material pile in the material pool by a laser radar to generate two-dimensional coordinates (x, y) of all points on the surface of the material pile relative to the excavator; filtering background points and noise points outside the material pool through point cloud preprocessing to retain valid points on the surface of the material pile; removing isolated points by using statistical filtering; taking the center coordinates (x1, y1) of each region as the initial position of the wave peak from the grid with H(i, j)>Havg obtained in step S1; The method for obtaining the position of the wave peak of the material in the material pool by the image acquisition device and the laser radar further comprises: performing secondary calibration on the initial position of the wave peak by the image acquisition device to obtain the final position of the wave peak: The image acquisition device aims at the initial position (x1, y1) output by the laser radar, and the camera verifies and corrects the coordinates through visual features to output the final wave peak position, which comprises: The camera and the laser radar ensure that the data acquisition time is consistent through a time synchronizer; based on the preset calibration parameters of the camera, the initial wave peak coordinates (x1, y1) of the laser radar are mapped to the two-dimensional image pixel coordinates (u, v) of the camera to locate the corresponding wave peak candidate region in the image; a 50*50 pixel sub-region around (u, v) in the image is intercepted, and the visual features of the region are analyzed: The relative height gradient of the region is calculated through the parallax of the binocular camera to confirm whether the center position (u, v) is the highest point in the region; if the visual feature verification (x1, y1) is true, the coordinates are directly retained; If the highest point in the region is offset through visual detection, the two-dimensional coordinates (x2, y2) after the offset are inversely calculated through coordinate mapping; and (x2, y2) is output as the final position of the wave peak; A straight line distance L1 is calculated from the final position coordinates (x2, y2) of the wave peak with the coordinates of the excavator as the original coordinates (0, 0). The method for obtaining the slope angle of the wave peak from the final position comprises: A rectangular calculation region is formed by expanding from the final position of the wave peak to the direction close to the excavator, the length of the rectangular calculation region is arranged along the extension direction of the slope, and the width is arranged perpendicular to the extension direction of the slope; the point cloud falling in the rectangular calculation region is screened from the three-dimensional point cloud of the material pile collected by the laser radar to form a local point cloud subset; the local point cloud subset is subjected to denoising processing to remove isolated points and retain valid point clouds reflecting the surface state of the slope of the material pile; the valid point clouds after denoising are subjected to plane fitting processing to obtain a fitting plane matching the shape of the slope surface of the material pile; and the included angle between the fitting plane and the horizontal plane is calculated, which is the slope angle of the corresponding slope region of the material pile; During the digging process, the image acquisition device transmits the digging images in real time for remote manual monitoring.

5. The image recognition based material excavation method of claim 2, wherein: Two said buckets are a first bucket and a second bucket, the first bucket and the second bucket form a bucket assembly, wherein the back of the first bucket is arranged in a straight line structure, and the back of the second bucket is arranged in an arc structure.

6. The image recognition based material excavation method of claim 5, wherein: The bucket assembly further comprises: A first upper arm and a second upper arm, the first upper arm is installed on the first bucket, and the second upper arm is installed on the second bucket, and the first upper arm and the second upper arm are hinged to each other; A support frame, the hinge point of the first upper arm and the second upper arm hinged to each other is rotatably connected to the inner side of the support frame through a rotating shaft; A first hydraulic cylinder and a second hydraulic cylinder, the cylinder end of the first hydraulic cylinder is hinged to the inner top side of the support frame, and the telescopic end is hinged to the upper end of the first upper arm, the cylinder end of the second hydraulic cylinder is hinged to the inner top side of the support frame, and the telescopic end is hinged to the upper end of the second upper arm, and the pressure sensor has two, which are installed at the pressure measuring port of the first hydraulic cylinder and the second hydraulic cylinder respectively; A connecting frame, the front end of the connecting frame is rotatably connected to the support frame, and the rear end is rotatably connected to the excavator arm; A third hydraulic cylinder, the cylinder end of the third hydraulic cylinder is hinged to the inner side of the top of the connecting frame, and the telescopic end is hinged to the outer side of the support frame; The control unit is connected with and controls the excavator arm, the first hydraulic cylinder, the second hydraulic cylinder and the third hydraulic cylinder, the control unit drives the opening and closing of the first bucket and the second bucket by controlling the telescopic of the first hydraulic cylinder and / or the second hydraulic cylinder, and adjusts the orientation of the first bucket and the second bucket by controlling the telescopic of the third hydraulic cylinder.

7. The image recognition based material excavation method of claim 6, wherein: The first digging mode comprises the following steps: S11, the control unit controls the first hydraulic cylinder and / or the second hydraulic cylinder, so that the first bucket and the second bucket are opened to the maximum angle, the bucket back of the first bucket is adjusted to be close to the inner side wall of the material pool, the whole bucket assembly is driven to move vertically downward, and the first bucket and the second bucket are inserted into the material pile; S12, the control unit controls the first hydraulic cylinder and / or the second hydraulic cylinder to close the first bucket and the second bucket to each other, and after the bucket assembly is pulled out by controlling the excavator arm, the digging is completed; In step S11, when the bucket assembly moves downward, it is judged that the bucket touches the bottom, and after the downward movement of the bucket is stopped, in step S12, the bucket lip of the first bucket and the second bucket is ensured to be close to the inner bottom surface of the material pool during the process of closing the first bucket and the second bucket to each other.

8. The image recognition based material excavation method of claim 7, wherein: The second digging mode comprises the following steps: S21, the control unit controls the first hydraulic cylinder and / or the second hydraulic cylinder, so that the first bucket and the second bucket are opened to the maximum angle, the angle of the bucket assembly is adjusted by the third hydraulic cylinder, so that the lip of the first bucket and the second bucket is towards the slope close to the side of the excavator, the control unit controls the excavator arm, so that the lip of the first bucket and the second bucket is inserted into the slope close to the side of the excavator; S22, the control unit controls the first hydraulic cylinder and / or the second hydraulic cylinder, so that the first bucket and the second bucket are closed to each other, after the bucket assembly is pulled out by controlling the excavator arm, the digging is completed.

9. The image recognition-based material digging method according to claim 7, characterized in that: The laser radar defines the area with a relative height ≥ 0.5 m or an area with a slope angle > 30° as the wave crest; The relative height is the difference between the average height Havg of the whole material pile and the height of the target material pile; The digging method further comprises: if no wave crest is detected, the control unit controls the excavator to dig in place, and for the area greater than the effective digging width D of the bucket, the following steps are adopted for digging: S311, the control unit controls the first hydraulic cylinder and / or the second hydraulic cylinder, so that the first bucket and the second bucket are opened to the maximum angle, the angle of the bucket assembly is adjusted by the third hydraulic cylinder, so that the lip of the first bucket and the second bucket is towards the surface of the material pile, the control unit controls the excavator arm, so that the lip of the first bucket and the second bucket is inserted into the material pile at a vertical angle; S312, the control unit controls the first bucket and the second bucket to be closed to each other, after the bucket assembly is pulled out by controlling the excavator arm, the digging is completed; For the area within the effective digging width D of the bucket, the first digging mode is adopted for digging.

Images