Excavator auxiliary perception method and device, excavator

Abstract

The present application relates to the technical field of excavators, and specifically to an excavator-assisted perception method and device, and an excavator. When the present application is applied, after obtaining the scene three-dimensional data of the current working area where the excavator is located, the scene three-dimensional data is cropped to obtain streamlined effective scene three-dimensional data, thereby reducing the amount of data, which can improve the overall execution efficiency of the excavator-assisted perception method and save computing power. Then, based on the current posture data and structural parameters of the excavator, the first spatial position of the bucket relative to the fixed vehicle body in the current state can be obtained, and then based on the effective scene three-dimensional data and the first spatial position, the second spatial position of the bucket relative to the current working area in the current state can be obtained, that is, the specific position of the bucket in the current state is obtained, and according to the specific position, precise control can be performed and collision accidents can be avoided.

Description

Technical Field

[0001] The present application relates to the technical field of excavators, and in particular to an excavator auxiliary perception method and device, and an excavator. Background Art

[0002] Remote-controlled excavator operations can free operators from physical constraints of time and space, improving operational flexibility and efficiency while also avoiding the risks of operating in harsh environments or hazardous conditions. Furthermore, mechanical equipment eliminates physical fatigue and personal safety concerns, significantly improving work efficiency, extending working hours, and expanding the scope of work. In recent years, with the advancement of computer vision technology, its application in 3D reconstruction and 3D measurement has become increasingly widespread.

[0003] When remotely controlling an excavator, operators can only refer to the two-dimensional images captured by cameras. They lack the three-dimensional perception of on-site operations, which can easily lead to empty buckets during excavation. With the rapid development of computer and artificial intelligence technologies, the use of machine learning to automatically detect and track observation targets in real time has become an inevitable trend. During excavator operation, how to use three-dimensional perception technology to determine the exact position of the excavator bucket within the current operating area is a technical problem that needs to be solved in this field. Summary of the Invention

[0004] In view of this, the present application provides an excavator auxiliary perception method and device, and an excavator, which can obtain the exact position of the excavator bucket in the current working area, making the excavation work more accurate.

[0005] In the first aspect, the present application provides an excavator assisted perception method, which is applied to an excavator, wherein the excavator includes a fixed body and a movable body, and the movable body includes a boom, a dipper arm and a bucket; the excavator assisted perception method includes: obtaining scene three-dimensional data of the current working area where the excavator is located, which is scanned with the fixed body as a reference system; cropping the scene three-dimensional data to obtain effective scene three-dimensional data within a preset area; obtaining current posture data and structural parameters of the excavator; determining the first spatial position of the bucket in the reference system of the fixed body in the current state based on the current posture data and the structural parameters; and determining the second spatial position of the bucket relative to the current working area based on the effective scene three-dimensional data and the first spatial position.

[0006] First, during use, after acquiring 3D scene data of the excavator's current operating area, the 3D scene data is cropped to obtain streamlined, effective 3D scene data, thereby reducing the amount of data, improving the overall execution efficiency of the excavator-assisted perception method, and saving computing power. Then, based on the excavator's current posture data and structural parameters, the bucket's first spatial position relative to the fixed vehicle body in its current state can be determined. By converting the effective 3D scene data and the first spatial position, the bucket's second spatial position relative to the current operating area in its current state can be determined, thus obtaining the bucket's specific position in its current state. Based on this specific position, precise control can be performed and collision accidents can be avoided.

[0007] In combination with the first aspect, in a possible implementation, a laser radar is provided on the fixed vehicle body; wherein, the obtaining of the three-dimensional scene data of the current working area where the excavator is located, which is scanned with the fixed vehicle body as a reference system, includes: obtaining first radar point cloud data of the ground of the current working area detected by the laser radar, the first radar point cloud data being based on the laser radar as a reference point; obtaining the structural parameters of the excavator; determining the third spatial position of the hinge point between the boom and the fixed vehicle body on the fixed vehicle body according to the structural parameters; obtaining the fourth spatial position of the laser radar on the fixed vehicle body; and converting the first radar point cloud data into second radar point cloud data with the hinge point as a reference point according to the third spatial position and the fourth spatial position, and using the second radar point cloud data as the three-dimensional scene data.

[0008] In combination with the first aspect, in a possible implementation, a depth-sensing camera device is provided on the fixed vehicle body; wherein, the obtaining of the three-dimensional scene data of the current working area where the excavator is located, which is scanned with the fixed vehicle body as a reference system, includes: obtaining first depth image data of the ground of the current working area detected by the depth-sensing camera device, the first depth image data being based on the depth-sensing camera device as a reference point; obtaining the structural parameters of the excavator; determining the third spatial position of the hinge point between the boom and the fixed vehicle body on the fixed vehicle body according to the structural parameters; obtaining the fifth spatial position of the depth-sensing camera device on the fixed vehicle body; and converting the first depth image data into second depth image data with the hinge point as a reference point according to the third spatial position and the fifth spatial position, and using the second depth image data as the three-dimensional scene data.

[0009] In combination with the first aspect, in a possible implementation method, it also includes: obtaining current control instruction data; and obtaining the vertical working area of ​​the bucket in the vertical direction based on the current control instruction data; wherein, the cropping of the scene three-dimensional data to obtain valid scene three-dimensional data within a preset area includes: obtaining a cropping area on the ground of the current working area based on the vertical working area; and cropping the scene three-dimensional data according to the cropping area to obtain the valid scene three-dimensional data.

[0010] In combination with the first aspect, in a possible implementation, obtaining the current posture data and structural parameters of the excavator includes: obtaining the first posture data and first structural data of the boom; obtaining the second posture data and second structural data of the dipper arm; obtaining the third posture data and third structural data of the bucket; and obtaining the fourth structural data of the fixed vehicle body; wherein, determining the first spatial position of the bucket in the reference system of the fixed vehicle body in the current state based on the current posture data and the structural parameters includes: determining the first spatial position based on the first structural data, the second structural data, the third structural data, the fourth structural data, the first posture data, the second posture data and the third posture data.

[0011] In combination with the first aspect, in a possible implementation method, determining the first spatial position of the bucket in the reference system of the fixed vehicle body in the current state based on the current posture data and the structural parameters also includes: determining the first tooth tip spatial position of the center tooth tip on the bucket in the reference system of the fixed vehicle body based on the first structural data, the second structural data, the third structural data, the fourth structural data, the first posture data, the second posture data and the third posture data; wherein, determining the second spatial position of the bucket relative to the current working area based on the effective scene three-dimensional data and the first spatial position includes: obtaining the first ground clearance of the center tooth tip based on the effective scene three-dimensional data and the first tooth tip spatial position.

[0012] In combination with the first aspect, in a possible implementation method, determining the first spatial position of the bucket in the reference system of the fixed vehicle body in the current state based on the current posture data and the structural parameters also includes: determining the second tooth tip spatial position of each tooth tip on the bucket in the reference system of the fixed vehicle body based on the first structural data, the second structural data, the third structural data, the fourth structural data, the first posture data, the second posture data and the third posture data; wherein, determining the second spatial position of the bucket relative to the current working area based on the effective scene three-dimensional data and the first spatial position includes: obtaining the second ground clearance height of each tooth tip based on the effective scene three-dimensional data and the second tooth tip spatial position.

[0013] In combination with the first aspect, in a possible implementation, after obtaining the three-dimensional scene data of the current working area where the excavator is located by scanning with the fixed vehicle body as the reference system, the method further includes: detecting whether the data points in the three-dimensional scene data are continuous with each other; and if there are discontinuous unknown points in the three-dimensional scene data, inferring the data points corresponding to the unknown points based on the data points adjacent to the unknown points.

[0014] In a second aspect, the present application provides an excavator auxiliary perception device, which is applied to an excavator, wherein the excavator includes a fixed body and a movable body, and the movable body includes a boom, a dipper arm and a bucket; the excavator auxiliary perception device includes: a scene data acquisition module, configured to: obtain scene three-dimensional data of the current working area where the excavator is located, scanned with the fixed body as a reference system; and crop the scene three-dimensional data to obtain effective scene three-dimensional data within a preset area; an excavator parameter acquisition module, configured to: obtain current posture data and structural parameters of the excavator; and a bucket position calculation module, which is communicatively connected to the scene data acquisition module and the excavator parameter acquisition module respectively, and the bucket position calculation module is configured to: determine the first spatial position of the bucket in the reference system of the fixed body in the current state according to the current posture data and the structural parameters; and obtain the second spatial position of the bucket relative to the current working area according to the effective scene three-dimensional data and the first spatial position.

[0015] Since the second aspect is a device corresponding to the first aspect, the technical effects of the second aspect will not be described here in detail.

[0016] In a third aspect, the present application provides an excavator comprising a fixed body and a movable body, the movable body comprising a boom, an arm and a bucket; the excavator further comprises: a scene data detection module, arranged on the fixed body, the scene data detection module being configured to: use the fixed body as a reference system to scan the three-dimensional scene data of the current working area where the excavator is located; and the aforementioned excavator auxiliary perception device, the scene data acquisition module being communicatively connected to the scene data detection module.

[0017] The third aspect includes the second aspect, and the technical effects of the third aspect will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 Shown is a schematic diagram of the method steps of an excavator assisted perception method provided by an embodiment of the present application.

[0019] Figure 2 Shown is a schematic diagram of the method steps of an excavator assisted perception method provided by another embodiment of the present application.

[0020] Figure 3 Shown is a schematic diagram of the method steps of an excavator assisted perception method provided by another embodiment of the present application.

[0021] Figure 4 Shown is a schematic diagram of the method steps of an excavator assisted perception method provided by another embodiment of the present application.

[0022] Figure 5 Shown is a schematic diagram of the method steps of an excavator assisted perception method provided by another embodiment of the present application.

[0023] Figure 6 Shown is a schematic diagram of the method steps of an excavator assisted perception method provided by another embodiment of the present application.

[0024] Figure 7 Shown is a schematic diagram of the method steps of an excavator assisted perception method provided by another embodiment of the present application.

[0025] Figure 8 Shown is a schematic diagram of the method steps of an excavator assisted perception method provided by another embodiment of the present application.

[0026] Figure 9 Shown is a structural schematic diagram of an excavator auxiliary sensing device provided in one embodiment of the present application.

[0027] Figure 10 Shown is a diagram of the usage status of an excavator provided in one embodiment of the present application. DETAILED DESCRIPTION

[0028] The following will be combined with the accompanying drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making any creative efforts are within the scope of protection of this application.

[0029] Exemplary Excavator Assisted Perception Method

[0030] The present application provides an excavator auxiliary perception method, which is applied to an excavator. The excavator includes a fixed body and a movable body, and the movable body includes a boom, a dipper arm and a bucket.

[0031] In one embodiment, if Figure 1 As shown, the excavator auxiliary perception method includes:

[0032] Step 110: Acquire three-dimensional scene data of the current working area where the excavator is located, obtained by scanning with the fixed vehicle body as a reference system.

[0033] In this step, the fixed vehicle body is used as a reference coordinate system, and the current working area where the excavator is located is scanned and detected by the detection equipment to obtain the three-dimensional scene data of the current working area.

[0034] Step 120: crop the scene 3D data to obtain valid scene 3D data within a preset area.

[0035] In this step, because the 3D scene data obtained by the detection equipment is large in volume, and the excavator's boom, arm, and bucket only operate within a small area, the 3D scene data is cropped to reduce the data volume. The working range of the boom, arm, and bucket, or a slightly larger range than the working range, is pre-set as the preset area range. The 3D scene data is then cropped to within the preset area, resulting in valid 3D scene data and reducing the data volume. This step improves the overall execution efficiency of the excavator-assisted perception method and saves computing power.

[0036] Step 130: Acquire the current posture data and structural parameters of the excavator.

[0037] In this step, the current posture of the excavator's movable body can be determined using the excavator's IMU (Inertial Measurement Unit) or other sensors. By acquiring the excavator's structural parameters, the specific shapes and dimensions of the excavator's fixed and movable bodies can be determined. The excavator's structural parameters are pre-stored in the excavator's onboard computer system.

[0038] Step 140: Determine a first spatial position of the bucket in the reference system of the fixed vehicle body in the current state according to the current posture data and the structural parameters.

[0039] In this step, the relative position of the bucket relative to the fixed vehicle body can be calculated based on the current posture and shape and size, that is, the position of the bucket in the reference coordinate system when the fixed vehicle body is used as the reference coordinate system is obtained, and this position is the first spatial position.

[0040] Step 150: Determine a second spatial position of the bucket relative to the current working area based on the effective scene three-dimensional data and the first spatial position.

[0041] In this step, because both the effective scene 3D data and the first spatial position use the fixed vehicle body as the reference coordinate system, coordinate conversion can be used to determine the bucket's relative position relative to the current work area. This relative position serves as the second spatial position. This second spatial position, in turn, determines the bucket's specific position within the current work area. This allows for precise excavator control based on the bucket's specific position, preventing collisions between the excavator's moving body and obstacles.

[0042] When this embodiment is used, after obtaining the scene three-dimensional data of the current working area where the excavator is located, the scene three-dimensional data is cropped to obtain streamlined effective scene three-dimensional data, thereby reducing the amount of data, which can improve the overall execution efficiency of the excavator auxiliary perception method and save computing power. Then, based on the current posture data and structural parameters of the excavator, the first spatial position of the bucket relative to the fixed vehicle body in the current state can be obtained, and then based on the effective scene three-dimensional data and the first spatial position, the second spatial position of the bucket relative to the current working area in the current state can be obtained, that is, the specific position of the bucket in the current state is obtained, and according to the specific position, precise control can be performed and collision accidents can be avoided. In addition, all reference coordinate systems of this application are based on the same principle, for example, they are all right-handed coordinate system principles.

[0043] In one embodiment, a laser radar is provided on the fixed vehicle body, that is, the laser radar is used to scan and obtain three-dimensional data of the scene. Figure 2 As shown, step 110 includes:

[0044] Step 111: Acquire first radar point cloud data of the ground in the current operation area obtained by laser radar detection.

[0045] In this step, the first radar point cloud data is based on the laser radar as a reference point. Specifically, the first radar point cloud data uses the laser radar as the reference system coordinate origin.

[0046] Step 112: Obtain structural parameters of the excavator.

[0047] In this step, the structural parameters of the excavator are obtained to know the specific shapes and sizes of the fixed body and the movable body of the excavator. The structural parameters of the excavator are pre-stored in the vehicle computer system of the excavator.

[0048] Step 113: Determine the third spatial position of the hinge point between the movable arm and the fixed body on the fixed body according to the structural parameters.

[0049] In this step, the specific position of the hinge point on the fixed vehicle body, ie, the third spatial position, can be known through the structural parameters.

[0050] Step 114: Obtain a fourth spatial position of the laser radar on the fixed vehicle body.

[0051] In this step, the installation position of the laser radar on the fixed vehicle body, that is, the fourth spatial position, is obtained. The installation position is pre-stored in the vehicle computer system of the excavator.

[0052] Step 115: Convert the first radar point cloud data into second radar point cloud data with the hinge point as a reference point based on the third spatial position and the fourth spatial position, and use the second radar point cloud data as the three-dimensional scene data.

[0053] In this step, since the third and fourth spatial positions are known, the coordinate system of the first radar point cloud data is translated, and the reference coordinate origin of the first radar point cloud data can be moved to the hinge point. The second radar point cloud data with the hinge point as the reference coordinate origin is used as the three-dimensional scene data. In application, this embodiment specifically installs a laser radar on an excavator, adjusts the laser radar's position to obtain a field of view covering the working area, and uses a pre-set filtering algorithm to crop the terrain point cloud to obtain the second radar point cloud data. This can provide streamlined terrain information to remote control operators around the clock without incurring a large information transmission burden.

[0054] In one embodiment, a depth sensing camera is provided on the fixed vehicle body, that is, the present embodiment uses the depth sensing camera to scan and obtain three-dimensional data of the scene. Figure 3 As shown, step 110 includes:

[0055] Step 116: Acquire first depth image data of the ground in the current working area detected by the depth sensing camera device.

[0056] In this step, the first depth image data is based on the depth sensing camera as a reference point. Specifically, the first depth image data uses the depth sensing camera as a reference coordinate origin.

[0057] Step 112: Obtain structural parameters of the excavator.

[0058] In this step, the structural parameters of the excavator are obtained to know the specific shapes and sizes of the fixed body and the movable body of the excavator. The structural parameters of the excavator are pre-stored in the vehicle computer system of the excavator.

[0059] Step 113: Determine the third spatial position of the hinge point between the movable arm and the fixed body on the fixed body according to the structural parameters.

[0060] In this step, the specific position of the hinge point on the fixed vehicle body, ie, the third spatial position, can be known through the structural parameters.

[0061] Step 117: Acquire a fifth spatial position of the depth-sensing camera device on the fixed vehicle body.

[0062] In this step, the installation position of the depth sensing camera device on the fixed vehicle body, that is, the fifth spatial position, is obtained. The installation position is pre-stored in the vehicle computer system of the excavator.

[0063] Step 118 : Convert the first depth image data into second depth image data with the hinge point as a reference point according to the third spatial position and the fifth spatial position, and use the second depth image data as the three-dimensional data of the scene.

[0064] In this step, since the third spatial position and the fifth spatial position are known, the first depth image data is translated in the coordinate system, so that the reference system coordinate origin of the first depth image data can be moved to the hinge point, and the second depth image data with the hinge point as the reference system coordinate origin is used as the scene three-dimensional data.

[0065] In one embodiment, if Figure 4 As shown, the excavator auxiliary perception method also includes:

[0066] Step 160: Obtain current control instruction data.

[0067] Step 170: Obtain the vertical working area of ​​the bucket in the vertical direction according to the current control instruction data.

[0068] During application, the excavator's upper vehicle may rotate, causing the boom, arm, and bucket to rotate with it. The current control command data indicates the upper vehicle's current rotational azimuth, and knowing the upper vehicle's rotational azimuth provides the current azimuth of the boom, arm, and bucket. Since the boom, arm, and bucket all move vertically, the bucket can only move within the vertical region corresponding to its current azimuth, which serves as the vertical working area.

[0069] Wherein, step 120 includes:

[0070] Step 121: Obtain a clipping area on the ground of the current working area according to the vertical working area.

[0071] In this step, the projection of the vertical working area on the ground can be used as the clipping area, or an area slightly larger than the projection can be used as the clipping area, and the size of the area larger than the projection can be pre-set.

[0072] Step 122: Crop the scene 3D data according to the cropping area to obtain valid scene 3D data.

[0073] In this step, the 3D scene data is cropped, retaining only the data within the cropped area obtained in step 121. The data within this cropped area is used as the valid 3D scene data, thereby streamlining the 3D scene data so that the data volume is precisely located within the ground area that the bucket can reach. Since the bucket can only reach the ground area corresponding to the cropped area, it is only necessary to obtain the 3D scene data within this ground area. The bucket will not reach other ground areas, so there is no need to obtain the 3D scene data for other areas, which greatly reduces the amount of data calculations and improves the overall computational efficiency of this application.

[0074] In one embodiment, if Figure 5 As shown, step 130 includes:

[0075] Step 131: Acquire first posture data and first structure data of the boom.

[0076] Step 132: Acquire the second posture data and second structure data of the arm.

[0077] Step 133: Acquire third posture data and third structure data of the bucket.

[0078] Step 134: Acquire fourth structural data of the fixed vehicle body.

[0079] Wherein, step 140 includes:

[0080] Step 141 : Determine a first spatial position according to the first structural data, the second structural data, the third structural data, the fourth structural data, the first posture data, the second posture data, and the third posture data.

[0081] In this embodiment, since the boom, arm, and bucket can essentially function as a multi-axis robotic arm, sensors such as IMUs installed at various locations can be used to obtain the boom, arm, and bucket postures in real time based on the robot's kinematic model. In step 141, a DH model of the boom, arm, and bucket is created based on the excavator's design model. The boom, arm, and bucket posture and structural data are then input into the DH model to determine the bucket's current first spatial position relative to the fixed vehicle body.

[0082] In one embodiment, if Figure 6As shown, step 140 also includes:

[0083] Step 142: Determine the spatial position of the first tooth tip of the center tooth tip on the bucket in the reference system of the fixed vehicle body based on the first structural data, the second structural data, the third structural data, the fourth structural data, the first posture data, the second posture data, and the third posture data.

[0084] The pre-set third structural data contains specific structural parameters for each part of the bucket, including the specific position of the central tooth tip at the center of the bucket. In this step, when calculating the current first spatial position of the bucket relative to the fixed vehicle body, the third structural data is used to convert the position of the central tooth tip within the fixed vehicle body's reference coordinate system. This position is the first tooth tip spatial position.

[0085] After step 142, step 150 includes:

[0086] Step 151: Obtain a first ground clearance of the center tooth tip based on the effective scene three-dimensional data and the spatial position of the first tooth tip.

[0087] In step 151, since both the effective scene 3D data and the spatial position of the first tooth tip are referenced to the fixed vehicle body, coordinate conversion can be used to determine the height of the bucket's central tooth tip relative to the ground in the current work area. This height is the first ground clearance. Obtaining the first ground clearance indicates the bucket's specific height relative to the ground in the current work area, allowing precise excavation to be performed based on this height. For work in uneven, bumpy, or protruding areas, this system provides real-time feedback on the vertical height of the excavator bucket tip around the clock, reducing the empty bucket rate and improving operational efficiency.

[0088] Specifically, in the fixed vehicle's reference coordinate system, the horizontal directions are defined as the X and Y axes, and the vertical direction is defined as the Z axis. The spatial position of the first tooth tip can be used to determine the Z-axis value Z1 of the center tooth tip in the fixed vehicle's reference coordinate system. Since the effective scene 3D data uses the fixed vehicle as its reference coordinate system, the Z-axis value Z2 of the ground in the effective scene 3D data in the vertical direction of the center tooth tip can be determined. This means the coordinate value Z2 of the ground directly below the center tooth tip is obtained. The first ground clearance is then determined by calculating the difference between Z1 and Z2.

[0089] In one embodiment, if Figure 7 As shown, step 140 also includes:

[0090] Step 143: Determine the spatial position of the second tooth tip of each tooth tip on the bucket in the reference system of the fixed vehicle body according to the first structural data, the second structural data, the third structural data, the fourth structural data, the first posture data, the second posture data, and the third posture data.

[0091] The third structural data contains the specific structural parameters of each bucket component, including the specific positions of each tooth tip on the bucket. In this step, when calculating the current first spatial position of the bucket relative to the fixed vehicle body, the third structural data is used to convert the positions of each tooth tip within the fixed vehicle body's reference coordinate system, thereby obtaining the corresponding second tooth tip spatial positions.

[0092] Wherein, step 150 includes:

[0093] Step 152: Obtain a second ground clearance of each tooth tip based on the effective scene three-dimensional data and the second tooth tip spatial position.

[0094] In step 152, coordinate conversion is performed to determine the height of each tooth tip relative to the ground in the current working area, i.e., the second ground clearance of each tooth tip. This second ground clearance allows for more accurate and precise determination of the bucket's position within the current working area, enabling more precise excavation. For example, in uneven working conditions, controlling the second ground clearance of all tooth tips to be less than or equal to zero ensures that all bucket tips can dig into the ground, ensuring that the entire bucket is fully engaged in excavation, further improving operational efficiency.

[0095] In one embodiment, if Figure 8 As shown, after step 110, the excavator assisted perception method further includes:

[0096] Step 180: Detect whether the data points in the three-dimensional scene data are continuous with each other.

[0097] If the judgment result of step 180 is no, that is, there are discontinuous unknown points in the three-dimensional data of the scene, step 190 is executed to infer the data point corresponding to the unknown point based on the data points adjacent to the unknown point.

[0098] In this embodiment, when there are discontinuous unknown points in the scene three-dimensional data, it means that there are places that have not been scanned. At this time, the data of the unknown point is obtained by extrapolating the adjacent data points. Specifically, the extrapolation can be performed by interpolation calculation. For example, through the scene three-dimensional data, the coordinate values ​​of the left and right adjacent points of the unknown point in the X-axis direction are obtained respectively: X1, Y1, Z1, X2, Y2, Z2, and then half of the sum of X1 and X2 is used as the X coordinate value of the unknown point, half of the sum of Y1 and Y2 is used as the Y coordinate value of the unknown point, and half of the sum of Z1 and Z2 is used as the Z coordinate value of the unknown point.

[0099] Exemplary excavator auxiliary perception device

[0100] The present application also provides an auxiliary sensing device for an excavator, which is applied to an excavator. The excavator includes a fixed body and a movable body, and the movable body includes a boom, a dipper arm, and a bucket. Figure 9 As shown, the excavator auxiliary perception device includes: a scene data acquisition module 901, an excavator parameter acquisition module 902 and a bucket position calculation module 903.

[0101] The scene data acquisition module 901 is configured to: obtain scene 3D data of the current working area of ​​the excavator obtained by scanning with the fixed vehicle body as the reference system; and crop the scene 3D data to obtain valid scene 3D data within a preset area.

[0102] The excavator parameter acquisition module 902 is configured to obtain the current posture data and structural parameters of the excavator.

[0103] The bucket position calculation module 903 is communicated with the scene data acquisition module 901 and the excavator parameter acquisition module 902 respectively. The bucket position calculation module 903 is configured to: determine the first spatial position of the bucket in the reference system of the fixed vehicle body in the current state based on the current posture data and structural parameters; and obtain the second spatial position of the bucket relative to the current working area based on the effective scene three-dimensional data and the first spatial position.

[0104] Exemplary excavator

[0105] The present application also provides an excavator, which includes a fixed body, a movable body, a scene data detection module and the aforementioned excavator auxiliary sensing device. The movable body includes a boom, a dipper arm and a bucket. The scene data detection module is set on the fixed body, and the scene data detection module is configured to: use the fixed body as a reference system to scan the three-dimensional scene data of the current working area where the excavator is located. The scene data acquisition module can be various types of detectors such as laser radar or depth sensing camera device. The scene data acquisition module is communicatively connected to the scene data detection module. Figure 10As shown, the scene data detection module 101 is arranged on the fixed vehicle body 100 of the excavator, and the detection direction of the scene data detection module 101 is obliquely downward, so as to detect the three-dimensional scene data of the ground.

[0106] The basic principles of the present application have been described above in conjunction with specific embodiments. However, it should be noted that the advantages, strengths, and effects mentioned in this application are merely illustrative and not restrictive, and it should not be assumed that these advantages, strengths, and effects are required of each embodiment of this application. In addition, the specific details disclosed above are merely illustrative and facilitating understanding, and are not restrictive. The above details do not limit this application to necessarily being implemented using the above specific details.

[0107] The block diagrams of the devices, devices, equipment, and systems involved in this application are merely illustrative examples and are not intended to require or imply that they must be connected, arranged, or configured in the manner shown in the block diagrams. As will be appreciated by those skilled in the art, these devices, devices, equipment, and systems can be connected, arranged, or configured in any manner. Words such as "include," "comprise," "have," and the like are open-ended words, meaning "including but not limited to," and can be used interchangeably therewith. The words "or" and "and" used herein refer to the words "and / or" and can be used interchangeably therewith, unless the context clearly indicates otherwise. The word "such as" used herein refers to the phrase "such as but not limited to," and can be used interchangeably therewith.

[0108] It should also be noted that in the apparatus, device, and method of the present application, each component or each step can be decomposed and / or recombined, and such decomposition and / or recombination should be regarded as equivalent solutions of the present application.

[0109] The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use the present application. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein may be applied to other aspects without departing from the scope of the present application. Therefore, the present application is not intended to be limited to the aspects shown herein, but rather to be applied in the widest sense consistent with the principles and novel features of the present invention.

[0110] The above description is only a preferred embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent replacements, etc. made within the spirit and principles of the present application shall be included in the scope of protection of the present application.

Claims

1. An excavator auxiliary perception method, applied to an excavator, the excavator comprising a fixed body and a movable body, the movable body comprising a boom, a dipper arm, and a bucket; characterized in that: The excavator auxiliary perception method includes: Acquire three-dimensional scene data of the current working area of ​​the excavator obtained by scanning with the fixed vehicle body as a reference system; Acquire current control instruction data; obtain a vertical working area of ​​the bucket in a vertical direction according to the current control instruction data; Clipping the scene three-dimensional data to obtain valid scene three-dimensional data within a preset area, including: obtaining a clipping area on the ground of the current working area according to the vertical working area; and clipping the scene three-dimensional data according to the clipping area to obtain the valid scene three-dimensional data; Acquiring current posture data and structural parameters of the excavator; Determining a first spatial position of the bucket in a reference system of the fixed vehicle body in a current state according to the current posture data and the structural parameters; and The second spatial position of the bucket relative to the current working area is determined based on the effective scene three-dimensional data and the first spatial position.

2. The excavator-assisted perception method according to claim 1, characterized in that: The fixed vehicle body is provided with a laser radar; The three-dimensional scene data of the current working area of ​​the excavator obtained by scanning with the fixed vehicle body as a reference system includes: Acquire first radar point cloud data of the ground of the current operation area detected by the laser radar, wherein the first radar point cloud data is based on the laser radar as a reference point; Acquiring structural parameters of the excavator; determining a third spatial position of a hinge point between the movable arm and the fixed vehicle body on the fixed vehicle body according to the structural parameters; Obtaining a fourth spatial position of the laser radar on the fixed vehicle body; and According to the third spatial position and the fourth spatial position, the first radar point cloud data is converted into second radar point cloud data with the hinge point as a reference point, and the second radar point cloud data is used as the scene three-dimensional data.

3. The excavator-assisted perception method according to claim 1, characterized in that: The fixed vehicle body is provided with a depth sensing camera device; The three-dimensional scene data of the current working area of ​​the excavator obtained by scanning with the fixed vehicle body as a reference system includes: Acquiring first depth image data of the ground of the current working area detected by the depth-sensing camera device, wherein the first depth image data is based on the depth-sensing camera device as a reference point; Acquiring structural parameters of the excavator; determining a third spatial position of a hinge point between the movable arm and the fixed vehicle body on the fixed vehicle body according to the structural parameters; Acquiring a fifth spatial position of the depth-sensing camera device on the fixed vehicle body; and According to the third spatial position and the fifth spatial position, the first depth image data is converted into second depth image data with the hinge point as a reference point, and the second depth image data is used as the scene three-dimensional data.

4. The excavator-assisted perception method according to claim 1, characterized in that: The obtaining of the current posture data and structural parameters of the excavator includes: Acquiring first posture data and first structural data of the movable arm; Acquiring second posture data and second structure data of the arm; acquiring third posture data and third structure data of the bucket; and acquiring fourth structural data of the fixed vehicle body; Wherein, determining the first spatial position of the bucket in the reference system of the fixed vehicle body in the current state according to the current posture data and the structural parameters includes: The first spatial position is determined according to the first structural data, the second structural data, the third structural data, the fourth structural data, the first posture data, the second posture data, and the third posture data.

5. The excavator-assisted perception method according to claim 4, characterized in that: The step of determining the first spatial position of the bucket in the reference system of the fixed vehicle body in the current state according to the current posture data and the structural parameters further comprises: Determining a spatial position of a first tooth tip of a center tooth tip on the bucket in a reference system of the fixed vehicle body based on the first structural data, the second structural data, the third structural data, the fourth structural data, the first posture data, the second posture data, and the third posture data; Wherein, determining the second spatial position of the bucket relative to the current working area based on the effective scene three-dimensional data and the first spatial position includes: A first ground clearance height of the center tooth tip is obtained based on the effective scene three-dimensional data and the first tooth tip spatial position.

6. The excavator-assisted perception method according to claim 4, characterized in that: The step of determining the first spatial position of the bucket in the reference system of the fixed vehicle body in the current state according to the current posture data and the structural parameters further comprises: Determining the spatial position of the second tooth tip of each tooth tip on the bucket in the reference system of the fixed vehicle body according to the first structural data, the second structural data, the third structural data, the fourth structural data, the first posture data, the second posture data, and the third posture data; Wherein, determining the second spatial position of the bucket relative to the current working area based on the effective scene three-dimensional data and the first spatial position includes: According to the effective scene three-dimensional data and the second tooth tip spatial position, the second ground clearance height of each tooth tip is obtained respectively.

7. The excavator-assisted perception method according to any one of claims 1 to 6, characterized in that: After acquiring the three-dimensional scene data of the current working area of ​​the excavator obtained by scanning with the fixed vehicle body as a reference system, the method further includes: Detecting whether data points in the three-dimensional scene data are continuous with each other; and If there are discontinuous unknown points in the three-dimensional data of the scene, the data points corresponding to the unknown points are inferred based on the data points adjacent to the unknown points.

8. An auxiliary sensing device for an excavator, applied to an excavator, wherein the excavator comprises a fixed body and a movable body, wherein the movable body comprises a boom, a dipper arm, and a bucket; characterized in that: The excavator auxiliary sensing device includes: The scene data acquisition module is configured to: acquire scene three-dimensional data of the current working area of ​​the excavator obtained by scanning with the fixed vehicle body as a reference system; and crop the scene three-dimensional data to obtain valid scene three-dimensional data within a preset area; An excavator parameter acquisition module is configured to: acquire current posture data and structural parameters of the excavator; and a bucket position calculation module, communicatively connected to the scene data acquisition module and the excavator parameter acquisition module, respectively, the bucket position calculation module being configured to: determine a first spatial position of the bucket in a current state within a reference frame of the fixed vehicle body based on the current posture data and the structural parameters; and obtain a second spatial position of the bucket relative to the current working area based on the effective scene three-dimensional data and the first spatial position; The bucket position calculation module is also used to obtain current control instruction data; obtain the vertical working area of ​​the bucket in the vertical direction according to the current control instruction data; determine the first spatial position of the bucket in the reference system of the fixed vehicle body in the current state according to the current posture data and the structural parameters, including: obtaining a clipping area on the ground of the current working area according to the vertical working area; and clipping the scene three-dimensional data according to the clipping area to obtain the effective scene three-dimensional data.

9. An excavator comprising a fixed body and a movable body, wherein the movable body comprises a boom, an arm and a bucket; The excavator further comprises: A scene data detection module is provided on the fixed vehicle body, and the scene data detection module is configured to: scan the three-dimensional scene data of the current working area where the excavator is located with the fixed vehicle body as a reference system; and According to the excavator auxiliary perception device as described in claim 8, the scene data acquisition module is communicatively connected to the scene data detection module.

Images