Tree branch pruning tool, tree branch pruning method, device and medium

Abstract

The present invention provides a tree branch pruning tool, a tree branch pruning method, a device and a medium, which relate to the field of robotics technology. The tool obtains image data of branches and leaves to be pruned; performs feature extraction and three-dimensional reconstruction based on the image data to obtain a three-dimensional point cloud image of the branches and leaves to be pruned; uses a path planning model to predict a pruning path based on the three-dimensional point cloud image of the branches and leaves to be pruned; controls an actuator to move the actuator to a working position based on the predicted pruning path, and clamps the branches and leaves to be pruned through a chain saw assembly and a support in the actuator; controls the chain saw assembly to perform a pruning action based on a pruning instruction, and controls the chain saw assembly to stop the action after a second limit switch is triggered; and controls the chain saw assembly to rotate, and stops rotating after a first limit switch is triggered, so as to realize fully automatic pruning of tree branches and leaves, thereby improving work efficiency and reducing labor costs.

Description

Technical Field

[0001] The present invention relates to the field of robotics, and in particular to a tree branch pruning tool, a tree branch pruning method, a device and a medium. Background Art

[0002] When pruning branches on both sides of urban roads, due to the high height and complex distribution of branches, as well as the proximity to live lines, workers are usually required to use a lift with a hand saw or an oil saw to manually prune the branches under proper insulation protection, resulting in high labor costs and low efficiency. In particular, serious safety hazards are prone to occur during the pruning of larger branches. Summary of the Invention

[0003] In view of this, an object of the present invention is to provide a tree branch pruning tool, a tree branch pruning method, a device and a medium to improve work efficiency and reduce labor costs.

[0004] In a first aspect, the present application provides a tree branch pruning tool, comprising:

[0005] The actuator includes an image sensor assembly, a limit switch assembly, a chain saw assembly, a support member, and a bracket assembly; the chain saw assembly is rotatably connected to the bracket assembly, the support member is fixedly connected to the bracket assembly, and a clamping space is formed between the chain saw assembly and the support member, and the clamping space is used to clamp the branches and leaves to be pruned; the limit switch assembly includes a first limit switch and a second limit switch, the first limit switch and the second limit switch are respectively located on both sides of the chain saw assembly and are fixedly connected to the bracket assembly, and the limit switch assembly is used to limit the rotation angle of the chain saw assembly; the image sensor assembly is fixedly connected to the bracket assembly, and is used to collect image data of the branches and leaves to be pruned;

[0006] A control device, the control device is connected to the image sensor component, the limit switch component, and the chain saw component respectively; the control device is used to: obtain image data of branches and leaves to be pruned; perform feature extraction and three-dimensional reconstruction based on the image data to obtain a three-dimensional point cloud image of the branches and leaves to be pruned; use a path planning model to predict a pruning path based on the three-dimensional point cloud image of the branches and leaves to be pruned; based on the predicted pruning path, control the action of the actuator to move the actuator to the working position, and clamp the branches and leaves to be pruned through the chain saw component and the support in the actuator; based on the pruning instruction, control the chain saw component to perform the pruning action until the second limit switch is triggered, and then control the chain saw component to stop the action; control the chain saw component to rotate until the first limit switch is triggered and then stops rotating.

[0007] Optionally, the image sensor assembly includes a first image sensor and a second image sensor, the first image sensor and the second image sensor are respectively located on both sides of the chain saw assembly, and the control device is respectively connected to the first image sensor and the second image sensor; the control device is further configured to:

[0008] Acquire first image data captured by the first image sensor and a second feature image captured by the second image sensor;

[0009] Performing feature extraction based on the first image data and the second feature image to obtain a first feature image corresponding to the first image data and a second feature data corresponding to the second feature image;

[0010] Performing three-dimensional reconstruction based on the first characteristic image and the second characteristic image to obtain a first reconstructed image corresponding to the first characteristic image and a second reconstructed image corresponding to the second characteristic image;

[0011] Matching processing is performed based on the first reconstructed image and the second reconstructed image to obtain a three-dimensional point cloud image of the branches and leaves to be pruned.

[0012] Optionally, the actuator further includes a shock absorbing assembly, which is disposed at one end of the bracket away from the chain saw assembly and connected to the bracket assembly.

[0013] Optionally, the actuator further includes a connecting assembly, which is used to mount the actuator on the robotic arm. The connecting assembly includes a connecting column and a connecting piece, and the connecting piece is fixedly connected to an end of the bracket away from the chain saw assembly through the connecting column.

[0014] Optionally, the control device is further configured to: predict a pruning path using a path planning model based on a three-dimensional point cloud image of the branches and leaves to be pruned, including:

[0015] Based on the 3D point cloud image of the branches and leaves to be pruned, an improved RRT model is used to generate a 3D point cloud image pruning path. The improved RRT model obtains the current detection step size based on the initial detection step size and the detection step size scale coefficient, and updates the current detection step size based on the current detection step size and the path boundary distance value.

[0016] Based on the determined mapping relationship between the robot arm coordinate system and the image sensor component coordinate system, a predicted pruning path corresponding to the pruning path of the three-dimensional point cloud image is determined; wherein the predicted pruning path is the target path of the actuator operation.

[0017] Optionally, the actuator further includes a laser radar, which is provided in the chain saw assembly and is used to obtain contour data of the branches and leaves to be pruned and environmental data of the branches and leaves to be pruned;

[0018] The control device is connected to the laser radar and is also used to:

[0019] Generate three-dimensional point cloud data based on the contour data of the branches and leaves to be pruned and the environmental data of the branches and leaves to be pruned;

[0020] Based on the mapping relationship between the image sensor component coordinate system and the 3D point cloud data coordinate system, RGB coloring is performed on the 3D point cloud image of the pruned branches and leaves.

[0021] Optionally, the control device is further configured to: forward a pruning instruction to a user client when the actuator moves to the working position, so that the user client makes a pruning determination based on a positional relationship between the actuator in the pruning instruction and the three-dimensional point cloud image of the branches and leaves to be pruned;

[0022] When receiving the confirmation trimming instruction returned by the user client, the chain saw component is controlled to perform the trimming action.

[0023] In a second aspect, the present application provides a tree branch pruning method, which is applied to the control device in the above-mentioned tree branch pruning tool, and the method includes:

[0024] Obtain image data of branches and leaves to be pruned;

[0025] Perform feature extraction and 3D reconstruction based on image data to obtain a 3D point cloud image of the branches and leaves to be pruned;

[0026] Based on the 3D point cloud image of the branches and leaves to be pruned, a path planning model is used to predict the pruning path;

[0027] Based on the predicted pruning path, the actuator is controlled to clamp the branches and leaves to be pruned through the chain saw assembly and the support member;

[0028] Based on the trimming instruction, the chain saw assembly is controlled to perform the trimming action until the second limit switch is triggered, and the chain saw assembly is controlled to stop the action;

[0029] The chain saw assembly is controlled to rotate until the first limit switch is triggered and the chain saw assembly stops rotating.

[0030] In a third aspect, the present application provides a control device comprising a processor and a memory, wherein the memory stores computer-executable instructions that can be executed by the processor, and the processor executes the computer-executable instructions to implement the above-mentioned tree branch pruning method.

[0031] In a fourth aspect, the present application provides a computer-readable storage medium, which stores computer-executable instructions. When the computer-executable instructions are called and executed by a processor, the above-mentioned tree branch pruning method is implemented.

[0032] The present invention provides a tree branch pruning tool, tree branch pruning method, device and medium, which obtain image data of branches and leaves to be pruned; perform feature extraction and three-dimensional reconstruction based on the image data to obtain a three-dimensional point cloud image of the branches and leaves to be pruned; use a path planning model to predict a pruning path based on the three-dimensional point cloud image of the branches and leaves to be pruned; based on the predicted pruning path, control the action of an actuator to move the actuator to an operating position, and clamp the branches and leaves to be pruned through a chain saw assembly and a support in the actuator; based on a pruning instruction, control the chain saw assembly to perform a pruning action until a second limit switch is triggered, and then control the chain saw assembly to stop; control the chain saw assembly to rotate until a first limit switch is triggered and then stops rotating, so as to realize fully automatic pruning of tree branches and leaves, thereby improving work efficiency and reducing labor costs.

[0033] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the preferred embodiments are specifically listed below and described in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the embodiments. It should be understood that the following drawings only illustrate certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other relevant drawings can be obtained based on these drawings without paying any creative work.

[0035] Figure 1 A schematic structural diagram of a tree branch and leaf pruning tool provided by an embodiment of the present invention is shown;

[0036] Figure 2 A schematic structural diagram of a chain saw assembly provided by an embodiment of the present invention is shown;

[0037] Figure 3 A schematic structural diagram showing a cross-sectional view of a chain saw assembly provided by an embodiment of the present invention is shown;

[0038] Figure 4 A schematic structural diagram of another tree branch and leaf pruning tool provided by an embodiment of the present invention is shown;

[0039] Figure 5 A schematic structural diagram showing a chain saw assembly provided by an embodiment of the present invention installed on a bracket assembly;

[0040] Figure 6 A schematic diagram showing a cutting member and a supporting member clamping branches and leaves to be pruned provided by an embodiment of the present invention is shown;

[0041] Figure 7 A schematic structural diagram of a shock absorbing assembly and a connecting assembly provided by an embodiment of the present invention is shown;

[0042] Figure 8 A schematic diagram showing a flow chart of a tree branch and leaf pruning method provided by an embodiment of the present invention;

[0043] Figure 9 A schematic structural diagram of a control device provided by an embodiment of the present invention is shown.

[0044] Explanation of the main component symbols: 100-chain saw assembly; 110-cutting piece; 120-rotating gear; 121-limiting bolt; 131-rotating shaft; 132-rotating sleeve; 133-positioning end cover; 134-connecting bolt; 135-limiting washer; 200-support member; 210-rotating support frame; 300-bracket assembly; 310-rotating fixing plate; 320-rotating motor; 330-driving gear; 340-gear baffle; 350-shock absorber assembly; 360-connecting assembly; 400-image sensor assembly; 510-first limit switch; 511-first adjustment bracket; 520-second limit switch; 521-second adjustment bracket. DETAILED DESCRIPTION

[0045] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. The components of the embodiments of the present invention generally described and shown in the drawings herein can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present invention provided in the drawings is not intended to limit the scope of the claimed invention, but merely represents selected embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without making creative work are within the scope of protection of the present invention.

[0046] The following describes a tree branch pruning tool provided by an embodiment of the present application. Figure 1As shown, the tree branch pruning tool provided by the embodiment of the present application includes at least: an actuator and a control device; wherein the actuator includes an image sensor assembly 400, a limit switch assembly, a chain saw assembly 100, a support member 200 and a bracket assembly 300, the chain saw assembly 100 is rotatably connected to the bracket assembly 300, the support member 200 is fixedly connected to the bracket assembly 300, and a clamping space is formed between the chain saw assembly 100 and the support member 200, and the clamping space is used to clamp the branches to be pruned; the limit switch assembly includes a first limit switch 510 and a second limit switch 520, the first limit switch 510 and the second limit switch 520 are respectively located on both sides of the chain saw assembly 100, and are fixedly connected to the bracket assembly 300, and the limit switch assembly is used to limit the rotation angle of the chain saw assembly 100; the image sensor assembly 400 is fixedly connected to the bracket assembly 300, and is used to collect image data of the branches to be pruned.

[0047] A control device, the control device is connected to the image sensor assembly 400, the limit switch assembly, and the chain saw assembly 100 respectively; the control device is used to: obtain image data of the branches and leaves to be pruned; perform feature extraction and three-dimensional reconstruction based on the image data to obtain a three-dimensional point cloud image of the branches and leaves to be pruned; use a path planning model to predict a pruning path based on the three-dimensional point cloud image of the branches and leaves to be pruned; based on the predicted pruning path, control the action of the actuator to move the actuator to the working position, and clamp the branches and leaves to be pruned through the chain saw assembly 100 and the support 200 in the actuator; based on the pruning instruction, control the chain saw assembly 100 to perform the pruning action until the second limit switch 520 is triggered, and then control the chain saw assembly 100 to stop; control the chain saw assembly 100 to rotate until the first limit switch 510 is triggered and then stops.

[0048] The tree branch pruning tool provided in the embodiment of the present application collects image data of the branches and leaves to be pruned through an image sensor component, and transmits the image data of the branches and leaves to be pruned to a control device. The control device performs feature extraction based on the acquired image data of the branches and leaves to be pruned to obtain feature data, performs coordinate calculation and three-dimensional reconstruction based on the feature data, and obtains a three-dimensional point cloud image of the branches and leaves to be pruned. Based on the three-dimensional point cloud image of the branches and leaves to be pruned, a path planning model is used to predict a pruning path; based on the predicted pruning path, the actuator is controlled to move the actuator to an operating position, and the branches and leaves to be pruned are clamped by the chain saw component and the support in the actuator; based on the pruning instruction, the chain saw component is controlled to perform the pruning action until the second limit switch is triggered, and the chain saw component is controlled to stop moving; the chain saw component is controlled to rotate until the first limit switch is triggered and stops rotating, and the actuator is repeatedly controlled to move the actuator to a new operating position based on the predicted pruning path until all the branches and leaves to be pruned are pruned, so as to realize fully automatic pruning of tree branches and leaves, thereby improving work efficiency and reducing labor costs.

[0049] In the embodiment of the present application, the specific structure of the chain saw assembly 100 is as follows: Figure 2 and Figure 3 As shown, the chain saw assembly 100 includes a cutting piece 110, a rotating gear 120 and a positioning assembly, wherein the positioning assembly includes a rotating shaft 131, a rotating shaft sleeve 132, a limiting washer 135, a positioning end cover 133 and a connecting bolt 134; one end of the rotating shaft 131 is fixedly connected to the cutting piece 110, and the rotating shaft sleeve 132 and the limiting washer 135 are sequentially sleeved on the circumference of the rotating shaft 131, and the positioning end cover 133 is fixedly connected to the end of the rotating shaft 131 away from the cutting piece 110 through the connecting bolt 134.

[0050] Furthermore, one end of the rotating gear 120 is fixedly connected to the preset positioning hole of the cutting member 110 to achieve precise centering of the rotating pair, and a limit bolt 121 is provided on the rotating gear 120. The limit bolt 121 is fixedly connected to the threaded positioning hole of the rotating gear 120. The limit bolt 121 is also used to touch the second limit switch 520 to stop the chain saw assembly 100 from rotating while completing the trimming of any branch or leaf.

[0051] In the embodiment of the present application, the specific structure of the bracket assembly 300 is as follows: Figure 4 As shown, the bracket assembly 300 includes a support frame, a rotating motor 320, a driving gear 330 and a gear baffle 340, wherein the rotating motor 320 is fixedly connected to the support frame, the output shaft of the rotating motor 320 is rotationally connected to the driving gear 330, and the gear baffle 340 is arranged between the rotating motor 320 and the driving gear 330, and is fixedly connected to the support frame.

[0052] Furthermore, the chain saw assembly 100 is arranged on the bracket assembly 300, and is rotatably connected to the bracket assembly 300 by engaging the rotating gear 120 with the driving gear 330 of the bracket assembly 300; the support member 200 is arranged on the bracket assembly 300 and is arranged opposite to the chain saw assembly 100, and the support member 200 is fixedly connected to the bracket assembly 300 by rotating the support frame 210, and a certain angle is formed between the support member 200 and the cutting member 110 of the chain saw assembly 100 for supporting the branches and leaves to be pruned, such as Figure 6 shown.

[0053] Further, if Figure 5 As shown, when the chain saw assembly 100 is fixedly connected to the rotating fixed plate 310 of the bracket assembly 300 through the positioning assembly, the connecting bolt 134 of the positioning assembly can be loosened to allow the limiting washer 135 to have a certain amount of movable margin, and the rotating shaft sleeve 132 is placed in the arc surface on the rotating fixed plate 310 to ensure that the rotating fixed plate 310 is located between the rotating shaft sleeve 132 and the limiting washer 135, and then the connecting bolt 134 is tightened to fix the chain saw assembly 100 on the bracket assembly 300.

[0054] In the embodiment of the present application, the chain saw assembly 100 is fixedly connected to the rotating fixed plate 310 of the bracket assembly 300 through the positioning assembly, and is connected to the driving gear 330 through the rotating gear 120, so that the rotating motor 320 drives the driving gear 330 to rotate, and the driving gear 330 drives the rotating gear 120 to rotate, so as to realize the rotational movement of the cutting member 110, and realize the formation of a gear transmission mechanism with dual positioning guarantees through the limit bolt 121 and the positioning end cover 133.

[0055] In the embodiment of the present application, the limit assembly includes a first limit switch 510 and a second limit switch 520. Figure 4 As shown, the second limit switch 520 is fixedly connected to the bracket assembly 300 through the second adjustment bracket 521. When the cutting member 110 moves close to the support member 200 and cuts off the branches to be pruned, the rotating gear 120 of the chain saw assembly 100 contacts the second limit switch 520; Figure 5 As shown, the first limit switch 510 is fixedly connected to the bracket assembly 300 through the first adjustment bracket 511. The first limit switch 510 and the second limit switch 520 are respectively located on both sides of the chain saw assembly 100. When the cutting member 110 moves away from the support member 200 and the angle between the support member 200 and the cutting member 110 reaches a preset angle, the rotating gear 120 of the chain saw assembly 100 contacts the first limit switch 510.

[0056] It should be noted that when the second limit switch 520 is triggered, it indicates that the actuator completes the trimming action; when the first limit switch 510 is triggered, it indicates that the cutting member 110 of the chain saw assembly 100 is in the initial position, that is, the subsequent trimming action can be performed.

[0057] In the embodiments of this application, Figure 7 As shown, the bracket assembly 300 also includes a shock absorbing assembly 350 and a connecting assembly 360, wherein the shock absorbing assembly 350 is arranged at the end of the bracket away from the chain saw assembly 100 and is connected to the bracket assembly 300; the connecting assembly 360 includes a connecting column and a connecting member, and the connecting member is fixedly connected to the end of the bracket away from the chain saw assembly 100 through the connecting column. The connecting assembly 360 is used to install the actuator on the robotic arm.

[0058] A shock absorbing assembly 350 is provided at the end of the connection between the bracket assembly 300 and the robotic arm, which can ensure that the torque generated by the chain saw assembly 100 during the process of trimming branches and leaves will not be transmitted to the robotic arm, thereby ensuring that the robotic arm is not affected by the torque during the process of trimming branches and leaves.

[0059] Furthermore, the pressure-bearing surface of the shock-absorbing assembly 350 is a conical contact surface that matches the three-dimensional contour of the bottom surface of the bracket assembly 300 and the inner cavity of the connecting column, and the prestressed fit in the assembly state is ensured through the vulcanization molding process; in the radial peripheral area of ​​the shock-absorbing assembly 350, based on the distribution characteristics of the stress cloud map, a bionic wavy shock-absorbing groove array is topologically optimized, and the shock-absorbing grooves extend at a 45° staggered angle to form a gradient energy dissipation structure. The conical contact surface of the shock-absorbing assembly 350 and the shock-absorbing grooves constitute a coordinated shock-absorbing system of axial constraint and radial energy dissipation. Finite element verification has shown that the vibration transmission rate of the actuator end can be reduced by 40%.

[0060] In an embodiment of the present application, the image sensor assembly 400 includes a first image sensor and a second image sensor, which are respectively located on both sides of the chain saw assembly 100. The first image sensor and the second image sensor respectively collect the first image data and the second characteristic image of the branches and leaves to be pruned, and the first image data and the second characteristic image constitute a binocular image.

[0061] Furthermore, in the embodiment of the present application, the first image sensor and the second image sensor may be a wide-angle camera or a high-definition camera, etc.

[0062] In an embodiment of the present application, the actuator further includes a laser radar, a laser ranging unit, and an electric field sensor, wherein the laser radar and the laser ranging unit are both provided in the chain saw assembly 100, for obtaining contour data of the branches and leaves to be pruned and environmental data of the branches and leaves to be pruned; the laser ranging unit adopts time-of-flight (TOF) technology to scan the working area (i.e., the area of ​​branches and leaves to be pruned) at a frequency of 200Hz to generate three-dimensional point cloud data; the electric field sensor is embedded in the actuator, for detecting the electric field intensity gradient generated by the high-voltage line, and dynamically calibrating the safe working boundary.

[0063] In an embodiment of the present application, the control device is connected to the image sensor assembly 400, the limit switch assembly, and the chain saw assembly 100, respectively; the control device is used to: obtain image data of the branches and leaves to be pruned; perform feature extraction and three-dimensional reconstruction based on the image data to obtain a three-dimensional point cloud image of the branches and leaves to be pruned; based on the three-dimensional point cloud image of the branches and leaves to be pruned, use a path planning model to predict the pruning path; based on the predicted pruning path, control the actuator to move the actuator to the working position, and clamp the branches and leaves to be pruned through the chain saw assembly 100 and the support 200 in the actuator; based on the pruning instruction, control the chain saw assembly 100 to perform the pruning action until the second limit switch 520 is triggered, and then control the chain saw assembly 100 to stop; control the chain saw assembly 100 to rotate until the first limit switch 510 is triggered and then stops rotating.

[0064] Furthermore, the control device is also connected to the first image sensor and the second image sensor respectively; the control device is also used to: obtain the first image data collected by the first image sensor and the second feature image collected by the second image sensor; perform feature extraction based on the first image data and the second feature image to obtain the first feature image corresponding to the first image data and the second feature data corresponding to the second feature image; perform three-dimensional reconstruction based on the first feature image and the second feature image to obtain a first reconstructed image corresponding to the first feature image and a second reconstructed image corresponding to the second feature image; perform matching processing based on the first reconstructed image and the second reconstructed image to obtain a three-dimensional point cloud image of the branches and leaves to be pruned.

[0065] Furthermore, the control device is also used to: predict the pruning path based on the three-dimensional point cloud image of the branches and leaves to be pruned using a path planning model, including: generating a three-dimensional point cloud image pruning path based on the three-dimensional point cloud image of the branches and leaves to be pruned using an improved RRT model; wherein, the improved RRT model is based on the initial detection step size and the detection step size proportional coefficient to obtain the current detection step size, and based on the current detection step size and the path boundary distance value, the current detection step size is updated; based on the determined mapping relationship between the robotic arm coordinate system and the image sensor component 400 coordinate system, the predicted pruning path corresponding to the three-dimensional point cloud image pruning path is determined; wherein, the predicted pruning path is the target path for the actuator operation.

[0066] Furthermore, the control device is also connected to the laser radar, and the control device is also used to: generate three-dimensional point cloud data based on the contour data of the branches and leaves to be pruned and the environmental data of the branches and leaves to be pruned; and perform RGB coloring on the three-dimensional point cloud image of the pruned branches and leaves based on the mapping relationship between the coordinate system of the image sensor component 400 and the coordinate system of the three-dimensional point cloud data.

[0067] Furthermore, the control device is also used to: forward the pruning instruction to the user client when the actuator moves to the working position, so that the user client can make pruning determination based on the positional relationship between the actuator in the pruning instruction and the three-dimensional point cloud image of the branches and leaves to be pruned; and control the chain saw assembly 100 to perform the pruning action when receiving the confirmation pruning instruction returned by the user client.

[0068] The following describes the specific process of pruning tree branches and leaves using a tree branch pruning tool provided in an embodiment of the present application:

[0069] Step 1: Acquire image data of branches and leaves to be pruned, i.e., first image data and second feature image;

[0070] Step 2: Using a deep learning convolutional neural network (such as yolov11), first perform feature recognition on the first image data and the second feature image to obtain a first recognition feature and a second recognition feature. Then, feature extraction is performed on the first recognition feature and the second recognition feature to obtain a first feature image and a second feature data. Then, using an extreme matching method, binocular matching is performed on the first feature image and the second feature data to obtain matching features. The matching features are then coordinate calculated and three-dimensionally reconstructed to obtain a three-dimensional point cloud image of the branches and leaves to be pruned. The recognition features and feature data include but are not limited to lines, leads, crossarms, lightning arresters, and other lines and trees in the operation scene.

[0071] Step 3: Through the mapping relationship between camera coordinates and lidar coordinates (i.e., the mapping relationship between the image sensor component coordinate system and the 3D point cloud data coordinate system), the colorless lidar point cloud is RGB-colored, fusing depth and color information so that the 3D point cloud image of the branches to be pruned has texture and color information, thereby improving the readability and scene understanding ability of the 3D point cloud image, as well as improving the integrity of the 3D point cloud image;

[0072] Step 4: Based on the colored three-dimensional point cloud image of the branches and leaves to be pruned, the global search capability of the improved RRT is used to generate a pruning path; wherein, the improved RRT is based on the initial detection step and the detection step ratio coefficient to obtain the current detection step. For example, if the preset detection step ratio coefficient is 1, if the current detection step exceeds the distance value of the path boundary, the detection step ratio coefficient is updated, that is, the detection step ratio coefficient is updated to 0.8, the current detection step is regenerated, and detection is performed; if the current detection step is less than the distance value of the path boundary, the detection step ratio coefficient is updated, that is, the detection step sharpness coefficient is updated to 1.2, the detection step is regenerated, and detection is performed. By dynamically adjusting the detection step ratio coefficient, this process is repeated to complete the path detection convergence. Further, if the current detection step is less than the distance value of the path boundary, the detection step ratio coefficient is updated, that is, the detection step sharpness coefficient is updated to 1.2, the detection step is regenerated, and detection is performed. During the generation process, if the current detection step length is less than the distance value of the path boundary, the detection step length proportional coefficient assigned to the current detection step length can be increased by 1; if the current detection step length is equal to the distance value of the path boundary, the detection step length proportional coefficient assigned to the current detection step length remains unchanged; if the current detection step length is greater than the distance value of the path boundary, the detection step length proportional coefficient assigned to the current detection step length can be reduced by 1; the weights of each detection step length on the entire search path are summed, and the path with the largest total weight value is marked as the preferred path, thereby improving the global search efficiency; when generating the pruned path using the improved RRT global search capability, the path safety is optimized through the artificial potential field method, and the safety distance constraint function is introduced to calculate the distance from any position to the nearest obstacle in real time to ensure that the path meets the dynamic obstacle avoidance and static safety distance requirements;

[0073] Step 5: Based on the mapping relationship between the robotic arm coordinate system and the image sensor assembly coordinate system, a predicted pruning path corresponding to the pruning path in the 3D point cloud image is determined. The predicted pruning path is the target path for the actuator to operate. The end of the robotic arm is equipped with a six-dimensional force sensor. The force feedback is used to determine the contact state with the tree branches and leaves, ensuring that the robotic arm and the actuator can always cut the branches.

[0074] Step 6: Based on the predicted trimming path, the control device controls the actuator to move to the working position and forwards the trimming instruction to the user client. After receiving the trimming instruction, the user client detects whether the positional relationship between the 3D point cloud image and the actuator and the status of the actuator are correct. If correct, the trimming is confirmed. When the control device receives the trimming confirmation instruction returned by the user client, it controls the chain saw assembly to perform the trimming action.

[0075] Step 7: Based on the pruning instruction, the control device controls the chain saw assembly to perform the pruning action, that is, controls the rotary motor to start, thereby driving the driving gear, the rotary gear, and the chain saw assembly to rotate, so that the cutting member moves closer to the support member to clamp the branches and leaves to be pruned, and controls the rotary motor to continue to move until the second limit switch is triggered, and it is determined that the branches and leaves to be pruned have been pruned, and then controls the chain saw assembly to stop moving;

[0076] Step 8: Control the rotary motor to rotate in the reverse direction to move the cutting piece away from the supporting piece until the first limit switch is triggered and the cutting piece stops rotating;

[0077] Step 9: Repeat steps 6 to 8 above until you have completed pruning the tree branches and leaves.

[0078] The following describes a tree branch pruning method provided by an embodiment of the present application. Figure 8 As shown, the tree branch pruning method provided in the embodiment of the present application is applied to the control device of the above-mentioned tree branch pruning tool. The tree branch pruning method generally includes the following steps:

[0079] Step 510: Obtain image data of branches and leaves to be pruned;

[0080] Step 520: Perform feature extraction and 3D reconstruction based on the image data to obtain a 3D point cloud image of the branches and leaves to be pruned;

[0081] Step 530: Based on the three-dimensional point cloud image of the branches and leaves to be pruned, a path planning model is used to predict a pruning path;

[0082] Step 540: Based on the predicted pruning path, control the actuator to clamp the branches and leaves to be pruned through the chain saw assembly and the support member;

[0083] Step 550: Based on the trimming instruction, control the chain saw assembly to perform the trimming action until the second limit switch is triggered, and then control the chain saw assembly to stop the action;

[0084] Step 560: Control the chain saw assembly to rotate until the first limit switch is triggered and the chain saw assembly stops rotating.

[0085] In an optional embodiment, feature extraction and three-dimensional reconstruction are performed based on image data to obtain a three-dimensional point cloud image of the branches and leaves to be pruned, including: acquiring first image data collected by a first image sensor and a second feature image collected by a second image sensor; performing feature extraction based on the first image data and the second feature image to obtain a first feature image corresponding to the first image data and a second feature data corresponding to the second feature image; performing three-dimensional reconstruction based on the first feature image and the second feature image to obtain a first reconstructed image corresponding to the first feature image and a second reconstructed image corresponding to the second feature image; performing matching processing based on the first reconstructed image and the second reconstructed image to obtain a three-dimensional point cloud image of the branches and leaves to be pruned.

[0086] In an optional embodiment, a path planning model is used to predict a pruning path based on a three-dimensional point cloud image of the branches and leaves to be pruned, including: based on the three-dimensional point cloud image of the branches and leaves to be pruned, an improved RRT model is used to generate a three-dimensional point cloud image pruning path; wherein, the improved RRT model obtains the current detection step size based on the initial detection step size and the detection step size proportional coefficient, and updates the current detection step size based on the current detection step size and the path boundary distance value; based on the determined mapping relationship between the robotic arm coordinate system and the image sensor component coordinate system, a predicted pruning path corresponding to the three-dimensional point cloud image pruning path is determined; wherein, the predicted pruning path is the target path of the actuator operation.

[0087] In an optional embodiment, the method also includes: generating three-dimensional point cloud data based on the contour data of the branches and leaves to be pruned and the environmental data of the branches and leaves to be pruned; and performing RGB coloring on the three-dimensional point cloud image of the pruned branches and leaves based on the mapping relationship between the image sensor component coordinate system and the three-dimensional point cloud data coordinate system.

[0088] In an optional embodiment, the method further includes: when the actuator moves to the working position, forwarding the pruning instruction to the user client, so that the user client determines the pruning based on the positional relationship between the actuator in the pruning instruction and the three-dimensional point cloud image of the branches and leaves to be pruned; upon receiving the confirmation pruning instruction returned by the user client, controlling the chain saw assembly to perform the pruning action.

[0089] The method provided in the embodiment of the present invention has the same implementation principle and technical effects as those in the aforementioned tree branch and leaf pruning tool embodiment. For the sake of brief description, for matters not mentioned in the method embodiment, reference can be made to the corresponding content in the aforementioned tree branch and leaf pruning tool embodiment.

[0090] An embodiment of the present invention provides a control device, which includes a processor and a memory; the memory stores computer-executable instructions that can be executed by the processor, and the processor executes the computer-executable instructions to implement the above-mentioned tree branch and leaf pruning method.

[0091] Figure 9 A structural schematic diagram of a control device provided in an embodiment of the present invention, the control device 610 includes: a processor 610, a memory 620, a bus 630 and a communication interface 640, the processor 610, the communication interface 640 and the memory 620 are connected via the bus 630; the processor 610 is used to execute an executable module stored in the memory 620, such as a computer program.

[0092] Memory 620 may include high-speed random access memory (RAM) and may also include non-volatile memory, such as at least one disk storage device. Communication between the system network element and at least one other network element is achieved through at least one communication interface 640 (which may be wired or wireless), and may utilize the Internet, a wide area network, a local area network, a metropolitan area network, or the like.

[0093] The bus 630 may be an ISA bus, a PCI bus, or an EISA bus. The bus may be divided into an address bus, a data bus, a control bus, and the like. For ease of representation, Figure 9 Only one bidirectional arrow is used in the diagram, but this does not mean that there is only one bus or one type of bus.

[0094] Among them, the memory 620 is used to store programs, and the processor 610 executes the program after receiving the execution instruction. The method executed by the device for flow process definition disclosed in any embodiment of the aforementioned embodiment of the present invention can be applied to the processor 610 or implemented by the processor 610.

[0095] The processor 610 may be an integrated circuit chip with signal processing capabilities. During implementation, each step of the above method can be completed by hardware integrated logic circuits in the processor 610 or by software instructions. The processor 610 may be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it may also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the various methods, steps, and logic block diagrams disclosed in the embodiments of the present invention. The general-purpose processor may be a microprocessor or any conventional processor. The steps of the method disclosed in conjunction with the embodiments of the present invention can be directly implemented and executed by a hardware decoding processor, or by a combination of hardware and software modules in the decoding processor. The software module can be located in a storage medium known in the art, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, or registers. The storage medium is located in memory 620. Processor 610 reads information from memory 620 and, in conjunction with its hardware, completes the steps of the tree pruning method.

[0096] The computer program product of the readable storage medium provided in the embodiment of the present invention includes a computer-readable storage medium storing program code. The instructions included in the program code can be used to execute the method in the previous method embodiment. The specific implementation can be referred to the aforementioned tree branch and leaf pruning method embodiment, which will not be repeated here.

[0097] If a function is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the portion that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the various embodiments of the method of the present invention. The aforementioned storage medium includes various media that can store program code, such as USB flash drives, mobile hard drives, read-only memories (ROMs), random access memories (RAMs), magnetic disks, or optical disks.

[0098] Finally, it should be noted that the above embodiments are only specific implementation methods of the present invention, which are used to illustrate the technical solutions of the present invention, rather than to limit them. The scope of protection of the present invention is not limited thereto. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that any person skilled in the art can modify or easily conceive of changes to the technical solutions described in the above embodiments within the technical scope disclosed by the present invention, or replace some of the technical features therein with equivalents. Such modifications, changes or replacements do not deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should be included in the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be based on the scope of protection of the claims.

Claims

1. A tree branch pruning tool, characterized in that: include: The actuator includes an image sensor assembly, a limit switch assembly, a chain saw assembly, a support member, and a bracket assembly; the chain saw assembly is rotatably connected to the bracket assembly, the support member is fixedly connected to the bracket assembly, a clamping space is formed between the chain saw assembly and the support member, and the clamping space is used to clamp branches and leaves to be pruned; the limit switch assembly includes a first limit switch and a second limit switch, the first limit switch and the second limit switch are respectively located on both sides of the chain saw assembly and are fixedly connected to the bracket assembly, and the limit switch assembly is used to limit the rotation angle of the chain saw assembly; the image sensor assembly is fixedly connected to the bracket assembly, and is used to collect image data of branches and leaves to be pruned; A control device, the control device being connected to the image sensor assembly, the limit switch assembly, and the chain saw assembly respectively; the control device being used to: obtain image data of the branches and leaves to be pruned; and perform feature extraction and three-dimensional reconstruction based on the image data to obtain a three-dimensional point cloud image of the branches and leaves to be pruned; Based on the three-dimensional point cloud image of the branches and leaves to be pruned, a path planning model is used to predict a pruning path; based on the predicted pruning path, the actuator is controlled to move the actuator to the working position, and the branches and leaves to be pruned are clamped by the chain saw assembly in the actuator and the support member; based on the pruning instruction, the chain saw assembly is controlled to perform the pruning action until the second limit switch is triggered, and the chain saw assembly is controlled to stop moving; the chain saw assembly is controlled to rotate until the first limit switch is triggered and stops rotating; wherein, based on the three-dimensional point cloud image of the branches and leaves to be pruned, a path planning model is used to predict the pruning path, including: based on the three-dimensional point cloud image of the branches and leaves to be pruned, an improved RRT model is used to generate a three-dimensional point cloud image pruning path; wherein, the improved RRT model obtains a current detection step size based on an initial detection step size and a detection step size proportional coefficient, and updates the current detection step size based on the current detection step size and a path boundary distance value; based on the determined mapping relationship between the manipulator coordinate system and the image sensor assembly coordinate system, the predicted pruning path corresponding to the three-dimensional point cloud image pruning path is determined; wherein, the predicted pruning path is the target path of the actuator operation; The actuator also includes a laser radar, which is arranged in the chain saw assembly and is used to obtain contour data of the branches and leaves to be pruned and environmental data of the branches and leaves to be pruned; the control device is connected to the laser radar, and the control device is also used to: generate three-dimensional point cloud data based on the contour data of the branches and leaves to be pruned and the environmental data of the branches and leaves to be pruned; based on the mapping relationship between the image sensor assembly coordinate system and the three-dimensional point cloud data coordinate system, perform RGB coloring on the three-dimensional point cloud image of the pruned branches and leaves.

2. The tree branch pruning tool according to claim 1, characterized in that: The image sensor assembly includes a first image sensor and a second image sensor, the first image sensor and the second image sensor are respectively located on both sides of the chain saw assembly, and the control device is respectively connected to the first image sensor and the second image sensor; the control device is further used to: Acquire first image data captured by the first image sensor and a second feature image captured by the second image sensor; Performing feature extraction based on the first image data and the second feature image to obtain a first feature image corresponding to the first image data and second feature data corresponding to the second feature image; Performing three-dimensional reconstruction based on the first characteristic image and the second characteristic image to obtain a first reconstructed image corresponding to the first characteristic image and a second reconstructed image corresponding to the second characteristic image; Matching processing is performed based on the first reconstructed image and the second reconstructed image to obtain a three-dimensional point cloud image of the branches and leaves to be pruned.

3. The tree branch pruning tool according to claim 1, characterized in that: The actuator further comprises a shock absorbing assembly, which is arranged at one end of the bracket away from the chain saw assembly and connected to the bracket assembly.

4. The tree branch pruning tool according to claim 1 or 3, characterized in that: The actuator further includes a connecting assembly, which is used to install the actuator on a robotic arm. The connecting assembly includes a connecting column and a connecting piece, and the connecting piece is fixedly connected to an end of the bracket away from the chain saw assembly through the connecting column.

5. The tree branch pruning tool according to claim 1, characterized in that: The control device is also used for: When the actuator moves to the working position, the pruning instruction is forwarded to the user client, so that the user client performs pruning determination based on the positional relationship between the actuator in the pruning instruction and the three-dimensional point cloud image of the branches and leaves to be pruned; When receiving the trimming confirmation instruction returned by the user client, the chain saw assembly is controlled to perform the trimming action.

6. A method for pruning tree branches, characterized in that: A control device applied to a tree branch pruning tool according to any one of claims 1 to 5, wherein the method comprises: Obtain image data of branches and leaves to be pruned; Perform feature extraction and three-dimensional reconstruction based on the image data to obtain a three-dimensional point cloud image of the branches and leaves to be pruned; Based on the three-dimensional point cloud image of the branches and leaves to be pruned, a path planning model is used to predict a pruning path; Based on the predicted pruning path, controlling the actuator to move so as to clamp the branches and leaves to be pruned through the chain saw assembly and the support member; Based on the trimming instruction, controlling the chain saw assembly to perform a trimming action until the second limit switch is triggered, and then controlling the chain saw assembly to stop the action; The chain saw assembly is controlled to rotate until the first limit switch is triggered and the chain saw assembly stops rotating.

7. A control device, characterized in that: The system comprises a processor and a memory, wherein the memory stores computer-executable instructions that can be executed by the processor, and the processor executes the computer-executable instructions to implement the tree branch pruning method according to claim 6.

8. A computer-readable storage medium, characterized in that The computer-readable storage medium stores computer-executable instructions, which, when called and executed by a processor, implement the tree branch pruning method according to claim 6.

Images