Method for positioning and removing secondary buds of wine grape

Abstract

The application discloses a kind of wine grape auxiliary bud positioning and removal methods, comprising: main controller, depth camera, movable chassis and actuator, the main controller is fixed on the mounting panel on the top of movable chassis with actuator through mounting hole;The depth camera is fixed to the end of actuator.This wine grape auxiliary bud positioning and removal method can accurately identify and classify the main bud and auxiliary bud in twin bud, effectively improve the identification and positioning accuracy of auxiliary bud, can fuse multiple sensor data and interact data, effectively improve the efficiency of user for the acquisition and command release of actuator state, can perceive complex environment and plan corresponding execution path, effectively improve the obstacle avoidance ability of actuator, improve the safety of actuator operation, can adaptively adjust the posture of actuator for dynamic environment, effectively improve the efficiency of actuator operation, enhance the robustness of system.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of fruit and vegetable flower and fruit thinning, in particular to a wine grape secondary bud positioning and removal method. BACKGROUND

[0002] Grapes are the raw materials for wine making, and during the growth process of the vines, the buds that develop abnormally, the buds that grow in inappropriate positions and the buds that grow in excessive density need to be removed, and finally one bud with good growth conditions is retained. The companion bud, also known as the twin bud, is the most common companion trait during grape budding, which is usually divided into a main bud and a secondary bud. When the main bud appears inflorescence, the existence of the secondary bud will rob the nutrients of the main bud, so the secondary bud needs to be removed.

[0003] The existing grape bud removal device is divided into mechanical and spraying types. The mechanical bud removal device and method (patent publication number: CN102165897A, the patent right has been terminated in 2015) uses a dial and a cutter disc to remove grape buds, but this method does not use an optical method to distinguish between main buds and secondary buds, and can only remove sharp buds and buds that develop abnormally, which has poor precision. Although it can alleviate the labor intensity of orchard managers during the bud removal period, the yield increase effect is not good (both main buds and secondary buds are removed). The spraying type bud removal device mainly uses spraying of inhibitory agents to inhibit the growth of buds, which is usually sprayed by manual backpack, and a few by machine. The same problem exists that the growth of main buds and secondary buds is inhibited at the same time, which is not conducive to the thinning of flowers and fruits in the orchard to increase yield. It is only suitable for winter bud removal in some areas, and pruning operation is generally used for winter bud removal in Guanzhong area, so it cannot be applied.

[0004] The above two existing technologies do not accurately distinguish between main buds and secondary buds, and the operation area is large, which leads to low bud removal precision, which is not conducive to the subsequent yield increase of grapes. SUMMARY

[0005] The purpose of the present application is to provide a wine grape secondary bud positioning and removal method. Through RGB-D means, ROS control, deep learning algorithm and path planning technology, the main bud and the secondary bud in the twin bud of spring wine grape are distinguished and accurately positioned, which helps the bud removal equipment to determine the operation range, operation target and operation path. While efficiently and accurately positioning the secondary buds of spring grape, the adaptive adjustment of the actuator posture can be performed according to the dynamic environment, and the corresponding execution trajectory can be planned to improve the accuracy and efficiency of the execution mechanism to remove the buds, so as to solve the problems raised in the above background technology.

[0006] In order to achieve the above object, the present application provides the following technical scheme: a wine grape auxiliary bud positioning and removing method, comprising: a main controller, a depth camera, a movable chassis and an execution mechanism, the main controller and the execution mechanism are fixed on the mounting panel on the top of the movable chassis through the mounting hole; and the depth camera is fixed on the tail end of the execution mechanism.

[0007] Preferably, a wine grape auxiliary bud positioning and removing method comprises the following steps:

[0008] S1: the program starts, loads the URDF model file of the main controller, the depth camera, the execution mechanism and the movable chassis, configures the collision matrix and the inertia matrix, and runs each functional node;

[0009] S2: the target pose acquisition node is run, the first twin bud recognition model is loaded, the first twin bud recognition is performed, and the movable chassis slowly moves forward;

[0010] S3: if the twin bud is detected, the movable chassis stops running; otherwise, the twin bud recognition is continuously performed;

[0011] S4: the second twin bud recognition model is loaded, and the second twin bud recognition is performed;

[0012] S5: if the detection time does not exceed the recognition time threshold, the twin bud recognition is continuously performed; otherwise, the recognition is ended, and the program is restarted;

[0013] S6: if the position error of the twin bud centroid three-dimensional coordinates recognized by the movable chassis is less than the threshold value for 4 consecutive frames, the recognition is ended; otherwise, the second twin bud recognition is restarted;

[0014] S7: the twin bud centroid three-dimensional coordinates of the 4 consecutive frames are subjected to mean value processing, and are published to the coordinate monitoring platform in the form of the relative depth camera coordinate system;

[0015] S8: the relative relationship between the twin bud coordinate system and the execution mechanism coordinate system is compared, the quaternion expression of the coordinate system conversion is obtained, and the expression is published to the path planning platform;

[0016] S9: the execution mechanism motion planning group subscribes the quaternion expression of the twin bud and the execution mechanism, and performs path planning;

[0017] S10: if there is an executable path, the coordinates of each waypoint in the position space are published, and the coordinates are converted in the joint space, and the state of the execution mechanism at each waypoint; otherwise, the path planning is ended, and the program is restarted;

[0018] S11: the main controller controls the execution mechanism to run to the position 300mm in front of the twin bud along the planned path at the set speed and pose;

[0019] S12: Load the first secondary bud recognition model and perform the first secondary bud recognition;

[0020] S13: If a secondary bud is detected, the identification process ends; otherwise, the secondary bud identification process continues.

[0021] S14: Load the second bud identification model and perform the second bud identification;

[0022] S15: If the detection time does not exceed the recognition time threshold, continue to identify secondary buds; otherwise, end the recognition and the program will restart.

[0023] S16: If the position error of the identified secondary bud centroid three-dimensional coordinates is less than the threshold for four consecutive frames, the identification ends; otherwise, the secondary bud identification is repeated for the second time.

[0024] S17: The three-dimensional coordinates of the centroid of the secondary bud in four consecutive frames are averaged and published to the coordinate monitoring platform in the form of a relative depth camera coordinate system.

[0025] S18: Compare the relative relationship between the bud coordinate system and the actuator coordinate system, obtain the quaternion expression for coordinate system transformation, and publish the expression to the path planning platform;

[0026] S19: The actuator motion planning group subscribes to the quaternion expressions of the secondary buds and the actuators to perform path planning;

[0027] S20: If an executable path exists, publish the coordinates of each waypoint in the location space and convert them into the state of the actuator at each waypoint in the joint space; otherwise, end the path planning and the program restarts.

[0028] S21: The main controller controls the actuator to run along the planned path, at the set speed and position, to 200mm in front of the secondary bud;

[0029] S22: Based on the three-dimensional coordinates of the secondary bud, set three different poses for the actuator to identify the secondary bud, obtain the quaternion expressions of the transformation of the three poses relative to the current actuator coordinate system, and publish the expressions to the path planning platform;

[0030] S23: The actuator motion planning group subscribes to the quaternion expressions of three poses and the actuator to perform path planning;

[0031] S24: If an executable path exists, publish the coordinates of each waypoint in the location space and transform them into the state of the actuator at each waypoint in the joint space; otherwise, end the path planning.

[0032] S25: the main controller controls the actuator to run to the specified pose along the planned path at the set speed and pose;

[0033] S26: a third secondary bud recognition model is loaded to perform a third secondary bud recognition, and the confidence of the recognized secondary bud at the current pose is recorded;

[0034] S27: the confidence of the recognized secondary bud at the three poses is compared to determine the pose with the maximum confidence;

[0035] S28: the main controller controls the actuator to run to the pose with the maximum confidence along the planned path at the set speed and pose;

[0036] S29: the actuator performs the bud wiping operation;

[0037] S30: the actuator returns to the initial pose and performs calibration;

[0038] S31: the program ends running, releases the memory, and enters the standby state.

[0039] Preferably, the running nodes in step S1 include a state listening node, a state publishing node, a motion path planning node, a motion path execution node, a target pose obtaining node, a twin bud pose publishing node, a twin bud pose conversion node, a secondary bud pose publishing node, a secondary bud pose conversion node, and a motion group state updating node.

[0040] Further preferably, the state listening node is real_listener, the state publishing node is robot_state_publisher, the motion path planning node is move_group, the motion path execution node is my_action_server, the target pose obtaining node is main, the twin bud pose publishing node is BUD_01_Publish_TF, the twin bud pose conversion node is BUD_02_Transform_TF, the secondary bud pose publishing node is BUD_SEC_01_Publish_TF, the secondary bud pose conversion node is BUD_SEC_02_Transform_TF, and the motion group state updating node is move_group_commander_wrappers.

[0041] Preferably, the pose confidence value at which the path planning ends in step S24 is 0.

[0042] Compared with the prior art, the present application has the following beneficial effects:

[0043] 1. The wine grape secondary bud positioning and removal method uses RGB-D technology to accurately identify and classify the main bud and secondary bud in the twin bud, effectively improving the positioning accuracy of the secondary bud.

[0044] 2. The wine grape secondary bud positioning and removal method uses the ROS operating system to integrate multi-sensor data and perform data interaction, effectively improving the efficiency of user access to the state of the execution mechanism and command issuance.

[0045] 3. The wine grape secondary bud positioning and removal method uses path planning technology to perceive complex environments and plan corresponding execution paths, effectively improving the obstacle avoidance ability of the execution mechanism and improving the safety of the execution mechanism during operation.

[0046] 4. The wine grape secondary bud positioning and removal method uses a nonlinear control system to adaptively adjust the execution mechanism posture in dynamic environments, effectively improving the efficiency of the execution mechanism operation and enhancing the robustness of the system. BRIEF DESCRIPTION OF DRAWINGS

[0047] Figure 1 The wine grape secondary bud positioning and removal method flowchart of the present application;

[0048] Figure 2 The wine grape secondary bud positioning and removal method communication node diagram of the present application. DETAILED DESCRIPTION

[0049] The technical solutions in the embodiments of the present application will be described in detail below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0050] Embodiment one: The present application provides a technical solution: a wine grape secondary bud positioning and removal method, comprising: a main controller, a depth camera, a movable chassis and an execution mechanism, the main controller and the execution mechanism are fixed on the mounting panel on the top of the movable chassis through the mounting hole; the depth camera is fixed at the end of the execution mechanism.

[0051] Embodiment two: The positioning process of the grape secondary bud is realized by the following steps:

[0052] S1: the program starts, loads the URDF model file of the main controller, the depth camera, the actuator and the movable chassis, configures the collision matrix and the inertia matrix, and runs the function nodes, which include: state monitoring node, state publishing node, motion path planning node, motion path execution node, target pose acquisition node, twin bud pose publishing node, twin bud pose conversion node, secondary bud pose publishing node, secondary bud pose conversion node and motion group state updating node;

[0053] S2: run the target pose acquisition node, load the first twin bud recognition model, and perform the first twin bud recognition. The movable chassis moves slowly forward;

[0054] S3: if the twin bud is detected, the movable chassis stops running; otherwise, continue to perform twin bud recognition;

[0055] S4: load the second twin bud recognition model and perform the second twin bud recognition;

[0056] S5: if the detection time does not exceed the recognition time threshold, continue to perform twin bud recognition; otherwise, end the recognition and the program starts running again;

[0057] S6: if the center of the recognized twin bud three-dimensional coordinates, the position error of the continuous 4 frames is less than the threshold, then end the recognition; otherwise, re-perform the second twin bud recognition;

[0058] S7: the obtained continuous 4 frames of twin bud center three-dimensional coordinates are processed by mean value, and are published in the form of relative depth camera coordinate system to the coordinate monitoring platform;

[0059] S8: compare the relative relationship between the twin bud coordinate system and the actuator coordinate system, obtain the quaternion expression of the coordinate system conversion, and publish the expression to the path planning platform;

[0060] S9: the actuator motion planning group subscribes the quaternion expression of the twin bud and the actuator, and performs path planning;

[0061] S10: if there is an executable path, publish the coordinates of each waypoint in the position space, and convert it in the joint space, the state of the actuator at each waypoint; otherwise, end the path planning and the program starts running again;

[0062] S11: the main controller controls the actuator to run along the planned path to the front 300mm of the twin bud according to the set speed and pose;

[0063] S12: load the first secondary bud recognition model and perform the first secondary bud recognition;

[0064] S13: if the secondary bud is detected, end the recognition; otherwise, continue to perform secondary bud recognition;

[0065] S14: loading a second secondary bud recognition model to perform secondary bud recognition for the second time;

[0066] S15: if the detection duration does not exceed the recognition time threshold, continue to perform secondary bud recognition; otherwise, end the recognition and the program starts to run again;

[0067] S16: if the position error of the recognized three-dimensional coordinates of the secondary bud heart in four consecutive frames is less than the threshold, end the recognition; otherwise, perform secondary bud recognition for the second time again;

[0068] S17: performing mean processing on the obtained three-dimensional coordinates of the secondary bud heart in four consecutive frames, and publishing the three-dimensional coordinates in the form of a relative depth camera coordinate system to a coordinate monitoring platform;

[0069] S18: comparing the relative relationship between the secondary bud coordinate system and the actuator coordinate system to obtain a quaternion expression of the coordinate system conversion, and publishing the expression to a path planning platform;

[0070] S19: the actuator motion planning group subscribes to the quaternion expression of the secondary bud and the actuator to perform path planning;

[0071] S20: if there is an executable path, publishing the coordinates of each waypoint in the position space and converting it to the state of the actuator at each waypoint in the joint space; otherwise, end the path planning and the program starts to run again;

[0072] S21: the main controller controls the actuator to run to a position 200 mm in front of the secondary bud along the planned path at a set speed and pose;

[0073] S22: setting three different poses of the actuator for secondary bud recognition according to the three-dimensional coordinates of the secondary bud, obtaining a quaternion expression of the coordinate system conversion of the three poses relative to the current actuator coordinate system, and publishing the expression to a path planning platform;

[0074] S23: the actuator motion planning group subscribes to the quaternion expression of the three poses and the actuator to perform path planning;

[0075] S24: if there is an executable path, publishing the coordinates of each waypoint in the position space and converting it to the state of the actuator at each waypoint in the joint space; otherwise, ending the path planning and the pose confidence value of the path planning is 0;

[0076] S25: the main controller controls the actuator to run to a specified pose along the planned path at a set speed and pose;

[0077] S26: load the third auxiliary bud recognition model, perform the third auxiliary bud recognition, and record the confidence of the recognized auxiliary bud in the current pose;

[0078] S27: compare the confidence of the recognized auxiliary bud in the three poses, and determine the pose with the maximum confidence;

[0079] S28: the main controller controls the execution mechanism to run to the pose with the maximum confidence along the planned path and at the set speed and pose;

[0080] S29: the execution mechanism performs the bud wiping operation;

[0081] S30: the execution mechanism returns to the initial pose and performs calibration;

[0082] S31: the program ends running, releases the memory, and enters the standby state.

[0083] Although the embodiments of the present application have been shown and described, it is to be understood that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the present application, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A device for positioning and removing of secondary buds of wine grapes, comprising: The main controller, the depth camera, the movable chassis and the actuator are characterized in that: the main controller and the actuator are fixed on the mounting panel on the top of the movable chassis through the mounting hole; and the depth camera is fixed on the end of the actuator. A method for positioning and removing secondary buds of wine grapes, comprising the following steps: S1: the program starts, loads the URDF model file of the main controller, the depth camera, the actuator and the movable chassis, configures the collision matrix and the inertia matrix, and runs each functional node; S2: the target pose acquisition node is run, the first twin bud recognition model is loaded, the first twin bud recognition is performed, and the movable chassis slowly moves forward; S3: if the twin bud is detected, the movable chassis stops running; otherwise, the twin bud recognition is continuously performed; S4: the second twin bud recognition model is loaded, and the second twin bud recognition is performed; S5: if the detection time does not exceed the recognition time threshold, the twin bud recognition is continuously performed; otherwise, the recognition is ended, and the program is restarted; S6: if the position error of the recognized twin bud centroid three-dimensional coordinates is less than the threshold for four consecutive frames, the recognition is ended; otherwise, the second twin bud recognition is performed again; S7: the mean value of the obtained twin bud centroid three-dimensional coordinates for four consecutive frames is processed, and the processed result is published to the coordinate monitoring platform in the form of the relative depth camera coordinate system; S8: the relative relationship between the twin bud coordinate system and the actuator coordinate system is compared, the quaternion expression of the coordinate system conversion is obtained, and the expression is published to the path planning platform; S9: the actuator motion planning group subscribes to the quaternion expression of the twin bud and the actuator, and performs path planning; S10: if there is an executable path, the coordinates of each waypoint in the position space are published, the coordinates are converted in the joint space, and the state of the actuator at each waypoint is obtained; otherwise, the path planning is ended, and the program is restarted; S11: the main controller controls the actuator to run to the position 300mm in front of the twin bud along the planned path at the set speed and pose; S12: the first secondary bud recognition model is loaded, and the first secondary bud recognition is performed; S13: if the secondary bud is detected, the recognition is ended; otherwise, the secondary bud recognition is continuously performed; S14: the second secondary bud recognition model is loaded, and the second secondary bud recognition is performed; S15: if the detection time does not exceed the recognition time threshold, the secondary bud recognition is continuously performed; otherwise, the recognition is ended, and the program is restarted; S16: if the position error of the recognized secondary bud centroid three-dimensional coordinates is less than the threshold for four consecutive frames, the recognition is ended; otherwise, the second secondary bud recognition is performed again; S17: the mean value of the obtained secondary bud centroid three-dimensional coordinates for four consecutive frames is processed, and the processed result is published to the coordinate monitoring platform in the form of the relative depth camera coordinate system; S18: the relative relationship between the secondary bud coordinate system and the actuator coordinate system is compared, the quaternion expression of the coordinate system conversion is obtained, and the expression is published to the path planning platform; S19: the actuator motion planning group subscribes to the quaternion expression of the secondary bud and the actuator, and performs path planning; and S20: If there is an executable path, publish the coordinates of each waypoint in the position space and convert it to the state of the actuator at each waypoint in the joint space; otherwise, end the path planning, and the program starts running again; S21: The main controller controls the actuator to run along the planned path to the front of the secondary bud by 200 mm at the set speed and pose S22: According to the three-dimensional coordinates of the secondary bud, set three different poses of the actuator for secondary bud recognition, obtain the quaternion expression of the three poses relative to the current actuator coordinate system, and publish the expression to the path planning platform; S23: The actuator motion planning group subscribes to the quaternion expression of the three poses and the actuator for path planning; S24: If there is an executable path, publish the coordinates of each waypoint in the position space and convert it to the state of the actuator at each waypoint in the joint space; otherwise, end the path planning; S25: The main controller controls the actuator to run along the planned path to the specified pose at the set speed and pose; S26: Load the third secondary bud recognition model to perform the third secondary bud recognition and record the confidence of the recognized secondary bud at the current pose; S27: Compare the confidence of the recognized secondary bud at the three poses to determine the pose with the maximum confidence; S28: The main controller controls the actuator to run along the planned path to the pose with the maximum confidence at the set speed and pose; S29: The actuator performs the bud wiping operation; S30: The actuator returns to the initial pose and performs calibration; S31: The program ends running, releases the memory, and enters the standby state.

2. A device for positioning and removing the secondary buds of wine grapes according to claim 1, characterized in that: The running of each functional node in step S1 includes: a state listening node, a state publishing node, a motion path planning node, a motion path execution node, a target pose obtaining node, a twin bud pose publishing node, a twin bud pose conversion node, a secondary bud pose publishing node, a secondary bud pose conversion node, and a motion group state updating node.

3. The positioning device for removing the side bud of wine grape according to claim 1, characterized in that: The pose confidence value for ending path planning in step S2 is 0.

Images